Re: [PATCH 4/6] Enable proper rebuilds on dependency changes

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/13/2018 10:22 PM, Jan Kiszka wrote:
> On 2018-02-13 20:02, Alexander Smirnov wrote:
>>
>> On 02/13/2018 09:44 PM, Jan Kiszka wrote:
>>> On 2018-02-13 19:08, Alexander Smirnov wrote:
>>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>>>>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>>
>>>>>>>> Install a basichash as signature handler and set the stamp policy to
>>>>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>>>>> affected
>>>>>>>> packages when some recipe down the dependency chain changed!
>>>>>>>>
>>>>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>>>>> setscene machinery like OE. Still good enough for us.
>>>>>>>>
>>>>>>>
>>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>>>>> didn't work properly, not all the dependents were rebuilt. If it
>>>>>>> works,
>>>>>>> it's a very good feature!
>>>>>>
>>>>>> Yes, I'm using it since this weekend for both Isar as well as
>>>>>> jailhouse-images development, and it helped a lot already. The only
>>>>>> limitation I found, but I do not remember right now if that isn't
>>>>>> inherent to bitbake, is that it does not detect changes in files that
>>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>>>>> clean" for the affected target - which now works as well.
>>>>>>
>>>>>
>>>>> Found another issue, which worked surprisingly well so far despite
>>>>> being
>>>>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>>>>
>>>> Investigating exactly this issue. It occures for clean builds also. My
>>>> guess is patch #3, currently I'm building with only 2 first applied.
>>>> With 3rd one applied build failed.
>>>>
>>>
>>> That's not the right approach, the concept of the setup_mounts task is
>>> broken.
>>
>> Why you think it's broken? Just tested, it works good:
> 
> It's broken in its design: you stamp a task and then remove that stamp
> again on restarts. That only works because we didn't generate the full
> dependency chain. Once you do, all dependent tasks we be kicked for
> re-run. Removing patch 3 is voodoo debugging. I bet you can't explain
> why that works around the issue, and I'm sure it will break eventually
> again. Moreover, it's needlessly complex.

CI was green, I apply this series, CI became red, I've reset the head 
patch by patch to get CI green again. After reset patch #2 CI became 
green. Which voodoo you found here? This series brakes the build without 
re-run.

> 
> The much simpler solution is to try-mount when needed, at user side.
> Patch will follow, likely by resending the whole series

Mount when needed like you proposed could lead to lots of mounts on the 
same folder, because do_build works in parallel.

The correct way in this case would be:

  - keep do_mounts
  - use [nostamp] = 1
  - use [lockfiles] = "${BUILDCHROOT_DIR}/lock"
  - mount folders like you proposed:
+    grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts &&
+        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} 
${BUILDCHROOT_DIR}/isar-apt || true

Alex

> 
> Jan
> 

-- 
With best regards,
Alexander Smirnov

ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov