Re: [PATCH 4/6] Enable proper rebuilds on dependency changes

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-13 20:38, Alexander Smirnov wrote:
> 
> 
> On 02/13/2018 10:22 PM, Jan Kiszka wrote:
>> On 2018-02-13 20:02, Alexander Smirnov wrote:
>>>
>>> On 02/13/2018 09:44 PM, Jan Kiszka wrote:
>>>> On 2018-02-13 19:08, Alexander Smirnov wrote:
>>>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>>>>>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>>>
>>>>>>>>> Install a basichash as signature handler and set the stamp
>>>>>>>>> policy to
>>>>>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>>>>>> affected
>>>>>>>>> packages when some recipe down the dependency chain changed!
>>>>>>>>>
>>>>>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>>>>>> setscene machinery like OE. Still good enough for us.
>>>>>>>>>
>>>>>>>>
>>>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>>>>>> didn't work properly, not all the dependents were rebuilt. If it
>>>>>>>> works,
>>>>>>>> it's a very good feature!
>>>>>>>
>>>>>>> Yes, I'm using it since this weekend for both Isar as well as
>>>>>>> jailhouse-images development, and it helped a lot already. The only
>>>>>>> limitation I found, but I do not remember right now if that isn't
>>>>>>> inherent to bitbake, is that it does not detect changes in files
>>>>>>> that
>>>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>>>>>> clean" for the affected target - which now works as well.
>>>>>>>
>>>>>>
>>>>>> Found another issue, which worked surprisingly well so far despite
>>>>>> being
>>>>>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>>>>>
>>>>> Investigating exactly this issue. It occures for clean builds also. My
>>>>> guess is patch #3, currently I'm building with only 2 first applied.
>>>>> With 3rd one applied build failed.
>>>>>
>>>>
>>>> That's not the right approach, the concept of the setup_mounts task is
>>>> broken.
>>>
>>> Why you think it's broken? Just tested, it works good:
>>
>> It's broken in its design: you stamp a task and then remove that stamp
>> again on restarts. That only works because we didn't generate the full
>> dependency chain. Once you do, all dependent tasks we be kicked for
>> re-run. Removing patch 3 is voodoo debugging. I bet you can't explain
>> why that works around the issue, and I'm sure it will break eventually
>> again. Moreover, it's needlessly complex.
> 
> CI was green, I apply this series, CI became red, I've reset the head
> patch by patch to get CI green again. After reset patch #2 CI became
> green. Which voodoo you found here? This series brakes the build without
> re-run.
> 
>>
>> The much simpler solution is to try-mount when needed, at user side.
>> Patch will follow, likely by resending the whole series
> 
> Mount when needed like you proposed could lead to lots of mounts on the
> same folder, because do_build works in parallel.

All solved, in a single line. Hold your breath, it's getting better.

Jan