Re: isar-bootstrap fails to rebuild after config changes

[Jan Kiszka <jan.kiszka@siemens.com>]

On 13.08.19 10:01, Claudius Heine wrote:
> Hi Jan,
> 
> On 12/08/2019 19.18, [ext] Jan Kiszka wrote:
>> Hi,
>>
>> I noticed today that I'm not getting a new bootstrap rootfs after adding a 
>> repo to DISTRO_APT_SOURCES. Digging deeper, this revealed missing STAMPCLEAN 
>> setting, but also more:
>>
>> Why are we probing for the existence of the DEPLOY_ISAR_BOOTSTRAP link in 
>> do_apt_config_prepare(), Claudius? This obviously exists after a first run, 
>> and that blocks refreshing all the apt settings if they have changed.
> 
> I do not know. That was added by a patch of Maxim 
> (ab0a1c8c357a18ddd67fe4c0efc66a296d93dae2), maybe he knows.
> 
>      isar-bootstrap: Eliminate multiple debootstraps for the same distro/host
> 
>      After applying this patch only single debootstrap for
>      particular platform is created and pointed by corresponding
>      link under DEPLOY_DIR_IMAGE.
> 
>      For proper locking of parallel builds shell tasks
>      apt_config_install, set_locale, setup_chroot, apt_update
>      and deploy were collapsed to common debootstrap helper.
>      Due to problems in bitbake with shell functions expansion
>      under quotes these functions were substituted  by their
>      bodies.
> 
>      Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
> 
> That is by the way a pretty big patch. Also parts of it could probably be 
> reverted, since the 'problems in bitbake with shell functions expansion under 
> quotes' are resolved in the current version by removing these quotes IIUC.
> 
>> There is CLEANFUNCS set with clean_deploy(), but that is only run on true 
>> do_clean(), not on rebuilds.
> 
> Right.
> 
>>
>> And finally, when I look at isar_bootstrap(), run by do_bootstrap(), it 
>> protects checking and creating DEPLOY_ISAR_BOOTSTRAP with a lock. But 
>> do_apt_config_prepare() runs lockless. This cannot work reliably.
> 
> When I initially wrote isar-bootstrap, it didn't contain any looks. IIRC the 
> locks where added when I wasn't active with isar and I haven't yet looked into 
> why exactly they are needed and what they protect against etc. So you know 
> probably more about them than me.

Ah, sorry, wrongly accounted you for this.

Maxim, can you comment on the design idea and the issues I found?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux