RE: [PATCH v4 0/2] Improve handling of ISAR_RELEASE_CMD

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

> From: Anton Mikanovich <amikan@ilbers.de>
> Sent: Thursday, December 2, 2021 3:55 PM
> To: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@siemens.com>;
> Schild, Henning (T RDA IOT SES-DE) <henning.schild@siemens.com>
> Cc: Schmidt, Adriaan (T RDA IOT SES-DE) <adriaan.schmidt@siemens.com>; isar-
> users@googlegroups.com; Baurzhan Ismagulov <ibr@ilbers.de>
> Subject: Re: [PATCH v4 0/2] Improve handling of ISAR_RELEASE_CMD
> 
> 30.11.2021 13:03, Moessbauer, Felix wrote:
> > That is what we expect. However something changes the git state during a
> build.
> > The state must ONLY be changed between builds (two bitbake invocations).
> > In case multiple tests are run in parallel in the same build-tree, this will break.
> > Even if the patches have not been fully tested in the CI, that's what we use in
> our daily business (it works).
> >
> > Unfortunately I have a really hard time to run the CI in the Siemens
> environment, as it often fails due to proxy issues or network timeouts.
> > Also running just the repro target itself does not work due to other issues.
> > I tried with: avocado run
> > testsuite/build_test/build_test.py:RebuildTest.test_rebuild
> >
> > [stdout] E: Unable to find a source package for
> > [stdlog] 2021-11-30 10:58:04,196 cibuilder        L0100 ERROR| E: Unable to
> find a source package for
> > [stdout] + bb_exit_handler
> > [stdlog] 2021-11-30 10:58:04,196 cibuilder        L0100 ERROR| +
> bb_exit_handler
> >
> > Anyways, if there are more issues to be ruled out for this patch series, I
> actually prefer to merge the SState series first.
> > Currently, a lot of patches are waiting for SState, as Henning mentioned.
> >
> > Felix
> 
> Ok, there are easy steps to reproduce the issue with kas:
> 
> $ cat isar.yml
> header:
>    version: 10
> 
> distro: debian-bullseye
> machine: qemuamd64
> target: mc:qemuamd64-bullseye:isar-image-base
> repos:
>    isar:
>      url: "git@github.com:/ilbers/isar"
>      refspec: "4dc41fba8d664fbc00ec56640fec066c9e57b19d"
>      layers:
>        meta:
>        meta-isar:
> 
> bblayers_conf_header:
>    standard: |
>      BBPATH = "${TOPDIR}"
>      BBFILES ?= ""
> 
> $ time ~/kas/kas-docker --isar build isar.yml $ echo -e "do_fetch_append()
> {\n\n}" >> isar/meta/classes/dpkg-base.bbclass
> $ time ~/kas/kas-docker --isar build isar.yml
> 
> On the first obtain of IMAGE_BUILD_ID it gets correct 'dirty' value, but compare
> it with old hash from the previous build at fails.

Ok thanks for the reproducer. By that I was able to navigate bitbake into the error.

Unfortunately this problem is not easy to solve, as it is simply not allowed to modify the metadata on the fly.
By that, we cannot use an anonymous python function to taint a task depending on a condition.
If we want up-to-date versioning information, we would have to use a "nostamp" task (but that makes rootfs caching irrelevant)...

A better solution would be to reorder the tasks to make sure that the sstate caches the rootfs without /etc/os-release information.
This at least restores the previous behavior (re-building without change does not re-run ISAR_RELEASE_CMD).
Without these changes, probably a lot of CI generated images that use a persistent sstate will end up with outdated version information.
@Adriaan: What's your opinion?

> 
> Even if the patchset is working for upstream it should not introduce such a
> regression.

Agreed!

Patch 2 of this series should be fine (just a bugfix).

Felix

> 
> --
> Anton Mikanovich
> Promwad Ltd.
> External service provider of ilbers GmbH Maria-Merian-Str. 8
> 85521 Ottobrunn, Germany
> +49 (89) 122 67 24-0
> Commercial register Munich, HRB 214197
> General Manager: Baurzhan Ismagulov