Re: [PATCH v5 00/10] Add SBOM generation with debsbom

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Thu, 2025-11-27 at 15:35 +0100, Zhihang Wei wrote:
> Hi,
> A CI failure occured in full test suite:
> citest.py:CustomizationsTest.test_single_customization
> 
> Log follows:
> builder@13751127e9ed:/work/testsuite$ avocado run 
> citest.py:CustomizationsTest.test_single_customization
> JOB ID     : 5a0c694b516485d29feccb614eb23dddb0b667af
> JOB LOG    : 
> /tmp/tmpo3rzdofv/avocado/job-results/job-2025-11-27T10.47-5a0c694/job.log
>   (1/1) citest.py:CustomizationsTest.test_single_customization: STARTED
> ERROR: mc:qemuamd64-bullseye:isar-image-ci-1.0-r0 do_image_tar: 
> ExecutionError('/work/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_tar.8314', 
> 1, None, None)
> ERROR: Logfile of failure stored in: 
> /work/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_tar.8314
> ERROR: Task 
> (mc:qemuamd64-bullseye:/work/meta-test/recipes-core/images/isar-image-ci.bb:do_image_tar) 
> failed with exit code '1'
> ERROR: mc:qemuamd64-bullseye:isar-image-ci-1.0-r0 do_image_cpio: 
> ExecutionError('/work/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_cpio.8315', 
> 1, None, None)
> ERROR: Logfile of failure stored in: 
> /work/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_cpio.8315
> ERROR: Task 
> (mc:qemuamd64-bullseye:/work/meta-test/recipes-core/images/isar-image-ci.bb:do_image_cpio) 
> failed with exit code '1'
> ERROR: mc:qemuamd64-bullseye:isar-image-ci-1.0-r0 do_image_wic: 
> ExecutionError('/work/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.8316', 
> 1, None, None)
> ERROR: Logfile of failure stored in: 
> /work/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/log.do_image_wic.8316
> 
> In all of these three failed tasks, error log includes:
> dpkg-query: no packages found matching grub-efi-amd64-bin
> 
> 
> To redo the test using avocado:
> 1. Have a clean clone of isar, checkout to branch next and apply your 
> patches:
> $ git clone -b next https://github.com/ilbers/isar.git
> $ cd isar
> $ git am /path-to/0001-my-contribution-to-isar.patch
> 2. Run kas shell, setup CI prerequisites (avocado, qemu) and cleanup:
> $ ./kas/kas-container shell kas/isar.yaml --command \
>      "rm -rf /work/build/conf && /work/scripts/ci_setup.sh"
> 4.Run the failed test in fast:
> $ cd /work/testsuite
> $ avocado run citest.py:CustomizationsTest.test_single_customization$

Hi, I was scratching my head how this could fail, but well ... it is a
badly written test:

In isar-image-ci [1], we do a ton conditional IMAGER_INSTALL:append and
IMAGER_INSTALL:remove, which are not aligned at all with the machine
configs.

Please add the following line to that section:

IMAGER_BOM:remove:qemuamd64:debian-bullseye ?=
"${GRUB_BOOTLOADER_INSTALL}"

This then should be added to p8. If you confirm that this works, I can
also send out a new version of the sbom series.

Anyways, I would be really happy if we could get rid of machine
manipulation parts in an image recipe. This is IMHO an anti-pattern.

[1]https://github.com/ilbers/isar/blob/master/meta-test/recipes-core/images/isar-image-ci.bb#L25

Felix

> 
> Zhihang
> 
> On 11/24/25 12:46, 'Felix Moessbauer' via isar-users wrote:
> > This patchset adds proper SBOM generation in the two standard formats
> > SPDX and CycloneDX during the rootfs generation process.
> > 
> > The generation is itself is handled by a SBOM generator  `debsbom` [1]
> > which is developed as an open source project at Siemens. It is still
> > early in development, but it has enough features for what we require
> > in isar. The required dependencies which are not yet available as
> > Debian packages were minimally packaged directly in isar too.
> > 
> > This is a followup of the previous RFC [2]. Since then the series has
> > changed a lot. The SBOM generation was moved from a simple OE lib to
> > `debsbom`. This also meant the introduction of a separate chroot was
> > necessary. The SBOM generation process was also moved from the image
> > step to the rootfs step, along with a lot of minor changes and
> > improvements.
> > 
> > [1] https://github.com/siemens/debsbom
> > [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ
> > 
> > Changes since v4:
> > 
> > - rebased onto next
> > - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE})
> > 
> > Changes since v3:
> > 
> > - fix issue on external bullseye initramfs (we now disable sbom generation
> >    on all unsupported distros rootfs instances)
> > - update debsbom to v0.4.0
> > - rebased onto next
> > 
> > Changes since v2:
> > 
> > - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions
> > - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2
> > - generate SBOM for imager as well and create merged sbom of .wic image
> > - resend imager manifest + wic manifest patches to reduce conflicts
> > 
> > Note, that the patches p1-p5 are most important as they add basic SBOM
> > support. The remaining patches address the imager + .wic bom part,
> > which also can be merged later on.
> > 
> > Changes since v1:
> > 
> > - remove tarball
> > - refactor packaging (auto-derive python dependencies)
> > - only build missing packages (varies on bookworm, trixie, noble)
> > - add ubuntu support
> > - only generate sboms for supported distributions (bookworm/jammy and
> >    onwards)
> > - update debsbom (includes bug fixes and more information for source
> >    packages)
> > 
> > Christoph Steiger (3):
> >    meta: package python libraries for SBOM generation
> >    meta: package python3-debsbom
> >    meta: add SBOM generation with debsbom
> > 
> > Felix Moessbauer (7):
> >    refactor: move get_rootfs_distro from sdk into rootfs
> >    override distro vendor in SBOM on Ubuntu
> >    add support to add imager dependencies to BOM
> >    wic: create uniform manifest describing all image components
> >    qemuamd64: add IMAGER_BOM entries
> >    imager: create SBOM of IMAGER_BOM packages
> >    wic: create uniform SBOM describing all image components
> > 
> >   doc/user_manual.md                            |  1 +
> >   meta-isar/conf/distro/ubuntu-common.inc       |  2 +
> >   meta-isar/conf/machine/qemuamd64.conf         |  1 +
> >   meta/classes/image-tools-extension.bbclass    | 29 +++++++++
> >   meta/classes/image.bbclass                    |  7 ++
> >   meta/classes/imagetypes_wic.bbclass           | 30 +++++++++
> >   meta/classes/initramfs.bbclass                |  3 +-
> >   meta/classes/rootfs.bbclass                   | 23 ++++++-
> >   meta/classes/sbom.bbclass                     | 65 +++++++++++++++++++
> >   meta/classes/sdk.bbclass                      | 10 +--
> >   .../sbom-chroot/sbom-chroot.bb                | 30 +++++++++
> >   .../python3-beartype/files/rules              |  8 +++
> >   .../python3-beartype_0.19.0.bb                | 29 +++++++++
> >   .../files/pybuild.testfiles                   |  1 +
> >   .../python3-cyclonedx-lib/files/rules         |  8 +++
> >   .../python3-cyclonedx-lib_9.1.0.bb            | 48 ++++++++++++++
> >   ...icense-description-in-pyproject.toml.patch | 28 ++++++++
> >   .../python3-debsbom/files/rules               |  8 +++
> >   .../python3-debsbom/python3-debsbom_0.4.0.bb  | 45 +++++++++++++
> >   .../python3-packageurl/files/rules            |  8 +++
> >   .../python3-packageurl_0.16.0.bb              | 33 ++++++++++
> >   .../python3-py-serializable/files/rules       |  8 +++
> >   .../python3-py-serializable_2.0.0.bb          | 38 +++++++++++
> >   .../python3-spdx-tools/files/rules            | 25 +++++++
> >   .../python3-spdx-tools_0.8.3.bb               | 46 +++++++++++++
> >   25 files changed, 523 insertions(+), 11 deletions(-)
> >   create mode 100644 meta/classes/sbom.bbclass
> >   create mode 100644 meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
> >   create mode 100644 meta/recipes-support/python3-beartype/files/rules
> >   create mode 100644 meta/recipes-support/python3-beartype/python3-beartype_0.19.0.bb
> >   create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/pybuild.testfiles
> >   create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/rules
> >   create mode 100644 meta/recipes-support/python3-cyclonedx-lib/python3-cyclonedx-lib_9.1.0.bb
> >   create mode 100644 meta/recipes-support/python3-debsbom/files/0001-Use-old-license-description-in-pyproject.toml.patch
> >   create mode 100644 meta/recipes-support/python3-debsbom/files/rules
> >   create mode 100644 meta/recipes-support/python3-debsbom/python3-debsbom_0.4.0.bb
> >   create mode 100644 meta/recipes-support/python3-packageurl/files/rules
> >   create mode 100644 meta/recipes-support/python3-packageurl/python3-packageurl_0.16.0.bb
> >   create mode 100644 meta/recipes-support/python3-py-serializable/files/rules
> >   create mode 100644 meta/recipes-support/python3-py-serializable/python3-py-serializable_2.0.0.bb
> >   create mode 100644 meta/recipes-support/python3-spdx-tools/files/rules
> >   create mode 100644 meta/recipes-support/python3-spdx-tools/python3-spdx-tools_0.8.3.bb
> > 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/1e26bd54-623a-48f0-a400-34a454ba4993%40ilbers.de.

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/2b58f5057a9dffdb136c49959a387e7b1f0e5a60.camel%40siemens.com.