Re: [PATCH v2 0/4] Add SBOM generation with debsbom

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Mon, 2025-10-20 at 10:44 +0200, Zhihang Wei wrote:
> Hello,
> 
> When testing this patch set on full CI, the following test failed:
> - citest.py:ReproTest.test_repro_signed
> 
> Error log follows:
> [stdlog] 2025-10-07 12:02:43,784 avocado.app cibuilder L0322 ERROR| 
> ERROR: mc:qemuarm64-bookworm:isar-image-base-1.0-r0 
> do_rootfs_postprocess: 
> ExecutionError('/build/isar_wzh_devel_2/8/build/tmp/work/debian-bookworm-arm64/isar-image-base-qemuarm64/1.0-r0/temp/run.generate_sbom.2562126', 
> 255, None, None)
> [stdlog] 2025-10-07 12:02:43,788 avocado.app cibuilder L0322 ERROR| 
> ERROR: Logfile of failure stored in: 
> /build/isar_wzh_devel_2/8/build/tmp/work/debian-bookworm-arm64/isar-image-base-qemuarm64/1.0-r0/temp/log.do_rootfs_postprocess.2562126 
> 
> [stdlog] 2025-10-07 12:02:43,788 avocado.test cibuilder L0320 INFO | Log 
> data follows:
> [stdlog] 2025-10-07 12:02:43,789 avocado.app cibuilder L0322 ERROR| 
> ERROR: Task 
> (mc:qemuarm64-bookworm:/build/isar_wzh_devel_2/8/meta-isar/recipes-core/images/isar-image-base.bb:do_rootfs_postprocess) 
> failed with exit code '1'
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing python function do_rootfs_postprocess
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function rootfs_do_mounts
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function rootfs_do_mounts finished
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function rootfs_do_qemu
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function rootfs_do_qemu finished
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function image_posprocess_disable_systemd_firstboot
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> Created symlink /etc/systemd/system/systemd-firstboot.service -> /dev/null.
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function image_posprocess_disable_systemd_firstboot finished
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function image_postprocess_sshd_key_regen
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> find: 
> '/build/isar_wzh_devel_2/8/build/tmp/work/debian-bookworm-arm64/isar-image-base-qemuarm64/1.0-r0/rootfs/etc/ssh/': 
> No such file or directory
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function image_postprocess_sshd_key_regen finished
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function image_postprocess_machine_id
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> uninitialized
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function image_postprocess_machine_id finished
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function image_postprocess_mark
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> WARNING: You are using external layers that will not be considered in 
> the build_id. Consider changing ISAR_RELEASE_CMD.
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> BUILD_ID="v0.11-79-g7dcbb204"
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> VARIANT="Isar target filesystem"
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> VARIANT_VERSION="1.0"
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function image_postprocess_mark finished
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function image_postprocess_configure
> [stdlog] 2025-10-07 12:02:43,811 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function image_postprocess_configure finished
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing python function do_generate_sbom
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function generate_sbom
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> ERROR:debsbom.cli:'Origin'
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> debsbom: error: 'Origin'
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> WARNING: exit code 255 from a shell command.
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Python function do_generate_sbom finished
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Executing shell function rootfs_do_umounts
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Shell function rootfs_do_umounts finished
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | | 
> DEBUG: Python function do_rootfs_postprocess finished
> [stdlog] 2025-10-07 12:02:43,812 avocado.test cibuilder L0320 INFO | 
> NOTE: recipe isar-image-base-1.0-r0: task do_rootfs_postprocess: Failed
> 
> I think it is complaining about some packages does not have the optional
> "Origin" field. This has been fixed in debsbom repo with
> 034fc6af775a796cd67f5e4bfca013e99f7b8a5f, but your python-sbom receipe
> pulls a earlier version which does not contain this patch.

Hi, thanks for testing this. I noticed another bug on HOST_ARCH !=
DISTRO_ARCH when creating a derived distro (e.g. in isar-cip-core). All
these will be fixed in a v3. We plan to cut a v0.3 release this week
and then send out a patch series with the fixes.

By then, we also will have a "merging" support in the sbom tooling to
merge the SBOMs of the initrd, imaging and rootfs into a uniform
document. This should finally solve the issue of missing dependencies
in the BOM.

> 
> You can redo the test on your machine using avocado:
> 1. Have a clean clone of isar, checkout to branch next and apply your 
> patches:
> $ git clone -b next https://github.com/ilbers/isar.git
> $ cd isar
> $ git am /path-to/0001-my-contribution-to-isar.patch
> 2.Run kas shell, setup CI prerequisites (avocado, qemu) and cleanup:
> 
> $ ./kas/kas-container shell kas/isar.yaml --command \
>      "rm -rf /work/build/conf && /work/scripts/ci_setup.sh"
> 3.Run the failed test:
> $ cd /work/testsuite
> $ avocado run citest.py:ReproTest.test_repro_signed

Many thanks, that helps a lot. Maybe this could also be added to the
contributing guide (if not already) as some of these steps were not
obvious for me.

Best regards,
Felix

> 
> Best regards,
> Zhihang
> 
> On 9/17/25 08:33, 'Christoph Steiger' via isar-users wrote:
> > This patchset adds proper SBOM generation in the two standard formats
> > SPDX and CycloneDX during the rootfs generation process.
> > 
> > The generation is itself is handled by a SBOM generator  `debsbom` [1]
> > which is developed as an open source project at Siemens. It is still
> > early in development, but it has enough features for what we require
> > in isar. The required dependencies which are not yet available as
> > Debian packages were minimally packaged directly in isar too.
> > 
> > This is a followup of the previous RFC [2]. Since then the series has
> > changed a lot. The SBOM generation was moved from a simple OE lib to
> > `debsbom`. This also meant the introduction of a separate chroot was
> > necessary. The SBOM generation process was also moved from the image
> > step to the rootfs step, along with a lot of minor changes and
> > improvements.
> > 
> > [1] https://github.com/siemens/debsbom
> > [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ
> > 
> > Changes since v1:
> > 
> > - remove tarball
> > - refactor packaging (auto-derive python dependencies)
> > - only build missing packages (varies on bookworm, trixie, noble)
> > - add ubuntu support
> > - only generate sboms for supported distributions (bookworm/jammy and
> >    onwards)
> > - update debsbom (includes bug fixes and more information for source
> >    packages)
> > 
> > Christoph Steiger (3):
> >    meta: package python libraries for SBOM generation
> >    meta: package python3-debsbom
> >    meta: add SBOM generation with debsbom
> > 
> > Felix Moessbauer (1):
> >    override distro vendor in SBOM on Ubuntu
> > 
> >   meta-isar/conf/distro/ubuntu-common.inc       |  2 +
> >   meta/classes/image.bbclass                    |  8 ++-
> >   meta/classes/rootfs.bbclass                   |  7 ++-
> >   meta/classes/sbom.bbclass                     | 62 +++++++++++++++++++
> >   meta/classes/sdk.bbclass                      |  2 +-
> >   .../sbom-chroot/sbom-chroot.bb                | 30 +++++++++
> >   .../python3-beartype/files/rules              |  8 +++
> >   .../python3-beartype_0.19.0.bb                | 29 +++++++++
> >   .../files/pybuild.testfiles                   |  1 +
> >   .../python3-cyclonedx-lib/files/rules         |  8 +++
> >   .../python3-cyclonedx-lib_9.1.0.bb            | 48 ++++++++++++++
> >   ...icense-description-in-pyproject.toml.patch | 28 +++++++++
> >   .../python3-debsbom/files/rules               |  8 +++
> >   .../python3-debsbom/python3-debsbom_0.0.1.bb  | 44 +++++++++++++
> >   .../python3-packageurl/files/rules            |  8 +++
> >   .../python3-packageurl_0.16.0.bb              | 33 ++++++++++
> >   .../python3-py-serializable/files/rules       |  8 +++
> >   .../python3-py-serializable_2.0.0.bb          | 38 ++++++++++++
> >   .../python3-spdx-tools/files/rules            | 25 ++++++++
> >   .../python3-spdx-tools_0.8.3.bb               | 46 ++++++++++++++
> >   20 files changed, 440 insertions(+), 3 deletions(-)
> >   create mode 100644 meta/classes/sbom.bbclass
> >   create mode 100644 meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
> >   create mode 100644 meta/recipes-support/python3-beartype/files/rules
> >   create mode 100644 meta/recipes-support/python3-beartype/python3-beartype_0.19.0.bb
> >   create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/pybuild.testfiles
> >   create mode 100644 meta/recipes-support/python3-cyclonedx-lib/files/rules
> >   create mode 100644 meta/recipes-support/python3-cyclonedx-lib/python3-cyclonedx-lib_9.1.0.bb
> >   create mode 100644 meta/recipes-support/python3-debsbom/files/0001-Use-old-license-description-in-pyproject.toml.patch
> >   create mode 100644 meta/recipes-support/python3-debsbom/files/rules
> >   create mode 100644 meta/recipes-support/python3-debsbom/python3-debsbom_0.0.1.bb
> >   create mode 100644 meta/recipes-support/python3-packageurl/files/rules
> >   create mode 100644 meta/recipes-support/python3-packageurl/python3-packageurl_0.16.0.bb
> >   create mode 100644 meta/recipes-support/python3-py-serializable/files/rules
> >   create mode 100644 meta/recipes-support/python3-py-serializable/python3-py-serializable_2.0.0.bb
> >   create mode 100644 meta/recipes-support/python3-spdx-tools/files/rules
> >   create mode 100644 meta/recipes-support/python3-spdx-tools/python3-spdx-tools_0.8.3.bb
> > 

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/ad7c6e06fc9b6524768a23c2e3d4c4c1df1615eb.camel%40siemens.com.