[PATCH v7 0/7] Add SBOM generation with debsbom

[PATCH v7 0/7] Add SBOM generation with debsbom

['Felix Moessbauer' via isar-users]

This patchset adds proper SBOM generation in the two standard formats
SPDX and CycloneDX during the rootfs generation process.

The generation is itself is handled by a SBOM generator  `debsbom` [1]
which is developed as an open source project at Siemens. It is still
early in development, but it has enough features for what we require
in isar. The required dependencies which are not yet available as
Debian packages were minimally packaged directly in isar too.

This is a followup of the previous RFC [2]. Since then the series has
changed a lot. The SBOM generation was moved from a simple OE lib to
`debsbom`. This also meant the introduction of a separate chroot was
necessary. The SBOM generation process was also moved from the image
step to the rootfs step, along with a lot of minor changes and
improvements.

[1] https://github.com/siemens/debsbom
[2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ

Changes since v6:

- fixed imager bom failure on transitive image types (detected in isar-cip,
  wic -> squashfs).
- updated debsbom to 0.6.0+git
- add support for license information
- rebased onto next

Note: I'm still not able to run the full testsuite. The related patches
to cleanup the testsuite are pending on the list for quite some time. I
did some extensive local testing with isar-cip core and product layers,
but any additional testing is highly welcome.

Changes since v5:

- fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to
  machine changes made in image file)
- rebased onto next

Changes since v4:

- rebased onto next
- fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE})

Changes since v3:

- fix issue on external bullseye initramfs (we now disable sbom generation
  on all unsupported distros rootfs instances)
- update debsbom to v0.4.0
- rebased onto next

Changes since v2:

- fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions
- update debsbom to v0.3.0, which fixes the Origin: bug reported in v2
- generate SBOM for imager as well and create merged sbom of .wic image
- resend imager manifest + wic manifest patches to reduce conflicts

Note, that the patches p1-p5 are most important as they add basic SBOM
support. The remaining patches address the imager + .wic bom part,
which also can be merged later on.

Changes since v1:

- remove tarball
- refactor packaging (auto-derive python dependencies)
- only build missing packages (varies on bookworm, trixie, noble)
- add ubuntu support
- only generate sboms for supported distributions (bookworm/jammy and
  onwards)
- update debsbom (includes bug fixes and more information for source
  packages)


Felix Moessbauer (7):
  debsbom: update to version 0.6.0
  feat: add license information to SBOM as well
  add support to add imager dependencies to BOM
  wic: create uniform manifest describing all image components
  qemuamd64: add IMAGER_BOM entries
  imager: create SBOM of IMAGER_BOM packages
  wic: create uniform SBOM describing all image components

 doc/user_manual.md                            |  1 +
 meta-isar/conf/machine/qemuamd64.conf         |  1 +
 .../recipes-core/images/isar-image-ci.bb      |  1 +
 .../image-tools-extension.bbclass             | 29 ++++++++++++++++++
 meta/classes-recipe/image.bbclass             |  9 ++++++
 meta/classes-recipe/imagetypes_wic.bbclass    | 30 +++++++++++++++++++
 meta/classes/sbom.bbclass                     |  3 +-
 ..._0.5.1.bb => python3-debsbom_0.6.0+git.bb} |  3 +-
 8 files changed, 75 insertions(+), 2 deletions(-)
 rename meta/recipes-support/python3-debsbom/{python3-debsbom_0.5.1.bb => python3-debsbom_0.6.0+git.bb} (91%)

-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-1-felix.moessbauer%40siemens.com.

[PATCH v7 1/7] debsbom: update to version 0.6.0

['Felix Moessbauer' via isar-users]

With that, we get support to extract license information as well. We
include a fix of debsbom relevant when creating imager boms (e.g. for
the wic imager environment). This fix will be part of the 0.6.1 release.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 .../{python3-debsbom_0.5.1.bb => python3-debsbom_0.6.0+git.bb} | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
 rename meta/recipes-support/python3-debsbom/{python3-debsbom_0.5.1.bb => python3-debsbom_0.6.0+git.bb} (91%)

diff --git a/meta/recipes-support/python3-debsbom/python3-debsbom_0.5.1.bb b/meta/recipes-support/python3-debsbom/python3-debsbom_0.6.0+git.bb
similarity index 91%
rename from meta/recipes-support/python3-debsbom/python3-debsbom_0.5.1.bb
rename to meta/recipes-support/python3-debsbom/python3-debsbom_0.6.0+git.bb
index f1524129..25b0e4be 100644
--- a/meta/recipes-support/python3-debsbom/python3-debsbom_0.5.1.bb
+++ b/meta/recipes-support/python3-debsbom/python3-debsbom_0.6.0+git.bb
@@ -24,6 +24,7 @@ DEBIAN_BUILD_DEPENDS = "dh-python, \
                         python3-debian, \
                         python3-requests, \
                         python3-zstandard, \
+                        python3-license-expression, \
                         "
 
 DEBIAN_DEPENDS = "python3-apt, \${python3:Depends}, \${misc:Depends}"
@@ -34,7 +35,7 @@ SRC_URI = "git://github.com/siemens/debsbom.git;protocol=https;branch=main; \
            file://rules \
            file://0001-Use-old-license-description-in-pyproject.toml.patch \
            "
-SRCREV = "ba63720dfc0dc6dcb01994031e092aaebb62a399"
+SRCREV = "66d8dbba255f26b5ce1d5113f8df559eb1961bd2"
 
 do_prepare_build[cleandirs] += "${S}/debian"
 do_prepare_build() {
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-2-felix.moessbauer%40siemens.com.

[PATCH v7 2/7] feat: add license information to SBOM as well

['Felix Moessbauer' via isar-users]

We now have support in debsbom to also extract the license
information from the packages. We now add a parameter
SBOM_DEBSBOM_EXTRA_ARGS to add extra arguments to the sbom and set the
default to --with-licenses. By that license information is added. If
this is not wanted, the user can overwrite the bitbake variable.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/sbom.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/sbom.bbclass b/meta/classes/sbom.bbclass
index 3ed755d9..a16f6ea0 100644
--- a/meta/classes/sbom.bbclass
+++ b/meta/classes/sbom.bbclass
@@ -15,6 +15,7 @@ SBOM_DISTRO_VERSION ?= "1"
 SBOM_DISTRO_SUMMARY ?= "Linux distribution built with ISAR"
 SBOM_BASE_DISTRO_VENDOR ??= "debian"
 SBOM_DOCUMENT_UUID ?= ""
+SBOM_DEBSBOM_EXTRA_ARGS ?= "--with-licenses"
 
 # SPDX specific user variables
 SBOM_SPDX_NAMESPACE_PREFIX ?= "https://spdx.org/spdxdocs"
@@ -55,7 +56,7 @@ generate_sbom() {
             --base-distro-vendor '${SBOM_BASE_DISTRO_VENDOR}' \
             --cdx-serialnumber '${SBOM_DOCUMENT_UUID}' \
             --spdx-namespace '${SBOM_SPDX_NAMESPACE_PREFIX}'-'${SBOM_DOCUMENT_UUID}' \
-            --timestamp $TIMESTAMP
+            --timestamp $TIMESTAMP ${SBOM_DEBSBOM_EXTRA_ARGS}
 }
 
 do_generate_sbom[dirs] += "${DEPLOY_DIR_SBOM}"
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-3-felix.moessbauer%40siemens.com.

[PATCH v7 3/7] add support to add imager dependencies to BOM

['Felix Moessbauer' via isar-users]

Currently the imager dependencies which end up in the image are not
tracked in any BOM (e.g. the manifest file). As these cannot be
automatically derived from the IMAGER_INSTALL packages, we add a new
variable IMAGER_BOM that takes a list of binary packages which are
looked-up using dpkg-query during imaging and added to a local manifest.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 doc/user_manual.md                                | 1 +
 meta/classes-recipe/image-tools-extension.bbclass | 7 +++++++
 meta/classes-recipe/image.bbclass                 | 9 +++++++++
 3 files changed, 17 insertions(+)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index bea7557c..7ee56301 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -449,6 +449,7 @@ Some other variables include:
  - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS.
  - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable.
  - `IMAGER_INSTALL` -  The list of package dependencies for an imager like wic.
+ - `IMAGER_BOM` - The list of packages that should be added to the image BOM (e.g. the bootloader). These packages must also be available in the imager rootfs.
 
 ---
 
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index 3f284b39..2027effb 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -18,6 +18,7 @@ SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
 
 imager_run() {
     local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
+    local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
 
     schroot_create_configs
     insert_mounts
@@ -68,6 +69,12 @@ EOAPT
 
     schroot -r -c ${session_id} "$@"
 
+    if [ -n "${local_bom}" ]; then
+        schroot -r -c ${session_id} -d / -- \
+            dpkg-query -W -f='${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' ${local_bom} > \
+        ${WORKDIR}/imager.manifest
+    fi
+
     schroot -e -c ${session_id}
 
     remove_mounts
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index e605bc80..afe6906e 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -221,9 +221,11 @@ python() {
 
     imager_install = set()
     imager_build_deps = set()
+    imager_bom = set()
     conversion_install = set()
     for bt in basetypes:
         local_imager_install = set()
+        local_imager_bom = set()
         local_conversion_install = set()
         vardeps = set()
         cmds = []
@@ -251,6 +253,10 @@ python() {
             local_imager_install.add(dep)
         for dep in (d.getVar('IMAGER_BUILD_DEPS:' + bt_clean) or '').split():
             imager_build_deps.add(dep)
+        for dep in (d.getVar('IMAGER_BOM:' + bt_clean) or '').split():
+            imager_bom.add(dep)
+            local_imager_bom.add(dep)
+        vardeps.add('IMAGER_BOM:' + bt_clean)
 
         # construct image command
         image_cmd = localdata.getVar('IMAGE_CMD:' + bt_clean)
@@ -325,11 +331,14 @@ python() {
         bb.build.addtask(task, 'do_image', after, d)
 
         # set per type imager dependencies
+        d.setVar('BOM_image_%s' % bt_clean, d.getVar('IMAGER_BOM'))
+        d.appendVar('BOM_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_bom)))
         d.setVar('INSTALL_image_%s' % bt_clean, d.getVar('IMAGER_INSTALL'))
         d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install)))
         d.appendVarFlag(task, 'vardeps', ' INSTALL_image_%s' % bt_clean)
 
     d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
+    d.appendVar('IMAGER_BOM', ' ' + ' '.join(sorted(imager_bom)))
     d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
 }
 
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-4-felix.moessbauer%40siemens.com.

[PATCH v7 4/7] wic: create uniform manifest describing all image components

['Felix Moessbauer' via isar-users]

A wic image consists of potentially many different components. All these
should be covered by a single BOM.

After creating the wic image, we collect the individual manifest files
(rootfs, initrd, imaging), deduplicate it and deploy it into the image
deploy dir (as .wic.manifest).

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes-recipe/imagetypes_wic.bbclass | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index fb0b81a9..c75d481d 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -196,4 +196,9 @@ EOIMAGER
     sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
     sudo chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
     rm -rf ${IMAGE_ROOTFS}/../pseudo
+
+    cat ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.manifest \
+        ${DEPLOY_DIR_IMAGE}/${INITRD_DEPLOY_FILE}.manifest \
+        ${WORKDIR}/imager.manifest 2>/dev/null \
+        | sort | uniq > "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.manifest"
 }
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-5-felix.moessbauer%40siemens.com.

[PATCH v7 5/7] qemuamd64: add IMAGER_BOM entries

['Felix Moessbauer' via isar-users]

To give an example how to add components to the imager BOM, we set the
corresponding variable for the qemuamd64 target.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf          | 1 +
 meta-test/recipes-core/images/isar-image-ci.bb | 1 +
 2 files changed, 2 insertions(+)

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index 7d5987c6..8d0753a8 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -7,6 +7,7 @@ KERNEL_NAME ?= "amd64"
 
 IMAGE_FSTYPES ?= "wic"
 WKS_FILE ?= "sdimage-efi"
+IMAGER_BOM:wic += "${GRUB_BOOTLOADER_INSTALL}"
 IMAGER_INSTALL:wic += "${GRUB_BOOTLOADER_INSTALL}"
 
 QEMU_ARCH ?= "x86_64"
diff --git a/meta-test/recipes-core/images/isar-image-ci.bb b/meta-test/recipes-core/images/isar-image-ci.bb
index 7f2b404d..3db7def8 100644
--- a/meta-test/recipes-core/images/isar-image-ci.bb
+++ b/meta-test/recipes-core/images/isar-image-ci.bb
@@ -24,6 +24,7 @@ WKS_FILE:qemuamd64:debian-bullseye ?= "sdimage-efi-btrfs"
 IMAGE_INSTALL:append:qemuamd64:debian-bullseye = " expand-on-first-boot"
 IMAGER_INSTALL:remove:qemuamd64:debian-bullseye ?= "${GRUB_BOOTLOADER_INSTALL}"
 IMAGER_INSTALL:append:qemuamd64:debian-bullseye ?= " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-progs"
+IMAGER_BOM:wic = "${SYSTEMD_BOOTLOADER_INSTALL}"
 IMAGE_PREINSTALL:append:qemuamd64:debian-bullseye ?= " btrfs-progs"
 # Explicitly remove from wic since it is set in qemuamd64.conf:
 IMAGER_INSTALL:wic:remove:qemuamd64:debian-bullseye ?= "${GRUB_BOOTLOADER_INSTALL}"
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-6-felix.moessbauer%40siemens.com.

[PATCH v7 6/7] imager: create SBOM of IMAGER_BOM packages

['Felix Moessbauer' via isar-users]

This uses the same interface as the .manifest file, but adds the
packagse to an SBOM.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 .../image-tools-extension.bbclass             | 22 +++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index 2027effb..b0f25a69 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -73,6 +73,8 @@ EOAPT
         schroot -r -c ${session_id} -d / -- \
             dpkg-query -W -f='${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' ${local_bom} > \
         ${WORKDIR}/imager.manifest
+
+        ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom', '', d)}
     fi
 
     schroot -e -c ${session_id}
@@ -80,3 +82,23 @@ EOAPT
     remove_mounts
     schroot_delete_configs
 }
+
+generate_imager_sbom() {
+    TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
+    sbom_document_uuid="${@d.getVar('SBOM_DOCUMENT_UUID') or generate_document_uuid(d, False)}"
+    bwrap \
+        --unshare-user \
+        --unshare-pid \
+        --bind ${SBOM_CHROOT} / \
+        --bind $schroot_dir /mnt/rootfs \
+        --bind ${WORKDIR} /mnt/deploy-dir \
+        -- debsbom -vv generate ${SBOM_DEBSBOM_TYPE_ARGS} \
+            --from-pkglist -r /mnt/rootfs -o /mnt/deploy-dir/imager \
+            --distro-name '${SBOM_DISTRO_NAME}-Imager' --distro-supplier '${SBOM_DISTRO_SUPPLIER}' \
+            --distro-version '${SBOM_DISTRO_VERSION}' --distro-arch '${DISTRO_ARCH}' \
+            --base-distro-vendor '${SBOM_BASE_DISTRO_VENDOR}' \
+            --cdx-serialnumber $sbom_document_uuid \
+            --spdx-namespace '${SBOM_SPDX_NAMESPACE_PREFIX}'-$sbom_document_uuid \
+            --timestamp $TIMESTAMP ${SBOM_DEBSBOM_EXTRA_ARGS} \
+    < ${WORKDIR}/imager.manifest
+}
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-7-felix.moessbauer%40siemens.com.

[PATCH v7 7/7] wic: create uniform SBOM describing all image components

['Felix Moessbauer' via isar-users]

A wic image consists of potentially many different components. All these
should be covered by a single SBOM.

After creating the wic image, we collect the individual sbom files
(rootfs, initrd, imaging) and semantically merge it with the debsbom
tool. The merge SBOM is then deployed as .wic.(spdx|cdx).json next to
the wic image.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes-recipe/imagetypes_wic.bbclass | 25 ++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index c75d481d..fe31e4e6 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -201,4 +201,29 @@ EOIMAGER
         ${DEPLOY_DIR_IMAGE}/${INITRD_DEPLOY_FILE}.manifest \
         ${WORKDIR}/imager.manifest 2>/dev/null \
         | sort | uniq > "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.manifest"
+
+    for bomtype in ${SBOM_TYPES}; do
+        merge_wic_sbom $bomtype
+    done
+}
+
+merge_wic_sbom() {
+    BOMTYPE="$1"
+    TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
+    sbom_document_uuid="${@d.getVar('SBOM_DOCUMENT_UUID') or generate_document_uuid(d, False)}"
+
+    cat ${IMAGE_FULLNAME}.${bomtype}.json \
+        ${INITRD_DEPLOY_FILE}.${bomtype}.json \
+        ${WORKDIR}/imager.${bomtype}.json 2>/dev/null | \
+    bwrap \
+        --unshare-user \
+        --unshare-pid \
+        --bind ${SBOM_CHROOT} / \
+        -- debsbom -v merge -t $BOMTYPE \
+            --distro-name '${SBOM_DISTRO_NAME}-Image' --distro-supplier '${SBOM_DISTRO_SUPPLIER}' \
+            --distro-version '${SBOM_DISTRO_VERSION}' --base-distro-vendor '${SBOM_BASE_DISTRO_VENDOR}' \
+            --cdx-serialnumber $sbom_document_uuid \
+            --spdx-namespace '${SBOM_SPDX_NAMESPACE_PREFIX}'-$sbom_document_uuid \
+            --timestamp $TIMESTAMP - -o - \
+     > ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.$bomtype.json
 }
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260121150719.2719579-8-felix.moessbauer%40siemens.com.

Re: [PATCH v7 0/7] Add SBOM generation with debsbom

[Zhihang Wei]

On 1/21/26 16:07, 'Felix Moessbauer' via isar-users wrote:
> This patchset adds proper SBOM generation in the two standard formats
> SPDX and CycloneDX during the rootfs generation process.
>
> The generation is itself is handled by a SBOM generator  `debsbom` [1]
> which is developed as an open source project at Siemens. It is still
> early in development, but it has enough features for what we require
> in isar. The required dependencies which are not yet available as
> Debian packages were minimally packaged directly in isar too.
>
> This is a followup of the previous RFC [2]. Since then the series has
> changed a lot. The SBOM generation was moved from a simple OE lib to
> `debsbom`. This also meant the introduction of a separate chroot was
> necessary. The SBOM generation process was also moved from the image
> step to the rootfs step, along with a lot of minor changes and
> improvements.
>
> [1] https://github.com/siemens/debsbom
> [2] https://groups.google.com/g/isar-users/c/8L-CF4BJY0I/m/p0N3o_zfAAAJ
>
> Changes since v6:
>
> - fixed imager bom failure on transitive image types (detected in isar-cip,
>    wic -> squashfs).
> - updated debsbom to 0.6.0+git
> - add support for license information
> - rebased onto next
>
> Note: I'm still not able to run the full testsuite. The related patches
> to cleanup the testsuite are pending on the list for quite some time. I
> did some extensive local testing with isar-cip core and product layers,
> but any additional testing is highly welcome.
>
> Changes since v5:
>
> - fix isar-image-ci on qemuamd64-bullseye (set IMAGER_BOM according to
>    machine changes made in image file)
> - rebased onto next
>
> Changes since v4:
>
> - rebased onto next
> - fix race condition on creation of ${DEPLOY_DIR_SBOM} (aka ${DEPLOY_DIR_IMAGE})
>
> Changes since v3:
>
> - fix issue on external bullseye initramfs (we now disable sbom generation
>    on all unsupported distros rootfs instances)
> - update debsbom to v0.4.0
> - rebased onto next
>
> Changes since v2:
>
> - fix issues when HOST_ARCH != DISTRO_ARCH on derived distributions
> - update debsbom to v0.3.0, which fixes the Origin: bug reported in v2
> - generate SBOM for imager as well and create merged sbom of .wic image
> - resend imager manifest + wic manifest patches to reduce conflicts
>
> Note, that the patches p1-p5 are most important as they add basic SBOM
> support. The remaining patches address the imager + .wic bom part,
> which also can be merged later on.
>
> Changes since v1:
>
> - remove tarball
> - refactor packaging (auto-derive python dependencies)
> - only build missing packages (varies on bookworm, trixie, noble)
> - add ubuntu support
> - only generate sboms for supported distributions (bookworm/jammy and
>    onwards)
> - update debsbom (includes bug fixes and more information for source
>    packages)
>
>
> Felix Moessbauer (7):
>    debsbom: update to version 0.6.0
>    feat: add license information to SBOM as well
>    add support to add imager dependencies to BOM
>    wic: create uniform manifest describing all image components
>    qemuamd64: add IMAGER_BOM entries
>    imager: create SBOM of IMAGER_BOM packages
>    wic: create uniform SBOM describing all image components
>
>   doc/user_manual.md                            |  1 +
>   meta-isar/conf/machine/qemuamd64.conf         |  1 +
>   .../recipes-core/images/isar-image-ci.bb      |  1 +
>   .../image-tools-extension.bbclass             | 29 ++++++++++++++++++
>   meta/classes-recipe/image.bbclass             |  9 ++++++
>   meta/classes-recipe/imagetypes_wic.bbclass    | 30 +++++++++++++++++++
>   meta/classes/sbom.bbclass                     |  3 +-
>   ..._0.5.1.bb => python3-debsbom_0.6.0+git.bb} |  3 +-
>   8 files changed, 75 insertions(+), 2 deletions(-)
>   rename meta/recipes-support/python3-debsbom/{python3-debsbom_0.5.1.bb => python3-debsbom_0.6.0+git.bb} (91%)
>
Hi,
I have tested this version on top of the merged test suite improvement
patches on CI. There are several failures, all related to 
'do_image_wic', and
the errors can be categorized into two classes:
1. "bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory"
2. "dpkg-query: no packages found matching systemd-boot-efi:arm64" or 
"systemd-boot-efi:amd64".

I'll write here some instructions on how to recreate the errors using 
avocado.
For each catagory of error, I'll show one example failed test case.

To redo the test using avocado:
1. Have a clean clone of isar, checkout to branch next and apply your 
patches:
$ git clone -b next https://github.com/ilbers/isar.git
$ cd isar
$ git am /path-to/0001-my-contribution-to-isar.patch
2. Disable unrelated targets to make error appears faster, by applying
the following diff to "testsuite/citest.py":
diff --git a/testsuite/citest.py b/testsuite/citest.py
index f11cd5af..225a07a8 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -78,7 +78,6 @@ class DevTest(CIBaseTest):
      def test_dev_apps(self):
          targets = [
              'mc:qemuamd64-bookworm:isar-image-ci',
-            'mc:qemuarm64-bookworm:isar-image-base',
          ]

          self.init()
@@ -238,10 +237,6 @@ class CrossTest(CIBaseTest):

      def test_cross(self):
          targets = [
-            'mc:qemuarm-buster:isar-image-ci',
-            'mc:qemuarm-bullseye:isar-image-ci',
-            'mc:qemuarm-bookworm:isar-image-ci',
-            'mc:qemuarm64-focal:isar-image-base',
              'mc:nanopi-neo-efi-bookworm:isar-image-base',
          ]

3. Run kas shell, setup CI prerequisites (avocado, qemu) and cleanup:
$ ./kas/kas-container shell kas/isar.yaml --command \
"rm -rf /work/build/conf && /work/scripts/ci_setup.sh"
4. Run the failed test related with "bwrap: Can't find source path ..."
$ cd /work/testsuite
$ avocado run citest.py:CrossTest.test_cross$
(took ~13 minutes on my laptop for this error to show up)
5. Run the failed test related with "dpkg-query ... systemd-boot-efi:amd64"
$ avocado run citest.py:DevTest.test_dev_apps$
(~4 minutes on my laptop)
6. Run the failed test related with "dpkg-query ... systemd-boot-efi:arm64"
$ avocado run citest.py:CrossTest.test_cross_debsrc$
(~11 minutes on my laptop)

You can also view the logs and download the artifacts on GitLab:
Fast CI: https://gitlab.isar-build.org/ilbers/isar/-/jobs/2741
Full CI: https://gitlab.isar-build.org/ilbers/isar/-/jobs/2744

Below is a summary of the reasons for failures on test cases:

Failed in fast:
01-/builds/ilbers/isar/testsuite/citest.py:DevTest.test_dev: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.109500
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
02-/builds/ilbers/isar/testsuite/citest.py:DevTest.test_dev_apps: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.263618
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
03-/builds/ilbers/isar/testsuite/citest.py:DevTest.test_dev_rebuild: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.346091
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
09-/builds/ilbers/isar/testsuite/citest.py:CrossTest.test_cross: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-armhf/isar-image-base-nanopi-neo-efi/1.0-r0/temp/run.do_image_wic.849937
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
19-/builds/ilbers/isar/testsuite/citest.py:CrossTest.test_cross_debsrc: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/run.do_image_wic.996506
   Error: dpkg-query: no packages found matching systemd-boot-efi:arm64
24-/builds/ilbers/isar/testsuite/citest.py:CrossTest.test_cross_rpi: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-arm64/isar-image-base-rpi-arm64-v8-efi/1.0-r0/temp/run.do_image_wic.1341846
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
25-/builds/ilbers/isar/testsuite/citest.py:CrossTest.test_cross_dependencies: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/run.do_image_wic.1449197
   Error: dpkg-query: no packages found matching systemd-boot-efi:arm64

Failed in full:
01-/builds/ilbers/isar/testsuite/citest.py:ReproTest.test_repro_signed: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/raspios-bookworm-armhf/isar-image-base-rpi-arm-v7/1.0-r0/temp/run.do_image_wic.282174
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
02-/builds/ilbers/isar/testsuite/citest.py:ReproTest.test_repro_unsigned: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-base-qemuamd64/1.0-r0/temp/run.do_image_wic.452427
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
04-/builds/ilbers/isar/testsuite/citest.py:InstallerTest.test_installer_build: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.662656
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
07-/builds/ilbers/isar/testsuite/citest.py:PrebuiltTest.test_prebuilt_containers: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.722781
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
10-/builds/ilbers/isar/testsuite/citest.py:KernelTests.test_kernel_cross: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bullseye-armhf/isar-image-base-de0-nano-soc/1.0-r0/temp/run.do_image_wic.1250406
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
11-/builds/ilbers/isar/testsuite/citest.py:KernelTests.test_per_kernel: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/run.do_image_wic.1509493
   Error: dpkg-query: no packages found matching systemd-boot-efi:arm64
12-/builds/ilbers/isar/testsuite/citest.py:InitRdTest.test_dracut_in_image: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.1541317
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
15-/builds/ilbers/isar/testsuite/citest.py:InitRdTest.test_var_initrd_image: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.1662133
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
16-/builds/ilbers/isar/testsuite/citest.py:InitRdTest.test_var_image_initrd: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.1673793
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
17-/builds/ilbers/isar/testsuite/citest.py:InitRdTest.test_var_image_initrd_and_initrd_image: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.1705223
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
18-/builds/ilbers/isar/testsuite/citest.py:InitRdCrossTests.test_dracut_in_image: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.1759604
   Error: dpkg-query: no packages found matching systemd-boot-efi:amd64
20-/builds/ilbers/isar/testsuite/citest.py:WicTest.test_wic_nodeploy_partitions: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/run.do_image_wic.2000487
   Error: dpkg-query: no packages found matching systemd-boot-efi:arm64
21-/builds/ilbers/isar/testsuite/citest.py:WicTest.test_wic_deploy_partitions: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-arm64/isar-image-ci-qemuarm64/1.0-r0/temp/run.do_image_wic.2010205
   Error: dpkg-query: no packages found matching systemd-boot-efi:arm64
22-/builds/ilbers/isar/testsuite/citest.py:NoCrossTest.test_nocross: FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-buster-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.737468
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/ubuntu-focal-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.736880
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
   This test case has 10 errors like the above two in total.
43-/builds/ilbers/isar/testsuite/citest.py:NoCrossTest.test_nocross_debsrc: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bookworm-armhf/isar-image-ci-qemuarm/1.0-r0/temp/run.do_image_wic.1846403
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
45-/builds/ilbers/isar/testsuite/citest.py:NoCrossTest.test_nocross_rpi: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/raspios-bullseye-armhf/isar-image-base-rpi-arm/1.0-r0/temp/run.do_image_wic.2624140
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
46-/builds/ilbers/isar/testsuite/citest.py:NoCrossTest.test_nocross_rpi_debsrc: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/raspios-bookworm-armhf/isar-image-base-rpi-arm/1.0-r0/temp/run.do_image_wic.2823215
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory
55-/builds/ilbers/isar/testsuite/citest.py:CustomizationsTest.test_single_customization: 
FAIL
   Failed task in: 
/builds/ilbers/isar/build/tmp/work/debian-bullseye-amd64/isar-image-ci-qemuamd64/1.0-r0/temp/run.do_image_wic.3377222
   Error: bwrap: Can't find source path 
/builds/ilbers/isar/build/tmp/deploy/sbom/sbom-chroot: No such file or 
directory

Zhihang

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/45067661-4b18-483d-8632-f8d568d37e36%40ilbers.de.