[RFC PATCH 0/3] Create uniform manifest file

[RFC PATCH 0/3] Create uniform manifest file

['Felix Moessbauer' via isar-users]

The .manifest file contains the list of installed packages, along with their
binary -> source package relations. We recently added support to generate
the manifest also for an external initrd rootfs, however support for the
imager was still missing.

This series adds this support and also creates a uniform manifest file for
all components in the .wic image. As the imager dependencies cannot be
tracked precisely in an automated way (the IMAGER_INSTALL contains tooling
as well as artifacts which are copied into the image), we add a new variable
IMAGER_BOM to define the set of packages that should end up in the BOM.

The RFC implements this support for the .manifest file. Later on, a similar
approach can be implemented for the SBOM.

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (3):
  Add support to add imager dependencies to BOM
  wic: create uniform manifest describing all image components
  qemuamd64: add IMAGER_BOM entries

 doc/user_manual.md                         | 1 +
 meta-isar/conf/machine/qemuamd64.conf      | 1 +
 meta/classes/image-tools-extension.bbclass | 7 +++++++
 meta/classes/image.bbclass                 | 6 ++++++
 meta/classes/imagetypes_wic.bbclass        | 4 ++++
 5 files changed, 19 insertions(+)

-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251010151205.2122000-1-felix.moessbauer%40siemens.com.

[RFC PATCH 1/3] Add support to add imager dependencies to BOM

['Felix Moessbauer' via isar-users]

Currently the imager dependencies which end up in the image are not
tracked in any BOM (e.g. the manifest file). As these cannot be
automatically derived from the IMAGER_INSTALL packages, we add a new
variable IMAGER_BOM that takes a list of binary packages which are
looked-up using dpkg-query during imaging and added to a local manifest.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 doc/user_manual.md                         | 1 +
 meta/classes/image-tools-extension.bbclass | 7 +++++++
 meta/classes/image.bbclass                 | 6 ++++++
 3 files changed, 14 insertions(+)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 67f91973..deb66a45 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -454,6 +454,7 @@ Some other variables include:
  - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS.
  - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable.
  - `IMAGER_INSTALL` -  The list of package dependencies for an imager like wic.
+ - `IMAGER_BOM` - The list of packages that should be added to the image BOM (e.g. the bootloader). These packages must also be available in the imager rootfs.
 
 ---
 
diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass
index 5e248f2e..6aa790f3 100644
--- a/meta/classes/image-tools-extension.bbclass
+++ b/meta/classes/image-tools-extension.bbclass
@@ -20,6 +20,7 @@ SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
 
 imager_run() {
     local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
+    local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
 
     schroot_create_configs
     insert_mounts
@@ -70,6 +71,12 @@ EOAPT
 
     schroot -r -c ${session_id} "$@"
 
+    if [ -n "${local_bom}" ]; then
+        schroot -r -c ${session_id} -d / -- \
+            dpkg-query -W -f='${source:Package}|${source:Version}|${binary:Package}|${Version}\n' ${local_bom} > \
+        ${WORKDIR}/imager.manifest
+    fi
+
     schroot -e -c ${session_id}
 
     remove_mounts
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index bd1b8552..53017963 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -184,6 +184,7 @@ python() {
 
     imager_install = set()
     imager_build_deps = set()
+    imager_bom = set()
     conversion_install = set()
     for bt in basetypes:
         local_imager_install = set()
@@ -214,6 +215,8 @@ python() {
             local_imager_install.add(dep)
         for dep in (d.getVar('IMAGER_BUILD_DEPS:' + bt_clean) or '').split():
             imager_build_deps.add(dep)
+        for dep in (d.getVar('IMAGER_BOM:' + bt_clean) or '').split():
+            imager_bom.add(dep)
 
         # construct image command
         image_cmd = localdata.getVar('IMAGE_CMD:' + bt_clean)
@@ -288,11 +291,14 @@ python() {
         bb.build.addtask(task, 'do_image', after, d)
 
         # set per type imager dependencies
+        d.setVar('BOM_image_%s' % bt_clean, d.getVar('IMAGER_BOM'))
+        d.appendVar('BOM_image_%s' % bt_clean, ' ' + ' '.join(sorted(imager_bom)))
         d.setVar('INSTALL_image_%s' % bt_clean, d.getVar('IMAGER_INSTALL'))
         d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install)))
         d.appendVarFlag(task, 'vardeps', ' INSTALL_image_%s' % bt_clean)
 
     d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
+    d.appendVar('IMAGER_BOM', ' ' + ' '.join(sorted(imager_bom)))
     d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
 }
 
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251010151205.2122000-2-felix.moessbauer%40siemens.com.

Re: [RFC PATCH 1/3] Add support to add imager dependencies to BOM

['Jan Kiszka' via isar-users]

On 10.10.25 17:12, Felix Moessbauer wrote:
> Currently the imager dependencies which end up in the image are not
> tracked in any BOM (e.g. the manifest file). As these cannot be
> automatically derived from the IMAGER_INSTALL packages, we add a new
> variable IMAGER_BOM that takes a list of binary packages which are
> looked-up using dpkg-query during imaging and added to a local manifest.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  doc/user_manual.md                         | 1 +
>  meta/classes/image-tools-extension.bbclass | 7 +++++++
>  meta/classes/image.bbclass                 | 6 ++++++
>  3 files changed, 14 insertions(+)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 67f91973..deb66a45 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -454,6 +454,7 @@ Some other variables include:
>   - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS.
>   - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable.
>   - `IMAGER_INSTALL` -  The list of package dependencies for an imager like wic.
> + - `IMAGER_BOM` - The list of packages that should be added to the image BOM (e.g. the bootloader). These packages must also be available in the imager rootfs.
>  

Hmm, how about IMAGE_BUILT_USING? Would be wording-wise closer to
Debian's Built-Using.

One could even imagine having a ROOTFS_BUILT_USING as well in case evil
hacks are done via post-process commands... But maybe better exclude that.

>  ---
>  
> diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass
> index 5e248f2e..6aa790f3 100644
> --- a/meta/classes/image-tools-extension.bbclass
> +++ b/meta/classes/image-tools-extension.bbclass
> @@ -20,6 +20,7 @@ SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
>  
>  imager_run() {
>      local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
> +    local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
>  
>      schroot_create_configs
>      insert_mounts
> @@ -70,6 +71,12 @@ EOAPT
>  
>      schroot -r -c ${session_id} "$@"
>  
> +    if [ -n "${local_bom}" ]; then
> +        schroot -r -c ${session_id} -d / -- \
> +            dpkg-query -W -f='${source:Package}|${source:Version}|${binary:Package}|${Version}\n' ${local_bom} > \
> +        ${WORKDIR}/imager.manifest
> +    fi
> +
>      schroot -e -c ${session_id}
>  
>      remove_mounts
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index bd1b8552..53017963 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -184,6 +184,7 @@ python() {
>  
>      imager_install = set()
>      imager_build_deps = set()
> +    imager_bom = set()
>      conversion_install = set()
>      for bt in basetypes:
>          local_imager_install = set()
> @@ -214,6 +215,8 @@ python() {
>              local_imager_install.add(dep)
>          for dep in (d.getVar('IMAGER_BUILD_DEPS:' + bt_clean) or '').split():
>              imager_build_deps.add(dep)
> +        for dep in (d.getVar('IMAGER_BOM:' + bt_clean) or '').split():
> +            imager_bom.add(dep)

No, this is about creating an image BOM. The imager remains a tool
outside of the distributed image.

>  
>          # construct image command
>          image_cmd = localdata.getVar('IMAGE_CMD:' + bt_clean)
> @@ -288,11 +291,14 @@ python() {
>          bb.build.addtask(task, 'do_image', after, d)
>  
>          # set per type imager dependencies
> +        d.setVar('BOM_image_%s' % bt_clean, d.getVar('IMAGER_BOM'))
> +        d.appendVar('BOM_image_%s' % bt_clean, ' ' + ' '.join(sorted(imager_bom)))

And you actually state that the imager BOM is for the image.

>          d.setVar('INSTALL_image_%s' % bt_clean, d.getVar('IMAGER_INSTALL'))
>          d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install)))
>          d.appendVarFlag(task, 'vardeps', ' INSTALL_image_%s' % bt_clean)
>  
>      d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
> +    d.appendVar('IMAGER_BOM', ' ' + ' '.join(sorted(imager_bom)))
>      d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
>  }
>  

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/68af7afb-d426-446d-9ed7-59110343370c%40siemens.com.

Re: [RFC PATCH 1/3] Add support to add imager dependencies to BOM

['MOESSBAUER, Felix' via isar-users]

On Wed, 2025-10-15 at 14:57 +0200, Jan Kiszka wrote:
> On 10.10.25 17:12, Felix Moessbauer wrote:
> > Currently the imager dependencies which end up in the image are not
> > tracked in any BOM (e.g. the manifest file). As these cannot be
> > automatically derived from the IMAGER_INSTALL packages, we add a new
> > variable IMAGER_BOM that takes a list of binary packages which are
> > looked-up using dpkg-query during imaging and added to a local manifest.
> > 
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >  doc/user_manual.md                         | 1 +
> >  meta/classes/image-tools-extension.bbclass | 7 +++++++
> >  meta/classes/image.bbclass                 | 6 ++++++
> >  3 files changed, 14 insertions(+)
> > 
> > diff --git a/doc/user_manual.md b/doc/user_manual.md
> > index 67f91973..deb66a45 100644
> > --- a/doc/user_manual.md
> > +++ b/doc/user_manual.md
> > @@ -454,6 +454,7 @@ Some other variables include:
> >   - `FILESEXTRAPATHS` - The default directories BitBake uses when it processes recipes are initially defined by the FILESPATH variable. You can extend FILESPATH variable by using FILESEXTRAPATHS.
> >   - `FILESOVERRIDES` - A subset of OVERRIDES used by the build system for creating FILESPATH. The FILESOVERRIDES variable uses overrides to automatically extend the FILESPATH variable.
> >   - `IMAGER_INSTALL` -  The list of package dependencies for an imager like wic.
> > + - `IMAGER_BOM` - The list of packages that should be added to the image BOM (e.g. the bootloader). These packages must also be available in the imager rootfs.
> >  
> 
> Hmm, how about IMAGE_BUILT_USING? Would be wording-wise closer to
> Debian's Built-Using.

The debian Built-Using is specific to a debian policy and references
source packages. While here we track binary packages and also the
debian policy does not apply. By that, I prefer to use a different
term.

> 
> One could even imagine having a ROOTFS_BUILT_USING as well in case evil
> hacks are done via post-process commands... But maybe better exclude that.

Yes, that's not in scope for now.

> 
> >  ---
> >  
> > diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass
> > index 5e248f2e..6aa790f3 100644
> > --- a/meta/classes/image-tools-extension.bbclass
> > +++ b/meta/classes/image-tools-extension.bbclass
> > @@ -20,6 +20,7 @@ SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
> >  
> >  imager_run() {
> >      local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
> > +    local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
> >  
> >      schroot_create_configs
> >      insert_mounts
> > @@ -70,6 +71,12 @@ EOAPT
> >  
> >      schroot -r -c ${session_id} "$@"
> >  
> > +    if [ -n "${local_bom}" ]; then
> > +        schroot -r -c ${session_id} -d / -- \
> > +            dpkg-query -W -f='${source:Package}|${source:Version}|${binary:Package}|${Version}\n' ${local_bom} > \
> > +        ${WORKDIR}/imager.manifest
> > +    fi
> > +
> >      schroot -e -c ${session_id}
> >  
> >      remove_mounts
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > index bd1b8552..53017963 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -184,6 +184,7 @@ python() {
> >  
> >      imager_install = set()
> >      imager_build_deps = set()
> > +    imager_bom = set()
> >      conversion_install = set()
> >      for bt in basetypes:
> >          local_imager_install = set()
> > @@ -214,6 +215,8 @@ python() {
> >              local_imager_install.add(dep)
> >          for dep in (d.getVar('IMAGER_BUILD_DEPS:' + bt_clean) or '').split():
> >              imager_build_deps.add(dep)
> > +        for dep in (d.getVar('IMAGER_BOM:' + bt_clean) or '').split():
> > +            imager_bom.add(dep)
> 
> No, this is about creating an image BOM. The imager remains a tool
> outside of the distributed image.

I just follow the existing terminology of the imagetypes logic here. 
The term "image_bom" is misleading as well, as image bom implies that
it contains a list of all packages which end up in the image (i.e. in
the .wic blob).

> 
> >  
> >          # construct image command
> >          image_cmd = localdata.getVar('IMAGE_CMD:' + bt_clean)
> > @@ -288,11 +291,14 @@ python() {
> >          bb.build.addtask(task, 'do_image', after, d)
> >  
> >          # set per type imager dependencies
> > +        d.setVar('BOM_image_%s' % bt_clean, d.getVar('IMAGER_BOM'))
> > +        d.appendVar('BOM_image_%s' % bt_clean, ' ' + ' '.join(sorted(imager_bom)))
> 
> And you actually state that the imager BOM is for the image.

Yes, again that follows the imagetypes naming scheme.

But I'm open for better names as long as they don't introduce yet
another terminology.

Felix

> 
> >          d.setVar('INSTALL_image_%s' % bt_clean, d.getVar('IMAGER_INSTALL'))
> >          d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install)))
> >          d.appendVarFlag(task, 'vardeps', ' INSTALL_image_%s' % bt_clean)
> >  
> >      d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
> > +    d.appendVar('IMAGER_BOM', ' ' + ' '.join(sorted(imager_bom)))
> >      d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
> >  }
> >  
> 
> Jan
> 
> -- 
> Siemens AG, Foundational Technologies
> Linux Expert Center

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/24af77db9435fb3d383c4d881c3f39deb6ccd60c.camel%40siemens.com.

[RFC PATCH 2/3] wic: create uniform manifest describing all image components

['Felix Moessbauer' via isar-users]

A wic image consists of potentially many different components. All these
should be covered by a single BOM.

After creating the wic image, we collect the individual manifest files
(rootfs, initrd, imaging), deduplicate it and deploy it into the image
deploy dir (as .wic.manifest).

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/imagetypes_wic.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/imagetypes_wic.bbclass b/meta/classes/imagetypes_wic.bbclass
index fb0b81a9..f052f943 100644
--- a/meta/classes/imagetypes_wic.bbclass
+++ b/meta/classes/imagetypes_wic.bbclass
@@ -195,5 +195,9 @@ EOIMAGER
 
     sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
     sudo chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
+    cat ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.manifest \
+        ${DEPLOY_DIR_IMAGE}/${INITRD_DEPLOY_FILE}.manifest \
+        ${WORKDIR}/imager.manifest 2>/dev/null \
+        | sort | uniq > "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.manifest"
     rm -rf ${IMAGE_ROOTFS}/../pseudo
 }
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251010151205.2122000-3-felix.moessbauer%40siemens.com.

[RFC PATCH 3/3] qemuamd64: add IMAGER_BOM entries

['Felix Moessbauer' via isar-users]

To give an example how to add components to the imager BOM, we set the
corresponding variable for the qemuamd64 target.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index 7d5987c6..8d0753a8 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -7,6 +7,7 @@ KERNEL_NAME ?= "amd64"
 
 IMAGE_FSTYPES ?= "wic"
 WKS_FILE ?= "sdimage-efi"
+IMAGER_BOM:wic += "${GRUB_BOOTLOADER_INSTALL}"
 IMAGER_INSTALL:wic += "${GRUB_BOOTLOADER_INSTALL}"
 
 QEMU_ARCH ?= "x86_64"
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251010151205.2122000-4-felix.moessbauer%40siemens.com.

Re: [RFC PATCH 3/3] qemuamd64: add IMAGER_BOM entries

['Jan Kiszka' via isar-users]

On 10.10.25 17:12, Felix Moessbauer wrote:
> To give an example how to add components to the imager BOM, we set the
> corresponding variable for the qemuamd64 target.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  meta-isar/conf/machine/qemuamd64.conf | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
> index 7d5987c6..8d0753a8 100644
> --- a/meta-isar/conf/machine/qemuamd64.conf
> +++ b/meta-isar/conf/machine/qemuamd64.conf
> @@ -7,6 +7,7 @@ KERNEL_NAME ?= "amd64"
>  
>  IMAGE_FSTYPES ?= "wic"
>  WKS_FILE ?= "sdimage-efi"
> +IMAGER_BOM:wic += "${GRUB_BOOTLOADER_INSTALL}"

Happens to work for grub as this variable only contains the EFI binaries
as it looks like. Hope people will understand that this is not a stable
pattern...

Jan

>  IMAGER_INSTALL:wic += "${GRUB_BOOTLOADER_INSTALL}"
>  
>  QEMU_ARCH ?= "x86_64"


-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/d2c81829-7115-41a3-a57c-7e1cbada8843%40siemens.com.