[PATCH v6 0/4] Deploy DTBs with separate recipe

[PATCH v6 0/4] Deploy DTBs with separate recipe

[Anton Mikanovich]

Image task "do_copy_boot_files" is used to deploy kernel, initrd and
devicetree binaries to deploy directory.

When different images for the same target are built (e.g., "-base" and
"-debug") in parallel, this causes issues with DTB deployment since
they have no any image_name-specific stuff in the path, unlike kernel
and initrd.

Here we make a separate recipe responsible for the DTB deployment.
DTBs are extracted from the kernel package (provided by distro or
self-built).

The recipe 'dtb-files' is MACHINE-dependent, so different DTB_FILES
may be deployed for different machines. At the same time, all these
'dtb-files-${MACHINE}' targets depend on the same kernel package, so
it is built only once.

The case when different distros (e.g., bullseye/bookworm) are using the
same name for DTB binaries is covered by the patch 3. It also covers
the case when image uses different kernel suffix (e.g., "realtime"
option for the kernel in some downstreams).

Related topics on maillist:

https://groups.google.com/g/isar-users/c/ht3YmKZcJlM
https://groups.google.com/g/isar-users/c/4BRDM02xC40
https://groups.google.com/g/isar-users/c/qAnjahjjfsw
https://groups.google.com/g/isar-users/c/ZMD4XY4dKWQ
https://groups.google.com/g/isar-users/c/PSGU_AcdPZ8
https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA
https://groups.google.com/g/isar-users/c/ixzytHYnTmI

Changes since v6:
- Fix isoimage-isohybrid-isar WIC plugin.
- Split testcases.
- Update wording.

Changes since v4:
- Rebase on next.
- Squash latest patch to first one.
- Fix deploy paths in initrd tests.

Changes since v3:
- Rebase on next.
- Fix target deploy dir path for installer.

Changes since v2:
- Patch 1: Consider also kernel suffix.
- Patch 2: Use separate testcase for possible dtb clashes.
- Minor cosmetic fixes.

Changes since v1:
- Separate `dtb-files` recipe used instead of linux one
- Included older "[PATCH] Fix do_copy_boot_files error" into the
series ( https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA )

Anton Mikanovich (2):
  wic: Obtain real machine name in isoimage source plugin
  testsuite: Add testcases to check dtb deployment

Ilia Skochilov (1):
  meta: Fix do_copy_boot_files error for different distros of same
    machine

Uladzimir Bely (1):
  meta: Deploy DTBs once per kernel

 RECIPE-API-CHANGELOG.md                       | 15 +++++++
 .../installer-add-rootfs.bbclass              |  4 +-
 meta/classes-recipe/image.bbclass             | 15 +++----
 meta/classes-recipe/imagetypes_wic.bbclass    |  2 +-
 meta/conf/bitbake.conf                        |  2 +-
 meta/recipes-kernel/dtb-files/dtb-files.bb    | 41 +++++++++++++++++++
 .../plugins/source/isoimage-isohybrid-isar.py |  4 +-
 testsuite/citest.py                           | 40 ++++++++++++++++--
 8 files changed, 106 insertions(+), 17 deletions(-)
 create mode 100644 meta/recipes-kernel/dtb-files/dtb-files.bb

-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260213074042.76277-1-amikan%40ilbers.de.

[PATCH v6 1/4] wic: Obtain real machine name in isoimage source plugin

[Anton Mikanovich]

The current implementation tries to obtain machine name from the deploy
directory name, which relies on hardcoded path format. Add MACHINE to
the WIC environment and read it in isoimage-isohybrid-isar.py to fix
that.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes-recipe/imagetypes_wic.bbclass                    | 2 +-
 .../scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index 6b82add3..d50022ce 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -107,7 +107,7 @@ WICVARS += "\
            ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS TRANSLATED_TARGET_ARCH"
 
 # Isar specific vars used in our plugins
-WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE"
+WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE MACHINE"
 
 python do_rootfs_wicenv () {
     wicvars = d.getVar('WICVARS')
diff --git a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
index fb6c6b39..0ed61fff 100644
--- a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
@@ -191,7 +191,9 @@ class IsoImagePlugin(SourcePlugin):
             if not image_type:
                 raise WicError("Couldn't find INITRAMFS_FSTYPES, exiting.")
 
-            machine = os.path.basename(initrd_dir)
+            machine = get_bitbake_var("MACHINE")
+            if not machine:
+                raise WicError("Couldn't find MACHINE, exiting.")
 
             pattern = '%s/%s*%s.%s' % (initrd_dir, image_name, machine, image_type)
             pattern = '%s/%s-%s-initrd.%s' % (initrd_dir, image_name, machine, image_type)
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260213074042.76277-2-amikan%40ilbers.de.

[PATCH v6 2/4] testsuite: Add testcases to check dtb deployment

[Anton Mikanovich]

Build several images with dtbs deployed to the same path.

 - test_dtb_deploy_distros

   Same machine, different distros, same image, same kernel, same dtb
   name and contents.

 - test_dtb_deploy_images

   Same machine, same distro, different images, same kernel, same dtb
   name and contents.

Both fail with:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
  build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
    (not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/citest.py | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index f11cd5af..c521f6fe 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -677,6 +677,39 @@ class WicTest(CIBaseTest):
         )
 
 
+class DtbDeployTest(CIBaseTest):
+
+    """
+    Build some targets potentially causing DTB deploy issues.
+
+    :avocado: tags=dtbdeploy,full
+    """
+
+    def test_dtb_deploy_distros(self):
+        """
+        Cover case: Same machine, different distros
+        """
+        targets = [
+            'mc:phyboard-mira-bullseye:isar-image-base',
+            'mc:phyboard-mira-bookworm:isar-image-base',
+        ]
+
+        self.init()
+        self.perform_build_test(targets)
+
+    def test_dtb_deploy_images(self):
+        """
+        Cover case: Same machine/distro, different images
+        """
+        targets = [
+            'mc:phyboard-mira-bookworm:isar-image-base',
+            'mc:phyboard-mira-bookworm:isar-image-ci',
+        ]
+
+        self.init()
+        self.perform_build_test(targets)
+
+
 class NoCrossTest(CIBaseTest):
 
     """
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260213074042.76277-3-amikan%40ilbers.de.

[PATCH v6 3/4] meta: Fix do_copy_boot_files error for different distros of same machine

[Anton Mikanovich]

From: Ilia Skochilov <iskochilov@ilbers.de>

When building different distros for the same machine (e.g.,
phyboard-mira-bullseye and phyboard-mira-bookworm), files with the same
name (e.g, DTB files) might be deployed to the same location and this
causes a build error.

Use DISTRO- and KERNEL-dependent deploy directory.

Also fix paths in installer and testsuite.

Fixes test_dtb_deploy_distros testcase:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
  build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
    (not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Ilia Skochilov <iskochilov@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Badrikesh Prusty <badrikesh.prusty@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 RECIPE-API-CHANGELOG.md                           | 15 +++++++++++++++
 .../classes-recipe/installer-add-rootfs.bbclass   |  4 +++-
 meta/conf/bitbake.conf                            |  2 +-
 testsuite/citest.py                               |  7 ++++---
 4 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 0bad8a44..6d7f12b2 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -962,3 +962,18 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
 # Optional: set countdown timeout in seconds (default 5)
 INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
 ```
+
+### Change DEPLOY_DIR_IMAGE path
+
+Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
+${DEPLOY_DIR}/images/${MACHINE}-${DISTRO}-${KERNEL_NAME}. When building
+different distros with the same machine the following error occurs:
+
+do_copy_boot_files: The recipe isar-image-base is trying to install files
+into a shared area when those files already exists. It happens when some
+files have the same names (e.g., dtb files) for different distros.
+
+To prevent this new path is separated also by distro and kernel values.
+
+This change will influence on build artifacts location and should be taken
+into account by downstreams.
diff --git a/meta-isar/classes-recipe/installer-add-rootfs.bbclass b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
index 2fa551bd..b79c2357 100644
--- a/meta-isar/classes-recipe/installer-add-rootfs.bbclass
+++ b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
@@ -13,7 +13,9 @@ INSTALLER_TARGET_IMAGES ??= "${INSTALLER_TARGET_IMAGE}"
 INSTALLER_TARGET_MC ??= "installer-target"
 INSTALLER_TARGET_DISTRO ??= "${DISTRO}"
 INSTALLER_TARGET_MACHINE ??= "${MACHINE}"
-INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}"
+INSTALLER_TARGET_KERNEL_NAME ??= "${KERNEL_NAME}"
+INSTALLER_TARGET_DEPLOY_DIR_POSTFIX ??= "${@ ('-'+d.getVar('INSTALLER_TARGET_KERNEL_NAME')) if d.getVar('INSTALLER_TARGET_KERNEL_NAME') else ''}"
+INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}-${INSTALLER_TARGET_DISTRO}${INSTALLER_TARGET_DEPLOY_DIR_POSTFIX}"
 
 IMAGE_DATA_FILE ??= "${INSTALLER_TARGET_IMAGE}-${INSTALLER_TARGET_DISTRO}-${INSTALLER_TARGET_MACHINE}"
 IMAGE_DATA_POSTFIX ??= "wic.zst"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0f84e715..ad3d6b53 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -57,7 +57,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
 GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
 DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
 DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
-DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
+DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}-${DISTRO}${@ ('-'+d.getVar('KERNEL_NAME')) if d.getVar('KERNEL_NAME') else ''}"
 DL_DIR ?= "${TOPDIR}/downloads"
 SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
 SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${DISTRO}-${DISTRO_ARCH}"
diff --git a/testsuite/citest.py b/testsuite/citest.py
index c521f6fe..072d8e92 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -496,8 +496,9 @@ class InitRdBaseTest(CIBaseTest):
         super().init()
         self.deploy_dir = os.path.join(self.build_dir, 'tmp', 'deploy')
 
-    def deploy_dir_image(self, machine):
-        return os.path.join(self.deploy_dir, 'images', machine)
+    def deploy_dir_image(self, mc, image):
+        multiconfig = f"{mc}:{image}"
+        return CIUtils.getVars('DEPLOY_DIR_IMAGE', target=multiconfig)
 
     def dracut_in_image(self, targets):
         machine = 'qemuamd64'
@@ -526,7 +527,7 @@ class InitRdBaseTest(CIBaseTest):
                                           bb_should_fail=False):
         mc = f'mc:{machine}-{distro.removeprefix("debian-")}'
         initrd_image = f'{initrd}-{distro}-{machine}-initrd.img'
-        initrd_path = os.path.join(self.deploy_dir_image(machine), initrd_image)
+        initrd_path = os.path.join(self.deploy_dir_image(mc, initrd), initrd_image)
 
         # cleansstate if the initrd image was already built/deployed to verify
         # that a new build does result in the image being deployed
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260213074042.76277-4-amikan%40ilbers.de.

[PATCH v6 4/4] meta: Deploy DTBs once per kernel

[Anton Mikanovich]

From: Uladzimir Bely <ubely@ilbers.de>

Task do_copy_boot_files deploys DTB files into the same location for
different images (e.g., isar-image-base and isar-image-ci). This causes
a build error.

do_copy_boot_files is called once for every image recipe while dtb files
belong to the kernel which is the same for both images. Performing dtb
deployment once for the same kernel solves the issue.

Introduce `dtb-files` recipe responsible for extracting the DTBs from
the linux-image package and deploying them. The kernel package is now
installed into sbuild chroot. This is achieved via the respective
dependency of `dtb-files` recipe.

Fixes test_dtb_deploy_images testcase:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
  build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
    (not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes-recipe/image.bbclass          | 15 +++-----
 meta/recipes-kernel/dtb-files/dtb-files.bb | 41 ++++++++++++++++++++++
 2 files changed, 46 insertions(+), 10 deletions(-)
 create mode 100644 meta/recipes-kernel/dtb-files/dtb-files.bb

diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index afe6906e..06b72b09 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -381,6 +381,11 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_DEPLOY_FILE}"
 # only one dtb file supported, pick the first
 DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"
 
+python() {
+    if d.getVar('DTB_FILES'):
+        d.appendVarFlag("do_copy_boot_files", "depends", "dtb-files-${MACHINE}:do_deploy")
+}
+
 do_copy_boot_files[cleandirs] += "${DEPLOYDIR}"
 do_copy_boot_files[sstate-inputdirs] = "${DEPLOYDIR}"
 do_copy_boot_files[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
@@ -394,16 +399,6 @@ do_copy_boot_files() {
         sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
     fi
 
-    for file in ${DTB_FILES}; do
-        dtb="$(find '${IMAGE_ROOTFS}/usr/lib' -type f \
-                    -iwholename '*linux-image-*/'${file} | head -1)"
-
-        if [ -z "$dtb" -o ! -e "$dtb" ]; then
-            die "${file} not found"
-        fi
-
-        cp -f "$dtb" "${DEPLOYDIR}/"
-    done
 }
 addtask copy_boot_files before do_rootfs_postprocess after do_rootfs_install
 
diff --git a/meta/recipes-kernel/dtb-files/dtb-files.bb b/meta/recipes-kernel/dtb-files/dtb-files.bb
new file mode 100644
index 00000000..6f1ef146
--- /dev/null
+++ b/meta/recipes-kernel/dtb-files/dtb-files.bb
@@ -0,0 +1,41 @@
+# This software is a part of Isar.
+# Copyright (C) ilbers GmbH, 2026
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg-raw
+
+PN:append = "-${MACHINE}"
+
+KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME")) if d.getVar("KERNEL_NAME") else ""}"
+
+DEPENDS = "${KERNEL_IMAGE_PKG}"
+DEBIAN_BUILD_DEPENDS = "${KERNEL_IMAGE_PKG}"
+
+DPKG_ARCH = "${PACKAGE_ARCH}"
+
+do_prepare_build:append() {
+    for dtb in ${DTB_FILES}; do
+        dir=debian/${PN}/usr/lib/${PN}/$(dirname ${dtb})
+        cat <<EOF >> ${S}/debian/rules
+	mkdir -p ${dir}
+	find /usr/lib/linux-image* -path "*${dtb}" -print -exec cp {} ${dir} \;
+EOF
+    done
+}
+
+DTB_PACKAGE ??= "${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb"
+
+do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
+do_deploy[cleandirs] = "${WORKDIR}/deploy"
+do_deploy() {
+    dpkg --fsys-tarfile ${WORKDIR}/${DTB_PACKAGE} | \
+    tar --wildcards --extract --directory ${WORKDIR}/deploy ./usr/lib/${PN}
+    for dtb in ${DTB_FILES}; do
+        mkdir -p ${DEPLOY_DIR_IMAGE}/$(dirname ${dtb})
+        find ${WORKDIR}/deploy/usr/lib/${PN} -path "*${dtb}" -print \
+            -exec cp {} ${DEPLOY_DIR_IMAGE}/${dtb} \;
+    done
+}
+
+addtask deploy before do_deploy_deb after do_dpkg_build
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260213074042.76277-5-amikan%40ilbers.de.

Re: [PATCH v6 4/4] meta: Deploy DTBs once per kernel

['Jan Kiszka' via isar-users]

On 13.02.26 08:40, Anton Mikanovich wrote:
> From: Uladzimir Bely <ubely@ilbers.de>
> 
> Task do_copy_boot_files deploys DTB files into the same location for
> different images (e.g., isar-image-base and isar-image-ci). This causes
> a build error.
> 
> do_copy_boot_files is called once for every image recipe while dtb files
> belong to the kernel which is the same for both images. Performing dtb
> deployment once for the same kernel solves the issue.
> 

This remains inconsistent: DTBs are just another artifact of the kernel.
The kernel image itself is also deployed via the image recipe. Why is
there no conflict is because of image-specific file names being used. We
can't change the DTB file names, but we could put them in image-specific
deployment folders, for the case that multiple images are pushed into
the same deployment folder.

Again: NAK, wrong approach, please finally process my feedback.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/00517f61-1ea9-4281-a2ee-a3d572f3a169%40siemens.com.

Re: [PATCH v6 4/4] meta: Deploy DTBs once per kernel

[Baurzhan Ismagulov]

On 2026-02-13 09:01, 'Jan Kiszka' via isar-users wrote:
> > Task do_copy_boot_files deploys DTB files into the same location for
> > different images (e.g., isar-image-base and isar-image-ci). This causes
> > a build error.
> > 
> > do_copy_boot_files is called once for every image recipe while dtb files
> > belong to the kernel which is the same for both images. Performing dtb
> > deployment once for the same kernel solves the issue.
> 
> This remains inconsistent: DTBs are just another artifact of the kernel.
> The kernel image itself is also deployed via the image recipe. Why is
> there no conflict is because of image-specific file names being used. We
> can't change the DTB file names, but we could put them in image-specific
> deployment folders, for the case that multiple images are pushed into
> the same deployment folder.
> 
> Again: NAK, wrong approach, please finally process my feedback.

v6 has been sent for documenting the reasons in the archive as previously
announced; I think we can solve this without escalating.

Now that you've shared your proposal of splitting by image -- that's an option
and we can discuss it.

Our use case is:
1. Two images, same dtb name, same dtb contents.
2. Downstream already has many deliverables.

My underlying question: Should we always duplicate, or should we at least try
to identify cases where we can avoid duplication? "Same file -- single
deployment".

Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
producing it, reducing shuffling around in the image recipe. Yes, the kernel is
still deployed in the image which is inconsistent at this step; I'd be fine
doing that later.

We'll try playing with splitting by image. Even if we end up doing it at this
time, I'd still propose to explore the other options step by step.

With kind regards,
Baurzhan

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/aZNQpq3wXA3HLJjG%40abai.de.

Re: [PATCH v6 4/4] meta: Deploy DTBs once per kernel

['Jan Kiszka' via isar-users]

On 16.02.26 18:15, Baurzhan Ismagulov wrote:
> On 2026-02-13 09:01, 'Jan Kiszka' via isar-users wrote:
>>> Task do_copy_boot_files deploys DTB files into the same location for
>>> different images (e.g., isar-image-base and isar-image-ci). This causes
>>> a build error.
>>>
>>> do_copy_boot_files is called once for every image recipe while dtb files
>>> belong to the kernel which is the same for both images. Performing dtb
>>> deployment once for the same kernel solves the issue.
>>
>> This remains inconsistent: DTBs are just another artifact of the kernel.
>> The kernel image itself is also deployed via the image recipe. Why is
>> there no conflict is because of image-specific file names being used. We
>> can't change the DTB file names, but we could put them in image-specific
>> deployment folders, for the case that multiple images are pushed into
>> the same deployment folder.
>>
>> Again: NAK, wrong approach, please finally process my feedback.
> 
> v6 has been sent for documenting the reasons in the archive as previously
> announced; I think we can solve this without escalating.
> 
> Now that you've shared your proposal of splitting by image -- that's an option
> and we can discuss it.
> 
> Our use case is:
> 1. Two images, same dtb name, same dtb contents.

Which is just the tip of this issue. Other layers deploy bootloader
files or firmware binaries under the same name, overwriting things right
now if two images are built under different names. And that overwriting
sometimes triggers the infamous bitbake deployment errors.

> 2. Downstream already has many deliverables.
> 
> My underlying question: Should we always duplicate, or should we at least try
> to identify cases where we can avoid duplication? "Same file -- single
> deployment".

How would you switch between de-duplicated and different artifacts by
name then? How would imagers or other users find deployed artifacts in
both cases?

> 
> Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
> producing it, reducing shuffling around in the image recipe. Yes, the kernel is
> still deployed in the image which is inconsistent at this step; I'd be fine
> doing that later.
> 
> We'll try playing with splitting by image. Even if we end up doing it at this
> time, I'd still propose to explore the other options step by step.

Please not so many user-visible changes "step by step". We need a design
that holds now, and we need one step to get there, at least when that
step implies path changes that downstream layers need to adjust to.

I guess we should rather introduce the pattern of creating a subfolder
${IMAGE_FULLNAME} and put all those potentially colliding artifacts there.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/743de9ab-12f8-4e8d-b6b5-b3b8af1e7482%40siemens.com.

Re: [PATCH v6 4/4] meta: Deploy DTBs once per kernel

[Baurzhan Ismagulov]

On 2026-02-16 18:51, 'Jan Kiszka' via isar-users wrote:
> > Our use case is:
> > 1. Two images, same dtb name, same dtb contents.
> 
> Which is just the tip of this issue. Other layers deploy bootloader
> files or firmware binaries under the same name, overwriting things right
> now if two images are built under different names. And that overwriting
> sometimes triggers the infamous bitbake deployment errors.

Thanks for the detailed elaboration. If the contents are the same, v6 patch 4
would handle it. The "same name, different contents" case we couldn't identify
from the past threads and the downstreams we see, so we had left it for later
when it would actually occur (often the real cases are different from what one
anticipates).

If "same name, different contents" is already the case today, duplicating is
the only option.

I'm a bit surprised that the errors occur only sometimes -- we can reproduce
them reliably given the right mix of build targets.


> > 2. Downstream already has many deliverables.
> > 
> > My underlying question: Should we always duplicate, or should we at least try
> > to identify cases where we can avoid duplication? "Same file -- single
> > deployment".
> 
> How would you switch between de-duplicated and different artifacts by
> name then? How would imagers or other users find deployed artifacts in
> both cases?

Yes, that would depend on the contents and might be counter-intuitive at times.
Maybe duplicating and having the complete sets for every machine would be the
easiest to understand for the end users. We need to see how it works in
practice.


> > Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
> > producing it, reducing shuffling around in the image recipe. Yes, the kernel is
> > still deployed in the image which is inconsistent at this step; I'd be fine
> > doing that later.
> > 
> > We'll try playing with splitting by image. Even if we end up doing it at this
> > time, I'd still propose to explore the other options step by step.
> 
> Please not so many user-visible changes "step by step". We need a design
> that holds now, and we need one step to get there, at least when that
> step implies path changes that downstream layers need to adjust to.

I agree -- this was the goal of v6 patch 4 -- it doesn't change the path if it
is not necessary, and a similar kernel consolidation also wouldn't change the
path.


> I guess we should rather introduce the pattern of creating a subfolder
> ${IMAGE_FULLNAME} and put all those potentially colliding artifacts there.

We'll send v7 with duplication.


Currently, some artifacts are duplicated via file names. Since we already break
the API, we could touch them as well. Opinions welcome.

1. Before the change:

   mc:phyboard-mira-bookworm:isar-image-base

   phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb                                  # bitbake conflict
   phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
   phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz

   mc:phyboard-mira-bookworm:isar-image-ci

   phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb                                  # bitbake conflict
   phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-initrd.img
   phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-vmlinuz

2. After the duplication e.g.:

   phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb  # Same contents in our testcase
   phyboard-mira/debian-bookworm-isar-image-ci/imx6q-phytec-mira-rdk-nand.dtb    # Same contents in our testcase
   phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img        # Currently not in the subdir
   phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz           # Currently not in the subdir
   phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-initrd.img          # Currently not in the subdir
   phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-vmlinuz             # Currently not in the subdir

   Or we could move the kernels and initrds into the subdir as well:

   phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb
   phyboard-mira/debian-bookworm-isar-image-base/initrd.img
   phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
   phyboard-mira/debian-bookworm-isar-image-ci/imx6q-phytec-mira-rdk-nand.dtb
   phyboard-mira/debian-bookworm-isar-image-ci/initrd.img
   phyboard-mira/debian-bookworm-isar-image-ci/vmlinuz


With kind regards,
Baurzhan

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/aZXm9bM3vhRuQDE4%40abai.de.

Re: [PATCH v6 4/4] meta: Deploy DTBs once per kernel

['Jan Kiszka' via isar-users]

On 18.02.26 17:21, Baurzhan Ismagulov wrote:
> On 2026-02-16 18:51, 'Jan Kiszka' via isar-users wrote:
>>> Our use case is:
>>> 1. Two images, same dtb name, same dtb contents.
>>
>> Which is just the tip of this issue. Other layers deploy bootloader
>> files or firmware binaries under the same name, overwriting things right
>> now if two images are built under different names. And that overwriting
>> sometimes triggers the infamous bitbake deployment errors.
> 
> Thanks for the detailed elaboration. If the contents are the same, v6 patch 4
> would handle it. The "same name, different contents" case we couldn't identify
> from the past threads and the downstreams we see, so we had left it for later
> when it would actually occur (often the real cases are different from what one
> anticipates).
> 
> If "same name, different contents" is already the case today, duplicating is
> the only option.
> 
> I'm a bit surprised that the errors occur only sometimes -- we can reproduce
> them reliably given the right mix of build targets.
> 

The massive scale of multiconfig targets that isar has is not
representative. I know the error primarily from reconfigured rebuilds.

> 
>>> 2. Downstream already has many deliverables.
>>>
>>> My underlying question: Should we always duplicate, or should we at least try
>>> to identify cases where we can avoid duplication? "Same file -- single
>>> deployment".
>>
>> How would you switch between de-duplicated and different artifacts by
>> name then? How would imagers or other users find deployed artifacts in
>> both cases?
> 
> Yes, that would depend on the contents and might be counter-intuitive at times.
> Maybe duplicating and having the complete sets for every machine would be the
> easiest to understand for the end users. We need to see how it works in
> practice.
> 
> 
>>> Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
>>> producing it, reducing shuffling around in the image recipe. Yes, the kernel is
>>> still deployed in the image which is inconsistent at this step; I'd be fine
>>> doing that later.
>>>
>>> We'll try playing with splitting by image. Even if we end up doing it at this
>>> time, I'd still propose to explore the other options step by step.
>>
>> Please not so many user-visible changes "step by step". We need a design
>> that holds now, and we need one step to get there, at least when that
>> step implies path changes that downstream layers need to adjust to.
> 
> I agree -- this was the goal of v6 patch 4 -- it doesn't change the path if it
> is not necessary, and a similar kernel consolidation also wouldn't change the
> path.

Having inconsistent paths would be highly counterintuitive. Better have
stable ones and live with duplications. We may already duplicate kernels
and initramfs images, and both are much larger some normal DTBs.

> 
> 
>> I guess we should rather introduce the pattern of creating a subfolder
>> ${IMAGE_FULLNAME} and put all those potentially colliding artifacts there.
> 
> We'll send v7 with duplication.
> 
> 
> Currently, some artifacts are duplicated via file names. Since we already break
> the API, we could touch them as well. Opinions welcome.
> 
> 1. Before the change:
> 
>    mc:phyboard-mira-bookworm:isar-image-base
> 
>    phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb                                  # bitbake conflict
>    phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
>    phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz
> 
>    mc:phyboard-mira-bookworm:isar-image-ci
> 
>    phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb                                  # bitbake conflict
>    phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-initrd.img
>    phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-vmlinuz
> 
> 2. After the duplication e.g.:
> 
>    phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb  # Same contents in our testcase
>    phyboard-mira/debian-bookworm-isar-image-ci/imx6q-phytec-mira-rdk-nand.dtb    # Same contents in our testcase
>    phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img        # Currently not in the subdir
>    phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz           # Currently not in the subdir
>    phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-initrd.img          # Currently not in the subdir
>    phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-vmlinuz             # Currently not in the subdir
> 
>    Or we could move the kernels and initrds into the subdir as well:
> 
>    phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb
>    phyboard-mira/debian-bookworm-isar-image-base/initrd.img
>    phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
>    phyboard-mira/debian-bookworm-isar-image-ci/imx6q-phytec-mira-rdk-nand.dtb
>    phyboard-mira/debian-bookworm-isar-image-ci/initrd.img
>    phyboard-mira/debian-bookworm-isar-image-ci/vmlinuz

Moving kernel and initramfs would make things more consistent, indeed.

Only thing to double-check first would be if we are already completely
off compared to how OE deploys or if we can follow a pattern there. If
are off, I would go for this consistent relocation. If not, we should
check if (re-)aligning would both solve our issues and make things
simpler for people coming from that world.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/a217047e-5200-4283-8a22-25cfe2c931db%40siemens.com.