Deploy DTBs and other image artifacts into subdirs to avoid conflicts

Deploy DTBs and other image artifacts into subdirs to avoid conflicts

[Zhihang Wei]

Image task "do_copy_boot_files" deploys kernel, initrd, and device tree
binaries to the deploy directory.

When different images for the same target (e.g., "-base" and "-debug")
are built in parallel, DTB deployment may fail because DTBs do not
contain image-specific components in their filenames, unlike the kernel
and initrd. As a result, artifacts from different builds may be written
to the same location.

Here we deploy the DTBs and other image artifacts to subdirectories,
with names of distro image encoded to the path.

Note that this results in separate copies of the same artifacts being
deployed for each image build.

Related topics on the mailing list:
https://lists.isar-build.org/isar-users/20250403170701.807745-1-badrikesh.prusty@siemens.com/T/#u
https://lists.isar-build.org/isar-users/20240705071856.904-1-cedric.hombourger@siemens.com/T/#u
https://lists.isar-build.org/isar-users/c426ad86-5cc5-42e6-8223-e7a7e5b3424en@googlegroups.com/T/#u
https://lists.isar-build.org/isar-users/20240416100746.5681-1-nicusor_huhulea@mentor.com/T/#u
https://lists.isar-build.org/isar-users/DB3PR10MB690820CF3DD2DCA71D822AF1E63B2@DB3PR10MB6908.EURPRD10.PROD.OUTLOOK.COM/T/#u
https://lists.isar-build.org/isar-users/DB3PR10MB690861E70EB246FEF845E253E64B2@DB3PR10MB6908.EURPRD10.PROD.OUTLOOK.COM/T/#u
https://lists.isar-build.org/isar-users/20240131144459.2386220-1-iskochilov@ilbers.de/T/#u

Changes since v6:
- Deploy DTBs and other image artifacts into distro- and
  image-specific subdirs.
- Shorten image artifact filenames.

Changes since v5:
- Fix isoimage-isohybrid-isar WIC plugin.
- Split testcases.
- Update wording.

Changes since v4:
- Rebase on next.
- Squash latest patch to first one.
- Fix deploy paths in initrd tests.

Changes since v3:
- Rebase on next.
- Fix target deploy dir path for installer.

Changes since v2:
- Patch 1: Consider also kernel suffix.
- Patch 2: Use separate testcase for possible dtb clashes.
- Minor cosmetic fixes.

Changes since v1:
- Separate `dtb-files` recipe used instead of linux one.
- Included older "[PATCH] Fix do_copy_boot_files error" into the
  series.

Anton Mikanovich (2):
  wic: Obtain real machine name in isoimage source plugin
  testsuite: Add testcases to check dtb deployment

Ilia Skochilov (1):
  meta: Deploy image build artifacts into distro- and image-specific
    subdirs

 RECIPE-API-CHANGELOG.md                       | 63 +++++++++++++++++++
 .../installer-add-rootfs.bbclass              |  9 +--
 meta/classes-recipe/image.bbclass             | 11 ++--
 .../imagetypes_container.bbclass              |  2 +-
 meta/classes-recipe/imagetypes_wic.bbclass    |  2 +-
 meta/conf/bitbake.conf                        |  3 +-
 .../plugins/source/isoimage-isohybrid-isar.py |  6 +-
 testsuite/cibase.py                           |  2 +-
 testsuite/citest.py                           | 40 +++++++++++-
 testsuite/start_vm.py                         |  2 +-
 10 files changed, 121 insertions(+), 19 deletions(-)

-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260226162613.1954014-1-wzh%40ilbers.de.

[PATCH v7 1/3] wic: Obtain real machine name in isoimage source plugin

[Zhihang Wei]

From: Anton Mikanovich <amikan@ilbers.de>

The current implementation tries to obtain machine name from the deploy
directory name, which relies on hardcoded path format. Add MACHINE to
the WIC environment and read it in isoimage-isohybrid-isar.py to fix
that.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes-recipe/imagetypes_wic.bbclass                    | 2 +-
 .../scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index 6b82add3..d50022ce 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -107,7 +107,7 @@ WICVARS += "\
            ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS TRANSLATED_TARGET_ARCH"
 
 # Isar specific vars used in our plugins
-WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE"
+WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE MACHINE"
 
 python do_rootfs_wicenv () {
     wicvars = d.getVar('WICVARS')
diff --git a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
index fb6c6b39..0ed61fff 100644
--- a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
@@ -191,7 +191,9 @@ class IsoImagePlugin(SourcePlugin):
             if not image_type:
                 raise WicError("Couldn't find INITRAMFS_FSTYPES, exiting.")
 
-            machine = os.path.basename(initrd_dir)
+            machine = get_bitbake_var("MACHINE")
+            if not machine:
+                raise WicError("Couldn't find MACHINE, exiting.")
 
             pattern = '%s/%s*%s.%s' % (initrd_dir, image_name, machine, image_type)
             pattern = '%s/%s-%s-initrd.%s' % (initrd_dir, image_name, machine, image_type)
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260226162613.1954014-2-wzh%40ilbers.de.

[PATCH v7 2/3] testsuite: Add testcases to check dtb deployment

[Zhihang Wei]

From: Anton Mikanovich <amikan@ilbers.de>

Build several images with dtbs deployed to the same path.

 - test_dtb_deploy_distros

   Same machine, different distros, same image, same kernel, same dtb
   name and contents.

 - test_dtb_deploy_images

   Same machine, same distro, different images, same kernel, same dtb
   name and contents.

Both fail with:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
  build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
    (not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/citest.py | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index ab28b68e..ca701422 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -677,6 +677,39 @@ class WicTest(CIBaseTest):
         )
 
 
+class DtbDeployTest(CIBaseTest):
+
+    """
+    Build some targets potentially causing DTB deploy issues.
+
+    :avocado: tags=dtbdeploy,full
+    """
+
+    def test_dtb_deploy_distros(self):
+        """
+        Cover case: Same machine, different distros
+        """
+        targets = [
+            'mc:phyboard-mira-bullseye:isar-image-base',
+            'mc:phyboard-mira-bookworm:isar-image-base',
+        ]
+
+        self.init()
+        self.perform_build_test(targets)
+
+    def test_dtb_deploy_images(self):
+        """
+        Cover case: Same machine/distro, different images
+        """
+        targets = [
+            'mc:phyboard-mira-bookworm:isar-image-base',
+            'mc:phyboard-mira-bookworm:isar-image-ci',
+        ]
+
+        self.init()
+        self.perform_build_test(targets)
+
+
 class NoCrossTest(CIBaseTest):
 
     """
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260226162613.1954014-3-wzh%40ilbers.de.

[PATCH v7 3/3] meta: Deploy image build artifacts into distro- and image-specific subdirs

[Zhihang Wei]

From: Ilia Skochilov <iskochilov@ilbers.de>

When building different distros or images for the same machine (e.g.
phyboard-mira-bullseye and phyboard-mira-bookworm, or isar-image-ci
and isar-image-base), artifacts with identical names (e.g. DTB files)
may be deployed to the same location, resulting file conflicts and
build failures.

Deploy image build artifacts into a subdirectory named
${DISTRO}-${IMAGE_PN} to avoid such collisions.

Since the directory now encodes the distro and image name, remove the
${DISTRO} and ${IMAGE_PN} prefixes from artifact filenames to
keep the names short and clear.

Fixes test_dtb_deploy_distros testcase:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
  build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
    (not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Ilia Skochilov <iskochilov@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Badrikesh Prusty <badrikesh.prusty@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Zhihang Wei <wzh@ilbers.de>
---
 RECIPE-API-CHANGELOG.md                       | 63 +++++++++++++++++++
 .../installer-add-rootfs.bbclass              |  9 +--
 meta/classes-recipe/image.bbclass             | 11 ++--
 .../imagetypes_container.bbclass              |  2 +-
 meta/conf/bitbake.conf                        |  3 +-
 .../plugins/source/isoimage-isohybrid-isar.py |  2 +-
 testsuite/cibase.py                           |  2 +-
 testsuite/citest.py                           |  7 ++-
 testsuite/start_vm.py                         |  2 +-
 9 files changed, 84 insertions(+), 17 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 0bad8a44..bc7abb37 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -962,3 +962,66 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
 # Optional: set countdown timeout in seconds (default 5)
 INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
 ```
+
+### Change DEPLOY_DIR_IMAGE path and artifacts naming
+
+Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
+${DEPLOY_DIR}/images/${MACHINE}/${DISTRO}-${IMAGE_PN}.
+
+When building different distros with the same machine the following
+error occurs:
+
+do_copy_boot_files: The recipe isar-image-base is trying to install
+files into a shared area when those files already exists. It happens
+when some files have the same names (e.g., dtb files) for different
+distros.
+
+To prevent such collisions, image artifacts are now deployed into a
+distro- and image-specific subdirectory.
+
+Additionally, artifact filenames have been shortened by removing the
+${DISTRO} and ${IMAGE_PN} prefix, since this information is now
+encoded in the directory path.
+
+Example 1: Build isar-image-base (phyboard-mira, debian-bookworm)
+Under "build/tmp/deploy/images/":
+Before:
+phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
+phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
+phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz
+phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.dpkg_status
+phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.ubi
+
+After:
+phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb
+phyboard-mira/debian-bookworm-isar-image-base/initrd.img
+phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
+phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.dpkg_status
+phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.ubi
+
+Example 2: Build isar-image-ci (qemuamd64, debian-bookworm)
+Under "build/tmp/deploy/images/":
+Before:
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-initrd.img
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-vmlinuz
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.dpkg_status
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.manifest
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.bmap
+qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.manifest
+
+After:
+qemuamd64/debian-bookworm-isar-image-ci/initrd.img
+qemuamd64/debian-bookworm-isar-image-ci/vmlinuz
+qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.dpkg_status
+qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.manifest
+qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic
+qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.bmap
+qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.manifest
+
+Artifacts that do not belong to a full image (e.g. isar-image-base,
+isar-image-ci) remain unchanged. For example, a customized initramfs
+built independently is not affected.
+
+This change affects the location and naming of build artifacts and must
+be taken into account by downstream users.
diff --git a/meta-isar/classes-recipe/installer-add-rootfs.bbclass b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
index 2fa551bd..edf93016 100644
--- a/meta-isar/classes-recipe/installer-add-rootfs.bbclass
+++ b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
@@ -13,9 +13,10 @@ INSTALLER_TARGET_IMAGES ??= "${INSTALLER_TARGET_IMAGE}"
 INSTALLER_TARGET_MC ??= "installer-target"
 INSTALLER_TARGET_DISTRO ??= "${DISTRO}"
 INSTALLER_TARGET_MACHINE ??= "${MACHINE}"
-INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}"
+INSTALLER_TARGET_IMAGE ??= "${IMAGE_PN}"
+INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}/${INSTALLER_TARGET_DISTRO}-${INSTALLER_TARGET_IMAGE}"
 
-IMAGE_DATA_FILE ??= "${INSTALLER_TARGET_IMAGE}-${INSTALLER_TARGET_DISTRO}-${INSTALLER_TARGET_MACHINE}"
+IMAGE_DATA_FILE ??= "${INSTALLER_TARGET_MACHINE}"
 IMAGE_DATA_POSTFIX ??= "wic.zst"
 IMAGE_DATA_POSTFIX:buster ??= "wic.xz"
 IMAGE_DATA_POSTFIX:bullseye ??= "wic.xz"
@@ -29,7 +30,7 @@ def get_installer_sources(d, suffix):
     target_machine = d.getVar('INSTALLER_TARGET_MACHINE')
     sources = []
     for image in installer_target_images:
-        image_data = f"{image}-{target_distro}-{target_machine}"
+        image_data = f"{target_machine}"
         sources.append(f"{target_deploy_dir}/{image_data}.{suffix}")
     return sources
 
@@ -41,7 +42,7 @@ def get_installer_destinations(d, suffix):
     target_machine = d.getVar('INSTALLER_TARGET_MACHINE')
     dests = []
     for image in installer_target_images:
-        image_data = f"{image}-{target_distro}-{target_machine}"
+        image_data = f"{target_machine}"
         dests.append(f"/install/{image_data}.{suffix}")
     return dests
 
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index afe6906e..d4f9e2c0 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -18,8 +18,9 @@ IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
 KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME")) if d.getVar("KERNEL_NAME") else ""}"
 IMAGE_INSTALL += "${KERNEL_IMAGE_PKG}"
 
-# Name of the image including distro&machine names
-IMAGE_FULLNAME = "${PN}-${DISTRO}-${MACHINE}"
+# Name the image as the machine name only, since the path includes distro name now
+IMAGE_FULLNAME = "${MACHINE}"
+IMAGE_PN = "${PN}"
 
 # Deprecated; this would be set to e.g. "${INITRAMFS_RECIPE}-${DISTRO}-${MACHINE}-initrd.img"
 INITRD_IMAGE ?= ""
@@ -29,7 +30,7 @@ INITRD_IMAGE ?= ""
 IMAGE_INITRD ?= ""
 
 # Name of the deployed initrd image
-INITRD_DEPLOY_FILE = "${@ d.getVar('IMAGE_INITRD') or '${PN}' }-${DISTRO}-${MACHINE}-initrd.img"
+INITRD_DEPLOY_FILE = "initrd.img"
 
 # Make sure dependent initramfs recipe is built
 do_image[depends] += "${@ '${IMAGE_INITRD}:do_build' if '${IMAGE_INITRD}' else '' }"
@@ -52,7 +53,7 @@ python() {
 ROOTFS_FEATURES += "${@ 'generate-initrd' if (d.getVar('INITRD_IMAGE') == '' and d.getVar('IMAGE_INITRD') == '') else ''}"
 
 # This variable is used by wic and start_vm
-KERNEL_IMAGE ?= "${IMAGE_FULLNAME}-${KERNEL_FILE}"
+KERNEL_IMAGE ?= "${KERNEL_FILE}"
 
 # This defines the deployed dtbs for reuse by imagers
 DTB_FILES ?= ""
@@ -108,7 +109,7 @@ ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${@isar_multiarch_packages('IMAGE_INSTAL
 ROOTFS_VARDEPS += "IMAGE_INSTALL"
 ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
 ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
-ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
+ROOTFS_PACKAGE_SUFFIX ?= "${MACHINE}"
 
 CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', '1', '0', d)}"
 python () {
diff --git a/meta/classes-recipe/imagetypes_container.bbclass b/meta/classes-recipe/imagetypes_container.bbclass
index ea15decf..1c5386a8 100644
--- a/meta/classes-recipe/imagetypes_container.bbclass
+++ b/meta/classes-recipe/imagetypes_container.bbclass
@@ -9,7 +9,7 @@
 CONTAINER_TYPES = "oci-archive docker-archive docker-daemon containers-storage"
 USING_CONTAINER = "${@bb.utils.contains_any('IMAGE_BASETYPES', d.getVar('CONTAINER_TYPES').split(), '1', '0', d)}"
 
-CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
+CONTAINER_IMAGE_NAME ?= "container-${DISTRO_ARCH}"
 CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
 CONTAINER_IMAGE_CMD ?= "/bin/dash"
 CONTAINER_IMAGE_ENTRYPOINT ?= ""
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0f84e715..7c544e5d 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -57,7 +57,8 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
 GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
 DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
 DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
-DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
+IMAGE_PN ?= ""
+DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}${@('/%s-%s' % (d.getVar('DISTRO'), d.getVar('IMAGE_PN'))) if d.getVar('IMAGE_PN') != '' else ''}"
 DL_DIR ?= "${TOPDIR}/downloads"
 SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
 SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${DISTRO}-${DISTRO_ARCH}"
diff --git a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
index 0ed61fff..eaef9c79 100644
--- a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
@@ -196,7 +196,7 @@ class IsoImagePlugin(SourcePlugin):
                 raise WicError("Couldn't find MACHINE, exiting.")
 
             pattern = '%s/%s*%s.%s' % (initrd_dir, image_name, machine, image_type)
-            pattern = '%s/%s-%s-initrd.%s' % (initrd_dir, image_name, machine, image_type)
+            pattern = '%s/initrd.%s' % (initrd_dir, image_type)
             files = glob.glob(pattern)
             if files:
                 initrd = files[0]
diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index fd6a3df9..fac2ab75 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -44,7 +44,7 @@ class CIBaseTest(CIBuilder):
         self.configure(wic_deploy_parts=wic_deploy_parts, **kwargs)
         self.bitbake(targets, **kwargs)
 
-        wic_path = f"{self.build_dir}/tmp/deploy/images/*/*.wic.p1"
+        wic_path = f"{self.build_dir}/tmp/deploy/images/*/*/*.wic.p1"
         partition_files = set(glob.glob(wic_path))
         if wic_deploy_parts and len(partition_files) == 0:
             self.fail("Found raw wic partitions in DEPLOY_DIR")
diff --git a/testsuite/citest.py b/testsuite/citest.py
index ca701422..f36da786 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -496,8 +496,9 @@ class InitRdBaseTest(CIBaseTest):
         super().init()
         self.deploy_dir = os.path.join(self.build_dir, 'tmp', 'deploy')
 
-    def deploy_dir_image(self, machine):
-        return os.path.join(self.deploy_dir, 'images', machine)
+    def deploy_dir_image(self, mc, image):
+        multiconfig = f"{mc}:{image}"
+        return CIUtils.getVars('DEPLOY_DIR_IMAGE', target=multiconfig)
 
     def dracut_in_image(self, targets):
         machine = 'qemuamd64'
@@ -526,7 +527,7 @@ class InitRdBaseTest(CIBaseTest):
                                           bb_should_fail=False):
         mc = f'mc:{machine}-{distro.removeprefix("debian-")}'
         initrd_image = f'{initrd}-{distro}-{machine}-initrd.img'
-        initrd_path = os.path.join(self.deploy_dir_image(machine), initrd_image)
+        initrd_path = os.path.join(self.deploy_dir_image(mc, initrd), initrd_image)
 
         # cleansstate if the initrd image was already built/deployed to verify
         # that a new build does result in the image being deployed
diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index 57d169df..a37ffe82 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -50,7 +50,7 @@ def format_qemu_cmdline(
     image_type = image_fstypes.split()[0]
     base = 'ubuntu' if distro in ['jammy', 'focal', 'noble'] else 'debian'
 
-    rootfs_image = f"{image}-{base}-{distro}-qemu{arch}.{image_type}"
+    rootfs_image = f"qemu{arch}.{image_type}"
 
     if image_type == 'ext4':
         kernel_image = deploy_dir_image + '/' + kernel_image
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260226162613.1954014-4-wzh%40ilbers.de.

Re: [PATCH v7 3/3] meta: Deploy image build artifacts into distro- and image-specific subdirs

['Jan Kiszka' via isar-users]

On 26.02.26 17:26, Zhihang Wei wrote:
> From: Ilia Skochilov <iskochilov@ilbers.de>
> 
> When building different distros or images for the same machine (e.g.
> phyboard-mira-bullseye and phyboard-mira-bookworm, or isar-image-ci
> and isar-image-base), artifacts with identical names (e.g. DTB files)
> may be deployed to the same location, resulting file conflicts and
> build failures.
> 
> Deploy image build artifacts into a subdirectory named
> ${DISTRO}-${IMAGE_PN} to avoid such collisions.
> 
> Since the directory now encodes the distro and image name, remove the
> ${DISTRO} and ${IMAGE_PN} prefixes from artifact filenames to
> keep the names short and clear.
> 
> Fixes test_dtb_deploy_distros testcase:
> 
> ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
> NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
>   build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>     (not matched to any task)
> Please verify which recipe should provide the above files.
> 
> Signed-off-by: Ilia Skochilov <iskochilov@ilbers.de>
> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> Signed-off-by: Badrikesh Prusty <badrikesh.prusty@siemens.com>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>

Were really all those people involved in developing and signing off this
change?

> Signed-off-by: Zhihang Wei <wzh@ilbers.de>
> ---
>  RECIPE-API-CHANGELOG.md                       | 63 +++++++++++++++++++
>  .../installer-add-rootfs.bbclass              |  9 +--
>  meta/classes-recipe/image.bbclass             | 11 ++--
>  .../imagetypes_container.bbclass              |  2 +-
>  meta/conf/bitbake.conf                        |  3 +-
>  .../plugins/source/isoimage-isohybrid-isar.py |  2 +-
>  testsuite/cibase.py                           |  2 +-
>  testsuite/citest.py                           |  7 ++-
>  testsuite/start_vm.py                         |  2 +-
>  9 files changed, 84 insertions(+), 17 deletions(-)
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 0bad8a44..bc7abb37 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -962,3 +962,66 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
>  # Optional: set countdown timeout in seconds (default 5)
>  INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
>  ```
> +
> +### Change DEPLOY_DIR_IMAGE path and artifacts naming
> +
> +Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
> +${DEPLOY_DIR}/images/${MACHINE}/${DISTRO}-${IMAGE_PN}.
> +
> +When building different distros with the same machine the following
> +error occurs:
> +
> +do_copy_boot_files: The recipe isar-image-base is trying to install
> +files into a shared area when those files already exists. It happens
> +when some files have the same names (e.g., dtb files) for different
> +distros.
> +
> +To prevent such collisions, image artifacts are now deployed into a
> +distro- and image-specific subdirectory.
> +
> +Additionally, artifact filenames have been shortened by removing the
> +${DISTRO} and ${IMAGE_PN} prefix, since this information is now
> +encoded in the directory path.
> +
> +Example 1: Build isar-image-base (phyboard-mira, debian-bookworm)
> +Under "build/tmp/deploy/images/":
> +Before:
> +phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz
> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.dpkg_status
> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.ubi
> +
> +After:
> +phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb
> +phyboard-mira/debian-bookworm-isar-image-base/initrd.img
> +phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.dpkg_status
> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.ubi
> +
> +Example 2: Build isar-image-ci (qemuamd64, debian-bookworm)
> +Under "build/tmp/deploy/images/":
> +Before:
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-initrd.img
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-vmlinuz
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.dpkg_status
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.manifest
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.bmap
> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.manifest
> +
> +After:
> +qemuamd64/debian-bookworm-isar-image-ci/initrd.img
> +qemuamd64/debian-bookworm-isar-image-ci/vmlinuz
> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.dpkg_status
> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.manifest
> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic
> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.bmap
> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.manifest
> +
> +Artifacts that do not belong to a full image (e.g. isar-image-base,
> +isar-image-ci) remain unchanged. For example, a customized initramfs
> +built independently is not affected.
> +
> +This change affects the location and naming of build artifacts and must
> +be taken into account by downstream users.

The chosen pattern looks reasonable to me. I'm just wondering what OE
does when generating multiple images per target, and why it did not use
something similar. Any ideas?

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/d89d4970-52ec-430e-a0e6-b28bc96f4489%40siemens.com.

Re: [PATCH v7 3/3] meta: Deploy image build artifacts into distro- and image-specific subdirs

[Zhihang Wei]

On 3/2/26 10:45, Jan Kiszka wrote:
> On 26.02.26 17:26, Zhihang Wei wrote:
>> From: Ilia Skochilov <iskochilov@ilbers.de>
>>
>> When building different distros or images for the same machine (e.g.
>> phyboard-mira-bullseye and phyboard-mira-bookworm, or isar-image-ci
>> and isar-image-base), artifacts with identical names (e.g. DTB files)
>> may be deployed to the same location, resulting file conflicts and
>> build failures.
>>
>> Deploy image build artifacts into a subdirectory named
>> ${DISTRO}-${IMAGE_PN} to avoid such collisions.
>>
>> Since the directory now encodes the distro and image name, remove the
>> ${DISTRO} and ${IMAGE_PN} prefixes from artifact filenames to
>> keep the names short and clear.
>>
>> Fixes test_dtb_deploy_distros testcase:
>>
>> ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
>> NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
>>    build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>>      (not matched to any task)
>> Please verify which recipe should provide the above files.
>>
>> Signed-off-by: Ilia Skochilov <iskochilov@ilbers.de>
>> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
>> Signed-off-by: Badrikesh Prusty <badrikesh.prusty@siemens.com>
>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> Were really all those people involved in developing and signing off this
> change?

I'll update the list for this specific version of the patch.

>> Signed-off-by: Zhihang Wei <wzh@ilbers.de>
>> ---
>>   RECIPE-API-CHANGELOG.md                       | 63 +++++++++++++++++++
>>   .../installer-add-rootfs.bbclass              |  9 +--
>>   meta/classes-recipe/image.bbclass             | 11 ++--
>>   .../imagetypes_container.bbclass              |  2 +-
>>   meta/conf/bitbake.conf                        |  3 +-
>>   .../plugins/source/isoimage-isohybrid-isar.py |  2 +-
>>   testsuite/cibase.py                           |  2 +-
>>   testsuite/citest.py                           |  7 ++-
>>   testsuite/start_vm.py                         |  2 +-
>>   9 files changed, 84 insertions(+), 17 deletions(-)
>>
>> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
>> index 0bad8a44..bc7abb37 100644
>> --- a/RECIPE-API-CHANGELOG.md
>> +++ b/RECIPE-API-CHANGELOG.md
>> @@ -962,3 +962,66 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
>>   # Optional: set countdown timeout in seconds (default 5)
>>   INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
>>   ```
>> +
>> +### Change DEPLOY_DIR_IMAGE path and artifacts naming
>> +
>> +Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
>> +${DEPLOY_DIR}/images/${MACHINE}/${DISTRO}-${IMAGE_PN}.
>> +
>> +When building different distros with the same machine the following
>> +error occurs:
>> +
>> +do_copy_boot_files: The recipe isar-image-base is trying to install
>> +files into a shared area when those files already exists. It happens
>> +when some files have the same names (e.g., dtb files) for different
>> +distros.
>> +
>> +To prevent such collisions, image artifacts are now deployed into a
>> +distro- and image-specific subdirectory.
>> +
>> +Additionally, artifact filenames have been shortened by removing the
>> +${DISTRO} and ${IMAGE_PN} prefix, since this information is now
>> +encoded in the directory path.
>> +
>> +Example 1: Build isar-image-base (phyboard-mira, debian-bookworm)
>> +Under "build/tmp/deploy/images/":
>> +Before:
>> +phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz
>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.dpkg_status
>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.ubi
>> +
>> +After:
>> +phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb
>> +phyboard-mira/debian-bookworm-isar-image-base/initrd.img
>> +phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
>> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.dpkg_status
>> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.ubi
>> +
>> +Example 2: Build isar-image-ci (qemuamd64, debian-bookworm)
>> +Under "build/tmp/deploy/images/":
>> +Before:
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-initrd.img
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-vmlinuz
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.dpkg_status
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.manifest
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.bmap
>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.manifest
>> +
>> +After:
>> +qemuamd64/debian-bookworm-isar-image-ci/initrd.img
>> +qemuamd64/debian-bookworm-isar-image-ci/vmlinuz
>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.dpkg_status
>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.manifest
>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic
>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.bmap
>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.manifest
>> +
>> +Artifacts that do not belong to a full image (e.g. isar-image-base,
>> +isar-image-ci) remain unchanged. For example, a customized initramfs
>> +built independently is not affected.
>> +
>> +This change affects the location and naming of build artifacts and must
>> +be taken into account by downstream users.
> The chosen pattern looks reasonable to me. I'm just wondering what OE
> does when generating multiple images per target, and why it did not use
> something similar. Any ideas?
>
> Jan
I built mc:arm:core-image-minimal and mc:arm:core-image-base in a
single command and enforced DTB deployment. These are the artifacts
generated by OE under "build/tmp/deploy/images/qemuarm":

core-image-base-qemuarm-20260302155417.qemuboot.conf
core-image-base-qemuarm-20260302155417.rootfs.ext4
core-image-base-qemuarm-20260302155417.rootfs.manifest
core-image-base-qemuarm-20260302155417.rootfs.tar.bz2
core-image-base-qemuarm-20260302155417.testdata.json
core-image-base-qemuarm.ext4 -> 
core-image-base-qemuarm-20260302155417.rootfs.ext4
core-image-base-qemuarm.manifest -> 
core-image-base-qemuarm-20260302155417.rootfs.manifest
core-image-base-qemuarm.qemuboot.conf -> 
core-image-base-qemuarm-20260302155417.qemuboot.conf
core-image-base-qemuarm.tar.bz2 -> 
core-image-base-qemuarm-20260302155417.rootfs.tar.bz2
core-image-base-qemuarm.testdata.json -> 
core-image-base-qemuarm-20260302155417.testdata.json
core-image-minimal-qemuarm-20260302155417.qemuboot.conf
core-image-minimal-qemuarm-20260302155417.rootfs.ext4
core-image-minimal-qemuarm-20260302155417.rootfs.manifest
core-image-minimal-qemuarm-20260302155417.rootfs.tar.bz2
core-image-minimal-qemuarm-20260302155417.testdata.json
core-image-minimal-qemuarm.ext4 -> 
core-image-minimal-qemuarm-20260302155417.rootfs.ext4
core-image-minimal-qemuarm.manifest -> 
core-image-minimal-qemuarm-20260302155417.rootfs.manifest
core-image-minimal-qemuarm.qemuboot.conf -> 
core-image-minimal-qemuarm-20260302155417.qemuboot.conf
core-image-minimal-qemuarm.tar.bz2 -> 
core-image-minimal-qemuarm-20260302155417.rootfs.tar.bz2
core-image-minimal-qemuarm.testdata.json -> 
core-image-minimal-qemuarm-20260302155417.testdata.json
modules--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.tgz
modules-qemuarm.tgz -> 
modules--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.tgz
versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.dtb
versatile-pb.dtb -> 
versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.dtb
versatile-pb-qemuarm.dtb -> 
versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.dtb
zImage -> 
zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.bin
zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.bin
zImage-qemuarm.bin -> 
zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.bin

For the same distro but different images, the kernel and DTB are
identical, and they were deployed only once. OE names the artifacts
with a timestamp and then creates symlinks pointing to them using
simpler names.

Zhihang

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/51f19e31-9642-401a-b464-824274221cd8%40ilbers.de.

Re: [PATCH v7 3/3] meta: Deploy image build artifacts into distro- and image-specific subdirs

['Jan Kiszka' via isar-users]

On 03.03.26 11:22, Zhihang Wei wrote:
> 
> 
> On 3/2/26 10:45, Jan Kiszka wrote:
>> On 26.02.26 17:26, Zhihang Wei wrote:
>>> From: Ilia Skochilov <iskochilov@ilbers.de>
>>>
>>> When building different distros or images for the same machine (e.g.
>>> phyboard-mira-bullseye and phyboard-mira-bookworm, or isar-image-ci
>>> and isar-image-base), artifacts with identical names (e.g. DTB files)
>>> may be deployed to the same location, resulting file conflicts and
>>> build failures.
>>>
>>> Deploy image build artifacts into a subdirectory named
>>> ${DISTRO}-${IMAGE_PN} to avoid such collisions.
>>>
>>> Since the directory now encodes the distro and image name, remove the
>>> ${DISTRO} and ${IMAGE_PN} prefixes from artifact filenames to
>>> keep the names short and clear.
>>>
>>> Fixes test_dtb_deploy_distros testcase:
>>>
>>> ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0
>>> do_copy_boot_files: The recipe isar-image-base is trying to install
>>> files into a shared area when those files already exist. Those files
>>> and their manifest location are:
>>> NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
>>>    build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>>>      (not matched to any task)
>>> Please verify which recipe should provide the above files.
>>>
>>> Signed-off-by: Ilia Skochilov <iskochilov@ilbers.de>
>>> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
>>> Signed-off-by: Badrikesh Prusty <badrikesh.prusty@siemens.com>
>>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>> Were really all those people involved in developing and signing off this
>> change?
> 
> I'll update the list for this specific version of the patch.
> 
>>> Signed-off-by: Zhihang Wei <wzh@ilbers.de>
>>> ---
>>>   RECIPE-API-CHANGELOG.md                       | 63 +++++++++++++++++++
>>>   .../installer-add-rootfs.bbclass              |  9 +--
>>>   meta/classes-recipe/image.bbclass             | 11 ++--
>>>   .../imagetypes_container.bbclass              |  2 +-
>>>   meta/conf/bitbake.conf                        |  3 +-
>>>   .../plugins/source/isoimage-isohybrid-isar.py |  2 +-
>>>   testsuite/cibase.py                           |  2 +-
>>>   testsuite/citest.py                           |  7 ++-
>>>   testsuite/start_vm.py                         |  2 +-
>>>   9 files changed, 84 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
>>> index 0bad8a44..bc7abb37 100644
>>> --- a/RECIPE-API-CHANGELOG.md
>>> +++ b/RECIPE-API-CHANGELOG.md
>>> @@ -962,3 +962,66 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
>>>   # Optional: set countdown timeout in seconds (default 5)
>>>   INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
>>>   ```
>>> +
>>> +### Change DEPLOY_DIR_IMAGE path and artifacts naming
>>> +
>>> +Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
>>> +${DEPLOY_DIR}/images/${MACHINE}/${DISTRO}-${IMAGE_PN}.
>>> +
>>> +When building different distros with the same machine the following
>>> +error occurs:
>>> +
>>> +do_copy_boot_files: The recipe isar-image-base is trying to install
>>> +files into a shared area when those files already exists. It happens
>>> +when some files have the same names (e.g., dtb files) for different
>>> +distros.
>>> +
>>> +To prevent such collisions, image artifacts are now deployed into a
>>> +distro- and image-specific subdirectory.
>>> +
>>> +Additionally, artifact filenames have been shortened by removing the
>>> +${DISTRO} and ${IMAGE_PN} prefix, since this information is now
>>> +encoded in the directory path.
>>> +
>>> +Example 1: Build isar-image-base (phyboard-mira, debian-bookworm)
>>> +Under "build/tmp/deploy/images/":
>>> +Before:
>>> +phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz
>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.dpkg_status
>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.ubi
>>> +
>>> +After:
>>> +phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-
>>> nand.dtb
>>> +phyboard-mira/debian-bookworm-isar-image-base/initrd.img
>>> +phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
>>> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.dpkg_status
>>> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.ubi
>>> +
>>> +Example 2: Build isar-image-ci (qemuamd64, debian-bookworm)
>>> +Under "build/tmp/deploy/images/":
>>> +Before:
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-initrd.img
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-vmlinuz
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.dpkg_status
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.manifest
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.bmap
>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.manifest
>>> +
>>> +After:
>>> +qemuamd64/debian-bookworm-isar-image-ci/initrd.img
>>> +qemuamd64/debian-bookworm-isar-image-ci/vmlinuz
>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.dpkg_status
>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.manifest
>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic
>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.bmap
>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.manifest
>>> +
>>> +Artifacts that do not belong to a full image (e.g. isar-image-base,
>>> +isar-image-ci) remain unchanged. For example, a customized initramfs
>>> +built independently is not affected.
>>> +
>>> +This change affects the location and naming of build artifacts and must
>>> +be taken into account by downstream users.
>> The chosen pattern looks reasonable to me. I'm just wondering what OE
>> does when generating multiple images per target, and why it did not use
>> something similar. Any ideas?
>>
>> Jan
> I built mc:arm:core-image-minimal and mc:arm:core-image-base in a
> single command and enforced DTB deployment. These are the artifacts
> generated by OE under "build/tmp/deploy/images/qemuarm":
> 
> core-image-base-qemuarm-20260302155417.qemuboot.conf
> core-image-base-qemuarm-20260302155417.rootfs.ext4
> core-image-base-qemuarm-20260302155417.rootfs.manifest
> core-image-base-qemuarm-20260302155417.rootfs.tar.bz2
> core-image-base-qemuarm-20260302155417.testdata.json
> core-image-base-qemuarm.ext4 -> core-image-base-
> qemuarm-20260302155417.rootfs.ext4
> core-image-base-qemuarm.manifest -> core-image-base-
> qemuarm-20260302155417.rootfs.manifest
> core-image-base-qemuarm.qemuboot.conf -> core-image-base-
> qemuarm-20260302155417.qemuboot.conf
> core-image-base-qemuarm.tar.bz2 -> core-image-base-
> qemuarm-20260302155417.rootfs.tar.bz2
> core-image-base-qemuarm.testdata.json -> core-image-base-
> qemuarm-20260302155417.testdata.json
> core-image-minimal-qemuarm-20260302155417.qemuboot.conf
> core-image-minimal-qemuarm-20260302155417.rootfs.ext4
> core-image-minimal-qemuarm-20260302155417.rootfs.manifest
> core-image-minimal-qemuarm-20260302155417.rootfs.tar.bz2
> core-image-minimal-qemuarm-20260302155417.testdata.json
> core-image-minimal-qemuarm.ext4 -> core-image-minimal-
> qemuarm-20260302155417.rootfs.ext4
> core-image-minimal-qemuarm.manifest -> core-image-minimal-
> qemuarm-20260302155417.rootfs.manifest
> core-image-minimal-qemuarm.qemuboot.conf -> core-image-minimal-
> qemuarm-20260302155417.qemuboot.conf
> core-image-minimal-qemuarm.tar.bz2 -> core-image-minimal-
> qemuarm-20260302155417.rootfs.tar.bz2
> core-image-minimal-qemuarm.testdata.json -> core-image-minimal-
> qemuarm-20260302155417.testdata.json
> modules--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.tgz
> modules-qemuarm.tgz -> modules--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
> qemuarm-20260302155417.tgz
> versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
> qemuarm-20260302155417.dtb
> versatile-pb.dtb -> versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-
> r0-qemuarm-20260302155417.dtb
> versatile-pb-qemuarm.dtb -> versatile-
> pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.dtb
> zImage -> zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
> qemuarm-20260302155417.bin
> zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.bin
> zImage-qemuarm.bin -> zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
> qemuarm-20260302155417.bin
> 
> For the same distro but different images, the kernel and DTB are
> identical, and they were deployed only once. OE names the artifacts
> with a timestamp and then creates symlinks pointing to them using
> simpler names.

So, we are going do deviate at least from the pattern
<image-name>.wic/ext4/etc. by using <image-name>/<machine>.wic/ext4.

Can you identify images that also deviate in kernel (configuration) and,
thus, deployed images? Does OE solve the duplication of non-identical
images then automatically?

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/21f5c81e-c217-4ff7-a3b6-a14adf473f4f%40siemens.com.

Re: [PATCH v7 3/3] meta: Deploy image build artifacts into distro- and image-specific subdirs

[Zhihang Wei]

On 3/3/26 13:50, Jan Kiszka wrote:
> On 03.03.26 11:22, Zhihang Wei wrote:
>>
>> On 3/2/26 10:45, Jan Kiszka wrote:
>>> On 26.02.26 17:26, Zhihang Wei wrote:
>>>> From: Ilia Skochilov <iskochilov@ilbers.de>
>>>>
>>>> When building different distros or images for the same machine (e.g.
>>>> phyboard-mira-bullseye and phyboard-mira-bookworm, or isar-image-ci
>>>> and isar-image-base), artifacts with identical names (e.g. DTB files)
>>>> may be deployed to the same location, resulting file conflicts and
>>>> build failures.
>>>>
>>>> Deploy image build artifacts into a subdirectory named
>>>> ${DISTRO}-${IMAGE_PN} to avoid such collisions.
>>>>
>>>> Since the directory now encodes the distro and image name, remove the
>>>> ${DISTRO} and ${IMAGE_PN} prefixes from artifact filenames to
>>>> keep the names short and clear.
>>>>
>>>> Fixes test_dtb_deploy_distros testcase:
>>>>
>>>> ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0
>>>> do_copy_boot_files: The recipe isar-image-base is trying to install
>>>> files into a shared area when those files already exist. Those files
>>>> and their manifest location are:
>>>> NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
>>>>     build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>>>>       (not matched to any task)
>>>> Please verify which recipe should provide the above files.
>>>>
>>>> Signed-off-by: Ilia Skochilov <iskochilov@ilbers.de>
>>>> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
>>>> Signed-off-by: Badrikesh Prusty <badrikesh.prusty@siemens.com>
>>>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>>> Were really all those people involved in developing and signing off this
>>> change?
>> I'll update the list for this specific version of the patch.
>>
>>>> Signed-off-by: Zhihang Wei <wzh@ilbers.de>
>>>> ---
>>>>    RECIPE-API-CHANGELOG.md                       | 63 +++++++++++++++++++
>>>>    .../installer-add-rootfs.bbclass              |  9 +--
>>>>    meta/classes-recipe/image.bbclass             | 11 ++--
>>>>    .../imagetypes_container.bbclass              |  2 +-
>>>>    meta/conf/bitbake.conf                        |  3 +-
>>>>    .../plugins/source/isoimage-isohybrid-isar.py |  2 +-
>>>>    testsuite/cibase.py                           |  2 +-
>>>>    testsuite/citest.py                           |  7 ++-
>>>>    testsuite/start_vm.py                         |  2 +-
>>>>    9 files changed, 84 insertions(+), 17 deletions(-)
>>>>
>>>> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
>>>> index 0bad8a44..bc7abb37 100644
>>>> --- a/RECIPE-API-CHANGELOG.md
>>>> +++ b/RECIPE-API-CHANGELOG.md
>>>> @@ -962,3 +962,66 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
>>>>    # Optional: set countdown timeout in seconds (default 5)
>>>>    INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
>>>>    ```
>>>> +
>>>> +### Change DEPLOY_DIR_IMAGE path and artifacts naming
>>>> +
>>>> +Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
>>>> +${DEPLOY_DIR}/images/${MACHINE}/${DISTRO}-${IMAGE_PN}.
>>>> +
>>>> +When building different distros with the same machine the following
>>>> +error occurs:
>>>> +
>>>> +do_copy_boot_files: The recipe isar-image-base is trying to install
>>>> +files into a shared area when those files already exists. It happens
>>>> +when some files have the same names (e.g., dtb files) for different
>>>> +distros.
>>>> +
>>>> +To prevent such collisions, image artifacts are now deployed into a
>>>> +distro- and image-specific subdirectory.
>>>> +
>>>> +Additionally, artifact filenames have been shortened by removing the
>>>> +${DISTRO} and ${IMAGE_PN} prefix, since this information is now
>>>> +encoded in the directory path.
>>>> +
>>>> +Example 1: Build isar-image-base (phyboard-mira, debian-bookworm)
>>>> +Under "build/tmp/deploy/images/":
>>>> +Before:
>>>> +phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
>>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz
>>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.dpkg_status
>>>> +phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira.ubi
>>>> +
>>>> +After:
>>>> +phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-
>>>> nand.dtb
>>>> +phyboard-mira/debian-bookworm-isar-image-base/initrd.img
>>>> +phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
>>>> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.dpkg_status
>>>> +phyboard-mira/debian-bookworm-isar-image-base/phyboard-mira.ubi
>>>> +
>>>> +Example 2: Build isar-image-ci (qemuamd64, debian-bookworm)
>>>> +Under "build/tmp/deploy/images/":
>>>> +Before:
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-initrd.img
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64-vmlinuz
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.dpkg_status
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.manifest
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.bmap
>>>> +qemuamd64/isar-image-ci-debian-bookworm-qemuamd64.wic.manifest
>>>> +
>>>> +After:
>>>> +qemuamd64/debian-bookworm-isar-image-ci/initrd.img
>>>> +qemuamd64/debian-bookworm-isar-image-ci/vmlinuz
>>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.dpkg_status
>>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.manifest
>>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic
>>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.bmap
>>>> +qemuamd64/debian-bookworm-isar-image-ci/qemuamd64.wic.manifest
>>>> +
>>>> +Artifacts that do not belong to a full image (e.g. isar-image-base,
>>>> +isar-image-ci) remain unchanged. For example, a customized initramfs
>>>> +built independently is not affected.
>>>> +
>>>> +This change affects the location and naming of build artifacts and must
>>>> +be taken into account by downstream users.
>>> The chosen pattern looks reasonable to me. I'm just wondering what OE
>>> does when generating multiple images per target, and why it did not use
>>> something similar. Any ideas?
>>>
>>> Jan
>> I built mc:arm:core-image-minimal and mc:arm:core-image-base in a
>> single command and enforced DTB deployment. These are the artifacts
>> generated by OE under "build/tmp/deploy/images/qemuarm":
>>
>> core-image-base-qemuarm-20260302155417.qemuboot.conf
>> core-image-base-qemuarm-20260302155417.rootfs.ext4
>> core-image-base-qemuarm-20260302155417.rootfs.manifest
>> core-image-base-qemuarm-20260302155417.rootfs.tar.bz2
>> core-image-base-qemuarm-20260302155417.testdata.json
>> core-image-base-qemuarm.ext4 -> core-image-base-
>> qemuarm-20260302155417.rootfs.ext4
>> core-image-base-qemuarm.manifest -> core-image-base-
>> qemuarm-20260302155417.rootfs.manifest
>> core-image-base-qemuarm.qemuboot.conf -> core-image-base-
>> qemuarm-20260302155417.qemuboot.conf
>> core-image-base-qemuarm.tar.bz2 -> core-image-base-
>> qemuarm-20260302155417.rootfs.tar.bz2
>> core-image-base-qemuarm.testdata.json -> core-image-base-
>> qemuarm-20260302155417.testdata.json
>> core-image-minimal-qemuarm-20260302155417.qemuboot.conf
>> core-image-minimal-qemuarm-20260302155417.rootfs.ext4
>> core-image-minimal-qemuarm-20260302155417.rootfs.manifest
>> core-image-minimal-qemuarm-20260302155417.rootfs.tar.bz2
>> core-image-minimal-qemuarm-20260302155417.testdata.json
>> core-image-minimal-qemuarm.ext4 -> core-image-minimal-
>> qemuarm-20260302155417.rootfs.ext4
>> core-image-minimal-qemuarm.manifest -> core-image-minimal-
>> qemuarm-20260302155417.rootfs.manifest
>> core-image-minimal-qemuarm.qemuboot.conf -> core-image-minimal-
>> qemuarm-20260302155417.qemuboot.conf
>> core-image-minimal-qemuarm.tar.bz2 -> core-image-minimal-
>> qemuarm-20260302155417.rootfs.tar.bz2
>> core-image-minimal-qemuarm.testdata.json -> core-image-minimal-
>> qemuarm-20260302155417.testdata.json
>> modules--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.tgz
>> modules-qemuarm.tgz -> modules--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
>> qemuarm-20260302155417.tgz
>> versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
>> qemuarm-20260302155417.dtb
>> versatile-pb.dtb -> versatile-pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-
>> r0-qemuarm-20260302155417.dtb
>> versatile-pb-qemuarm.dtb -> versatile-
>> pb--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.dtb
>> zImage -> zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
>> qemuarm-20260302155417.bin
>> zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-qemuarm-20260302155417.bin
>> zImage-qemuarm.bin -> zImage--5.15.199+git0+78eca082b6_0ea8d4a7d2-r0-
>> qemuarm-20260302155417.bin
>>
>> For the same distro but different images, the kernel and DTB are
>> identical, and they were deployed only once. OE names the artifacts
>> with a timestamp and then creates symlinks pointing to them using
>> simpler names.
> So, we are going do deviate at least from the pattern
> <image-name>.wic/ext4/etc. by using <image-name>/<machine>.wic/ext4.
>
> Can you identify images that also deviate in kernel (configuration) and,
> thus, deployed images? Does OE solve the duplication of non-identical
> images then automatically?
According to [1], OE can deploy multiple non-identical images or
kernels in the same dir, as each artifact includes a timestamp in its
filename. The symlinks point to the most recent built.

[1] https://docs.yoctoproject.org/overview-manual/concepts.html#images

Zhihang

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/25470ac5-9790-48b9-bdc4-3de7050d31f0%40ilbers.de.