* [PATCH v12 1/3] meta: Fix do_copy_boot_files error for different distros of same machine
2026-07-03 15:08 [PATCH v12 0/3] Move kernel artifacts deployment from image recipe Anton Mikanovich
@ 2026-07-03 15:08 ` Anton Mikanovich
2026-07-03 15:08 ` [PATCH v12 2/3] meta: Move kernel artifacts deployment from image recipe Anton Mikanovich
` (2 subsequent siblings)
3 siblings, 0 replies; 8+ messages in thread
From: Anton Mikanovich @ 2026-07-03 15:08 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
When building different distros for the same machine (e.g.,
phyboard-mira-bullseye and phyboard-mira-bookworm), files with the same
name (e.g, DTB files) might be deployed to the same location and this
causes a build error.
Use DISTRO-dependent deploy directory as a fix.
Also fix paths in installer and testsuite.
Fixes test_dtb_deploy_distros testcase:
ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
(not matched to any task)
Please verify which recipe should provide the above files.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
RECIPE-API-CHANGELOG.md | 15 +++++++++++++++
.../classes-recipe/installer-add-rootfs.bbclass | 2 +-
meta/conf/bitbake.conf | 2 +-
testsuite/citest.py | 12 +++++-------
4 files changed, 22 insertions(+), 9 deletions(-)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 5d2514c0..a6ded089 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1087,3 +1087,18 @@ and `arch=all` binary packages). Recipes for such mixed packages should append
`PROVIDES="foo-doc-archall"` for an `arch=all` binary package `foo-doc`).
Consumers can then simply reference the package by its original name (e.g.,
`foo-doc`).
+
+### Change DEPLOY_DIR_IMAGE path
+
+Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
+${DEPLOY_DIR}/images/${MACHINE}-${DISTRO}. When building different distros
+with the same machine the following error occurs:
+
+do_copy_boot_files: The recipe isar-image-base is trying to install files
+into a shared area when those files already exists. It happens when some
+files have the same names (e.g., dtb files) for different distros.
+
+To prevent this new path is separated also by distro and kernel values.
+
+This change will influence on build artifacts location and should be taken
+into account by downstreams.
diff --git a/meta-isar/classes-recipe/installer-add-rootfs.bbclass b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
index 111506df..22c23879 100644
--- a/meta-isar/classes-recipe/installer-add-rootfs.bbclass
+++ b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
@@ -13,7 +13,7 @@ INSTALLER_TARGET_IMAGES ??= "${INSTALLER_TARGET_IMAGE}"
INSTALLER_TARGET_MC ??= "installer-target"
INSTALLER_TARGET_DISTRO ??= "${DISTRO}"
INSTALLER_TARGET_MACHINE ??= "${MACHINE}"
-INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}"
+INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}-${INSTALLER_TARGET_DISTRO}"
IMAGE_DATA_FILE ??= "${INSTALLER_TARGET_IMAGE}-${INSTALLER_TARGET_DISTRO}-${INSTALLER_TARGET_MACHINE}"
IMAGE_DATA_POSTFIX ??= "wic.zst"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6db10eb3..8134c10c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -57,7 +57,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
-DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
+DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}-${DISTRO}"
DL_DIR ?= "${TOPDIR}/downloads"
SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${DISTRO}-${DISTRO_ARCH}"
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 250c01c6..7d666880 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -522,8 +522,9 @@ class InitRdBaseTest(CIBaseTest):
super().init()
self.deploy_dir = os.path.join(self.build_dir, 'tmp', 'deploy')
- def deploy_dir_image(self, machine):
- return os.path.join(self.deploy_dir, 'images', machine)
+ def deploy_dir_image(self, mc, image):
+ multiconfig = f"{mc}:{image}"
+ return CIUtils.getVars('DEPLOY_DIR_IMAGE', target=multiconfig)
def dracut_in_image(self, targets):
machine = 'qemuamd64'
@@ -555,7 +556,7 @@ class InitRdBaseTest(CIBaseTest):
bb_should_fail=False):
mc = f'mc:{machine}-{distro.removeprefix("debian-")}'
initrd_image = f'{initrd}-{distro}-{machine}-initrd.img'
- initrd_path = os.path.join(self.deploy_dir_image(machine), initrd_image)
+ initrd_path = os.path.join(self.deploy_dir_image(mc, initrd), initrd_image)
# cleansstate if the initrd image was already built/deployed to verify
# that a new build does result in the image being deployed
@@ -725,10 +726,7 @@ class DtbDeployTest(CIBaseTest):
self.init()
self.move_in_build_dir('tmp', 'tmp_before_dtbdeploy')
- try:
- self.perform_build_test(targets, image_install='')
- except exceptions.TestFail:
- self.cancel('KFAIL')
+ self.perform_build_test(targets, image_install='')
def test_dtb_deploy_images(self):
"""
--
2.34.1
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260703150835.2490688-2-amikan%40ilbers.de.
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v12 2/3] meta: Move kernel artifacts deployment from image recipe
2026-07-03 15:08 [PATCH v12 0/3] Move kernel artifacts deployment from image recipe Anton Mikanovich
2026-07-03 15:08 ` [PATCH v12 1/3] meta: Fix do_copy_boot_files error for different distros of same machine Anton Mikanovich
@ 2026-07-03 15:08 ` Anton Mikanovich
2026-07-07 9:06 ` 'MOESSBAUER, Felix' via isar-users
2026-07-03 15:08 ` [PATCH v12 3/3] CI: Check kernel artifacts deployment Anton Mikanovich
2026-07-07 9:52 ` [PATCH v12 0/3] Move kernel artifacts deployment from image recipe 'Quirin Gylstorff' via isar-users
3 siblings, 1 reply; 8+ messages in thread
From: Anton Mikanovich @ 2026-07-03 15:08 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Task do_copy_boot_files deploys DTB files into the same location for
different images (e.g., isar-image-base and isar-image-ci). This causes
a build error.
do_copy_boot_files is called once for every image recipe while dtb
files belong to the kernel which is the same for both images. Performing
dtb deployment once for the same kernel solves the issue.
Introduce universal `kernel-deploy` bbclass responsible for extracting
the DTBs and kernel from the linux-image package and deploying them.
For the distro kernel type its package is now installed into sbuild
chroot. After that, deb which contains vmlinu[xz] file is extracted
from the apt cache.
Fixes test_dtb_deploy_images testcase:
ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
(not matched to any task)
Please verify which recipe should provide the above files.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
RECIPE-API-CHANGELOG.md | 13 ++++++
meta/classes-recipe/image.bbclass | 39 +++++++++--------
meta/classes-recipe/linux-deploy.bbclass | 46 ++++++++++++++++++++
meta/classes-recipe/linux-kernel.bbclass | 3 ++
meta/recipes-kernel/linux/files/getkernel.sh | 40 +++++++++++++++++
meta/recipes-kernel/linux/files/rules.tmpl | 12 +++++
meta/recipes-kernel/linux/linux-distro.bb | 27 ++++++++++++
testsuite/citest.py | 2 -
8 files changed, 161 insertions(+), 21 deletions(-)
create mode 100644 meta/classes-recipe/linux-deploy.bbclass
create mode 100755 meta/recipes-kernel/linux/files/getkernel.sh
create mode 100644 meta/recipes-kernel/linux/files/rules.tmpl
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index a6ded089..ef9dce00 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1102,3 +1102,16 @@ To prevent this new path is separated also by distro and kernel values.
This change will influence on build artifacts location and should be taken
into account by downstreams.
+
+### Move kernel and DTBs deployment from image recipe to kernel one
+
+To prevent parallel conflicting artifacts deployment from the same kernel
+used by different images, move original deployment logic from
+do_copy_boot_files() task of the image recipe to the kernel recipe. As we have
+two types of kernel (distro and self build), both are affected. With this
+aproach the deployment of any kernel artifacts will be performed only once per
+kernel.
+
+Also allow using different kernels for one machine and distro without
+overwritting by storing kernel artifacts in kernel-{KERNEL_NAME} subdirectory
+of DEPLOY_DIR_IMAGE. Previous image linking names are kept by symlinks.
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 26a4ec06..5f34cd4f 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -1,5 +1,7 @@
# This software is a part of Isar.
-# Copyright (C) 2015-2017 ilbers GmbH
+# Copyright (C) 2015-2026 ilbers GmbH
+#
+# SPDX-License-Identifier: MIT
# Make workdir and stamps machine-specific without changing common PN target
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
@@ -9,7 +11,6 @@ STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
# Sstate also needs to be machine-specific
SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
-SSTATETASKS += "do_copy_boot_files"
IMAGE_INSTALL ?= ""
IMAGE_FSTYPES ?= "ext4"
@@ -381,37 +382,37 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_DEPLOY_FILE}"
# only one dtb file supported, pick the first
DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"
-do_copy_boot_files[cleandirs] += "${DEPLOYDIR}"
-do_copy_boot_files[sstate-inputdirs] = "${DEPLOYDIR}"
-do_copy_boot_files[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
-do_copy_boot_files[network] = "${TASK_USE_SUDO}"
+KERNEL_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}/kernel-${KERNEL_NAME}"
+KERNEL_WILDCARD = "${@ 'vmlinu[xz]*' if (p := d.getVar('KERNEL_FILE')) == 'vmlinux' else p+'*'}"
+
+python() {
+ if d.getVar('KERNEL_NAME'):
+ pn = d.getVar('KERNEL_IMAGE_PKG') or ''
+ task = 'do_deploy_kernel_%s' % (d.getVar('MACHINE').replace('-','_') or '')
+ d.appendVarFlag("do_copy_boot_files", "depends", f"{pn}:{task}")
+}
+
+# Associate kernel with image by symlinks
+do_copy_boot_files[dirs] += "${DEPLOY_DIR_IMAGE}"
+do_copy_boot_files[file-checksums] += "${KERNEL_DEPLOY_DIR}/${KERNEL_WILDCARD}:True"
do_copy_boot_files() {
- kernel="$(realpath -q '${IMAGE_ROOTFS}'/vmlinu[xz])"
- if [ ! -f "$kernel" ]; then
- kernel="$(realpath -q '${IMAGE_ROOTFS}'/boot/vmlinu[xz])"
- fi
+ kernel="$(realpath -mq '${KERNEL_DEPLOY_DIR}/'${KERNEL_WILDCARD} | head -n1)"
if [ -f "$kernel" ]; then
- sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
+ ln -sfr "$kernel" "${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE}"
fi
for file in ${DTB_FILES}; do
- dtb="$(find '${IMAGE_ROOTFS}/usr/lib' -type f \
- -iwholename '*linux-image-*/'${file} | head -1)"
+ dtb="${KERNEL_DEPLOY_DIR}/$(basename ${file})"
if [ -z "$dtb" -o ! -e "$dtb" ]; then
die "${file} not found"
fi
- cp -f "$dtb" "${DEPLOYDIR}/"
+ ln -sfr "$dtb" "${DEPLOY_DIR_IMAGE}/$(basename $dtb)"
done
}
addtask copy_boot_files before do_rootfs_postprocess after do_rootfs_install
-python do_copy_boot_files_setscene () {
- sstate_setscene(d)
-}
-addtask do_copy_boot_files_setscene
-
python do_image_tools() {
"""Virtual task"""
pass
diff --git a/meta/classes-recipe/linux-deploy.bbclass b/meta/classes-recipe/linux-deploy.bbclass
new file mode 100644
index 00000000..4b33e234
--- /dev/null
+++ b/meta/classes-recipe/linux-deploy.bbclass
@@ -0,0 +1,46 @@
+# This software is a part of Isar.
+# Copyright (C) 2026 ilbers GmbH
+#
+# SPDX-License-Identifier: MIT
+
+DEPLOYDIR = "${WORKDIR}/deploy_${@ d.getVar('MACHINE').replace('-','_') or ''}"
+KERNEL_DEPLOY_TASKNAME ?= "do_deploy_kernel_${@ d.getVar('MACHINE').replace('-','_') or ''}"
+SSTATETASKS += "${KERNEL_DEPLOY_TASKNAME}"
+
+python () {
+ kernel_name = d.getVar('KERNEL_NAME_PROVIDED') or ''
+ if "linux-image-"+kernel_name in d.getVar('PROVIDES'):
+ task = d.getVar('KERNEL_DEPLOY_TASKNAME')
+ d.setVar(task, d.expand('kernel_deploy'))
+ d.setVarFlag(task, 'func', '1')
+ d.setVarFlag(task, 'sstate-inputdirs', d.getVar('DEPLOYDIR'))
+ d.setVarFlag(task, 'sstate-outputdirs', d.getVar('KERNEL_DEPLOY_DIR'))
+ d.appendVarFlag(task, 'cleandirs', d.getVar('DEPLOYDIR'))
+ d.appendVarFlag(task, 'stamp-extra-info', d.getVar('MACHINE'))
+ bb.build.addtask(task, 'do_build', 'do_dpkg_build', d)
+}
+
+KERNEL_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}/kernel-${KERNEL_NAME_PROVIDED}"
+
+KERNEL_LOCATION ?= "./boot"
+KERNEL_DEB ?= "linux-image-${KERNEL_NAME_PROVIDED}_${CHANGELOG_V}_${DISTRO_ARCH}.deb"
+
+# Take care the case when requested kernel format doesn't match distro one
+DEPLOY_WILDCARDS = "'${KERNEL_LOCATION}/${@ 'vmlinu[xz]*' if (p := d.getVar('KERNEL_FILE')) == 'vmlinux' else p+'*'}'"
+DEPLOY_WILDCARDS += "${@(' '.join("'*%s'" % p for p in (d.getVar('DTB_FILES') or '').split()))}"
+
+kernel_deploy() {
+ case "${PROVIDES}" in
+ *linux-image-${KERNEL_NAME_PROVIDED}*)
+ dpkg --fsys-tarfile ${WORKDIR}/${KERNEL_DEB} | \
+ tar xvf - -C "${DEPLOYDIR}" \
+ --transform='s|^.*/||' \
+ --wildcards ${DEPLOY_WILDCARDS}
+ ;;
+ esac
+}
+
+python do_copy_boot_files_setscene () {
+ sstate_setscene(d)
+}
+addtask do_copy_boot_files_setscene
diff --git a/meta/classes-recipe/linux-kernel.bbclass b/meta/classes-recipe/linux-kernel.bbclass
index fffdf12c..42e0c6b3 100644
--- a/meta/classes-recipe/linux-kernel.bbclass
+++ b/meta/classes-recipe/linux-kernel.bbclass
@@ -3,6 +3,7 @@
# This software is a part of Isar.
# Copyright (c) Siemens AG, 2022
# Copyright (c) Mentor Graphics, a Siemens business, 2022
+# Copyright (C) 2022-2026 ilbers GmbH
#
# SPDX-License-Identifier: MIT
@@ -338,3 +339,5 @@ do_dpkg_source:prepend() {
dpkg_configure_kernel
get_localversion_auto
}
+
+inherit linux-deploy
diff --git a/meta/recipes-kernel/linux/files/getkernel.sh b/meta/recipes-kernel/linux/files/getkernel.sh
new file mode 100755
index 00000000..7070dbc0
--- /dev/null
+++ b/meta/recipes-kernel/linux/files/getkernel.sh
@@ -0,0 +1,40 @@
+#!/bin/bash -e
+
+deb_cache="/var/cache/apt/archives"
+
+paths="/vmlinu[xz] /boot/vmlinu[xz]"
+if [ -n "$1" ]; then
+ paths="/$1 /boot/$1 $paths"
+fi
+
+# Lookup for the kernel file
+for path in ${paths}; do
+ kernel="$(realpath -q ${path})"
+ if [ -f "${kernel}" ]; then
+ break
+ fi
+done
+
+# Obtain package name for the kernel file
+pkg="$(dpkg -S ${kernel} | cut -d':' -f1)"
+if [ -z "${pkg}" ]; then
+ >&2 echo "No package providing ${kernel} found!"
+ exit 1
+fi
+
+# Query for deb filename
+deb_name=$(dpkg-query -W -f='${Package}_${Version}_${Architecture}.deb\n' ${pkg})
+
+# Take care about special symbols
+deb_name="${deb_name//%/%25}"
+deb_name="${deb_name//:/%3a}"
+deb_name="${deb_name//~/%7e}"
+
+# Search for deb in cache dir
+deb_path="$(find ${deb_cache} -name "${deb_name}" 2>/dev/null | head -n1)"
+if [ ! -f "${deb_path}" ]; then
+ >&2 echo "Package ${deb_name} not found in ${deb_cache}!"
+ exit 1
+fi
+
+echo "${deb_path}"
diff --git a/meta/recipes-kernel/linux/files/rules.tmpl b/meta/recipes-kernel/linux/files/rules.tmpl
new file mode 100644
index 00000000..69d79bb5
--- /dev/null
+++ b/meta/recipes-kernel/linux/files/rules.tmpl
@@ -0,0 +1,12 @@
+#!/usr/bin/make -f
+
+KERNEL_PATH := $(shell ./debian/getkernel.sh ${KERNEL_FILE})
+DEB_NAME := ${KERNEL_DEB}
+
+binary:
+ @[ -z "$(KERNEL_PATH)" ] && { echo "Kernel not found!"; exit 1; } || true
+ cp "$(KERNEL_PATH)" ../$(DEB_NAME)
+ echo "$(DEB_NAME) misc optional" > debian/files
+
+%:
+ true
diff --git a/meta/recipes-kernel/linux/linux-distro.bb b/meta/recipes-kernel/linux/linux-distro.bb
index 8fc1bcb7..47fe3fb4 100644
--- a/meta/recipes-kernel/linux/linux-distro.bb
+++ b/meta/recipes-kernel/linux/linux-distro.bb
@@ -2,6 +2,7 @@
#
# This software is a part of Isar.
# Copyright (c) Siemens AG, 2018
+# Copyright (C) 2022-2026 ilbers GmbH
#
# SPDX-License-Identifier: MIT
@@ -27,3 +28,29 @@ python() {
}
inherit multiarch
+inherit dpkg
+inherit linux-deploy
+
+# Always use target arch for kernel package lookup
+ISAR_CROSS_COMPILE = "0"
+
+MAINTAINER = "isar-users <isar-users@googlegroups.com>"
+
+PN .= "-${KERNEL_NAME}"
+KERNEL_NAME_PROVIDED ??= "${KERNEL_NAME}"
+DEBIAN_BUILD_DEPENDS ?= "${@d.getVar('KERNEL_IMAGE_PKG') or ('linux-image-' + (d.getVar('KERNEL_NAME') or ''))}"
+
+FILESPATH:prepend = "${LAYERDIR_core}/recipes-kernel/linux/files:"
+
+SRC_URI = "file://getkernel.sh \
+ file://rules.tmpl"
+
+TEMPLATE_VARS += "KERNEL_FILE KERNEL_DEB"
+TEMPLATE_FILES = "rules.tmpl"
+
+do_prepare_build[cleandirs] += "${S}/debian"
+do_prepare_build() {
+ deb_debianize
+ cp "${WORKDIR}/getkernel.sh" "${S}/debian/"
+}
+do_deploy_deb[noexec] = "1"
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 7d666880..5aa3e799 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -740,8 +740,6 @@ class DtbDeployTest(CIBaseTest):
self.init()
try:
self.perform_build_test(targets, image_install='')
- except exceptions.TestFail:
- self.cancel('KFAIL')
finally:
self.move_in_build_dir('tmp', 'tmp_dtbdeploy')
--
2.34.1
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260703150835.2490688-3-amikan%40ilbers.de.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v12 2/3] meta: Move kernel artifacts deployment from image recipe
2026-07-03 15:08 ` [PATCH v12 2/3] meta: Move kernel artifacts deployment from image recipe Anton Mikanovich
@ 2026-07-07 9:06 ` 'MOESSBAUER, Felix' via isar-users
2026-07-07 9:09 ` [PATCH 1/1] dpkg: directly deploy via sstate cache 'Felix Moessbauer' via isar-users
0 siblings, 1 reply; 8+ messages in thread
From: 'MOESSBAUER, Felix' via isar-users @ 2026-07-07 9:06 UTC (permalink / raw)
To: amikan, isar-users
On Fri, 2026-07-03 at 18:08 +0300, Anton Mikanovich wrote:
> Task do_copy_boot_files deploys DTB files into the same location for
> different images (e.g., isar-image-base and isar-image-ci). This causes
> a build error.
>
> do_copy_boot_files is called once for every image recipe while dtb
> files belong to the kernel which is the same for both images. Performing
> dtb deployment once for the same kernel solves the issue.
>
> Introduce universal `kernel-deploy` bbclass responsible for extracting
> the DTBs and kernel from the linux-image package and deploying them.
>
> For the distro kernel type its package is now installed into sbuild
> chroot. After that, deb which contains vmlinu[xz] file is extracted
> from the apt cache.
>
> Fixes test_dtb_deploy_images testcase:
>
> ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
> build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
> (not matched to any task)
> Please verify which recipe should provide the above files.
>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
> RECIPE-API-CHANGELOG.md | 13 ++++++
> meta/classes-recipe/image.bbclass | 39 +++++++++--------
> meta/classes-recipe/linux-deploy.bbclass | 46 ++++++++++++++++++++
> meta/classes-recipe/linux-kernel.bbclass | 3 ++
> meta/recipes-kernel/linux/files/getkernel.sh | 40 +++++++++++++++++
> meta/recipes-kernel/linux/files/rules.tmpl | 12 +++++
> meta/recipes-kernel/linux/linux-distro.bb | 27 ++++++++++++
> testsuite/citest.py | 2 -
> 8 files changed, 161 insertions(+), 21 deletions(-)
> create mode 100644 meta/classes-recipe/linux-deploy.bbclass
> create mode 100755 meta/recipes-kernel/linux/files/getkernel.sh
> create mode 100644 meta/recipes-kernel/linux/files/rules.tmpl
>
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index a6ded089..ef9dce00 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -1102,3 +1102,16 @@ To prevent this new path is separated also by distro and kernel values.
>
> This change will influence on build artifacts location and should be taken
> into account by downstreams.
> +
> +### Move kernel and DTBs deployment from image recipe to kernel one
> +
> +To prevent parallel conflicting artifacts deployment from the same kernel
> +used by different images, move original deployment logic from
> +do_copy_boot_files() task of the image recipe to the kernel recipe. As we have
> +two types of kernel (distro and self build), both are affected. With this
> +aproach the deployment of any kernel artifacts will be performed only once per
> +kernel.
> +
> +Also allow using different kernels for one machine and distro without
> +overwritting by storing kernel artifacts in kernel-{KERNEL_NAME} subdirectory
>
[snip]
> diff --git a/meta/classes-recipe/linux-deploy.bbclass b/meta/classes-recipe/linux-deploy.bbclass
> new file mode 100644
> index 00000000..4b33e234
> --- /dev/null
> +++ b/meta/classes-recipe/linux-deploy.bbclass
> @@ -0,0 +1,46 @@
> +# This software is a part of Isar.
> +# Copyright (C) 2026 ilbers GmbH
> +#
> +# SPDX-License-Identifier: MIT
> +
> +DEPLOYDIR = "${WORKDIR}/deploy_${@ d.getVar('MACHINE').replace('-','_') or ''}"
Changing the deploy dir is dangerous, as this also affects other tasks
of the dpkg class. If more than one task of a class performs a sstate
operation, the deploy dirs should also be split.
While this currently is not a problem, it will be once we cleanup the
deployment of the dpkg files. I have a patch for that, which is
currently queued behind the rootless series, but it will send it here
for reference as well.
Felix
> +KERNEL_DEPLOY_TASKNAME ?= "do_deploy_kernel_${@ d.getVar('MACHINE').replace('-','_') or ''}"
> +SSTATETASKS += "${KERNEL_DEPLOY_TASKNAME}"
> +
> +python () {
> + kernel_name = d.getVar('KERNEL_NAME_PROVIDED') or ''
> + if "linux-image-"+kernel_name in d.getVar('PROVIDES'):
> + task = d.getVar('KERNEL_DEPLOY_TASKNAME')
> + d.setVar(task, d.expand('kernel_deploy'))
> + d.setVarFlag(task, 'func', '1')
> + d.setVarFlag(task, 'sstate-inputdirs', d.getVar('DEPLOYDIR'))
> + d.setVarFlag(task, 'sstate-outputdirs', d.getVar('KERNEL_DEPLOY_DIR'))
> + d.appendVarFlag(task, 'cleandirs', d.getVar('DEPLOYDIR'))
> + d.appendVarFlag(task, 'stamp-extra-info', d.getVar('MACHINE'))
> + bb.build.addtask(task, 'do_build', 'do_dpkg_build', d)
> +}
> +
> +KERNEL_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}/kernel-${KERNEL_NAME_PROVIDED}"
> +
> +KERNEL_LOCATION ?= "./boot"
> +KERNEL_DEB ?= "linux-image-${KERNEL_NAME_PROVIDED}_${CHANGELOG_V}_${DISTRO_ARCH}.deb"
> +
> +# Take care the case when requested kernel format doesn't match distro one
> +DEPLOY_WILDCARDS = "'${KERNEL_LOCATION}/${@ 'vmlinu[xz]*' if (p := d.getVar('KERNEL_FILE')) == 'vmlinux' else p+'*'}'"
> +DEPLOY_WILDCARDS += "${@(' '.join("'*%s'" % p for p in (d.getVar('DTB_FILES') or '').split()))}"
> +
> +kernel_deploy() {
> + case "${PROVIDES}" in
> + *linux-image-${KERNEL_NAME_PROVIDED}*)
> + dpkg --fsys-tarfile ${WORKDIR}/${KERNEL_DEB} | \
> + tar xvf - -C "${DEPLOYDIR}" \
> + --transform='s|^.*/||' \
> + --wildcards ${DEPLOY_WILDCARDS}
> + ;;
> + esac
> +}
> +
> +python do_copy_boot_files_setscene () {
> + sstate_setscene(d)
> +}
> +addtask do_copy_boot_files_setscene
>
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/ac2f7d165688d3fc8c20cadbdfb0160e1763ed2a.camel%40siemens.com.
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 1/1] dpkg: directly deploy via sstate cache
2026-07-07 9:06 ` 'MOESSBAUER, Felix' via isar-users
@ 2026-07-07 9:09 ` 'Felix Moessbauer' via isar-users
0 siblings, 0 replies; 8+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-07-07 9:09 UTC (permalink / raw)
To: isar-users; +Cc: Felix Moessbauer
By that, we can get rid of the custom sstate logic, which should be
avoided if not needed. Further, the deploy process is now cleaner, as we
deploy from a dedicated dir instead of the WORKDIR.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
doc/technical_overview.md | 2 +-
meta/classes-recipe/dpkg-base.bbclass | 37 +++++++++------------------
2 files changed, 13 insertions(+), 26 deletions(-)
diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index f7e00f7b..fe427cae 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -217,7 +217,7 @@ Both consist of the following steps:
finally unmount again (`dpkg_undo_mounts`).
6. Task `do_deploy_deb`: add successfully built packages
- `${WORKDIR}/*.deb` to the isar-apt repository
+ `${DEPLOYDIR}/*.deb` to the isar-apt repository
`${REPO_ISAR_DIR}/${DISTRO}`
## 3.6 Populate Target Filesystem
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index 153b91c4..1a1d8da7 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -12,6 +12,8 @@ inherit repository
inherit deb-dl-dir
inherit essential
+DEPLOYDIR = "${WORKDIR}/deploy"
+
DEPENDS ?= ""
RPROVIDES ?= "${PROVIDES}"
@@ -186,6 +188,10 @@ dpkg_prepare_unshare_ccache() {
setfacl -m u:${UNSHARE_SUBUID_BASE}:rwX -m u:${@int(d.getVar('UNSHARE_SUBUID_BASE')) + 999}:rwx "${CCACHE_DIR}"
}
+dpkg_collect_debs() {
+ find ${WORKDIR} -maxdepth 1 -name "*.deb" -exec mv {} ${DEPLOYDIR} \;
+}
+
python do_dpkg_build() {
bb.build.exec_func('dpkg_chroot_prepare', d)
try:
@@ -193,45 +199,26 @@ python do_dpkg_build() {
finally:
bb.build.exec_func('dpkg_chroot_finalize', d)
}
+do_dpkg_build[cleandirs] = "${DEPLOYDIR}"
+do_dpkg_build[sstate-plaindirs] = "${DEPLOYDIR}"
do_dpkg_build[network] = "${TASK_USE_NETWORK_AND_SUDO}"
+do_dpkg_build[depends] = "${SCHROOT_DEP}"
+do_dpkg_build[postfuncs] += "dpkg_collect_debs"
addtask dpkg_build
SSTATETASKS += "do_dpkg_build"
-SSTATECREATEFUNCS += "dpkg_build_sstate_prepare"
-SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize"
-
-dpkg_build_sstate_prepare() {
- [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
-
- # this runs in SSTATE_BUILDDIR, which will be deleted automatically
- if [ -n "$(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print -quit)" ]; then
- cp -f ${WORKDIR}/*.deb -t .
- fi
-}
-
-dpkg_build_sstate_finalize() {
- [ "${SSTATE_CURRTASK}" = "dpkg_build" ] || return 0
-
- # this runs in SSTATE_INSTDIR
- if [ -n "$(find . -maxdepth 1 -name '*.deb' -print -quit)" ]; then
- mv -f ./*.deb -t ${WORKDIR}/
- fi
-}
python do_dpkg_build_setscene() {
sstate_setscene(d)
}
addtask dpkg_build_setscene
-do_dpkg_build_setscene[dirs] += "${S}/.."
-
-do_dpkg_build[depends] = "${SCHROOT_DEP}"
CLEANFUNCS += "deb_clean"
deb_clean() {
- DEBS=$( find ${WORKDIR} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
+ DEBS=$( find ${DEPLOYDIR} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
if [ -n "${DEBS}" ]; then
for d in ${DEBS}; do
repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \
@@ -246,7 +233,7 @@ do_clean[network] = "${TASK_USE_SUDO}"
do_deploy_deb() {
deb_clean
repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \
- "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${WORKDIR}/*.deb
+ "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${DEPLOYDIR}/*.deb
}
addtask deploy_deb after do_dpkg_build before do_build
--
2.53.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260707090941.2667874-1-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v12 3/3] CI: Check kernel artifacts deployment
2026-07-03 15:08 [PATCH v12 0/3] Move kernel artifacts deployment from image recipe Anton Mikanovich
2026-07-03 15:08 ` [PATCH v12 1/3] meta: Fix do_copy_boot_files error for different distros of same machine Anton Mikanovich
2026-07-03 15:08 ` [PATCH v12 2/3] meta: Move kernel artifacts deployment from image recipe Anton Mikanovich
@ 2026-07-03 15:08 ` Anton Mikanovich
2026-07-07 9:52 ` [PATCH v12 0/3] Move kernel artifacts deployment from image recipe 'Quirin Gylstorff' via isar-users
3 siblings, 0 replies; 8+ messages in thread
From: Anton Mikanovich @ 2026-07-03 15:08 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Improve deploy test cases to verify kernel and DTB files are present
in the deploy directory after the build.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 5aa3e799..662af1de 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -715,6 +715,20 @@ class DtbDeployTest(CIBaseTest):
:avocado: tags=dtbdeploy,full
"""
+ def check_kernel_deploy(self, targets):
+ for t in targets:
+ dd, kernel_name, dtbs = \
+ CIUtils.getVars('DEPLOY_DIR_IMAGE', 'KERNEL_IMAGE',
+ 'DTB_FILES', target=t)
+ self.log.info(f"dd={dd}, kernel_name={kernel_name}, dtbs={dtbs}")
+ kernel_path = f"{dd}/{kernel_name}"
+ if not os.path.exists(kernel_path):
+ self.fail(f"Target {t} failed to deploy kernel image!")
+ for dtb_name in dtbs.split():
+ dtb_path = f"{dd}/{dtb_name}"
+ if not os.path.exists(dtb_path):
+ self.fail(f"Target {t} failed to deploy {dtb_name}!")
+
def test_dtb_deploy_distros(self):
"""
Cover case: Same machine, different distros
@@ -727,6 +741,7 @@ class DtbDeployTest(CIBaseTest):
self.init()
self.move_in_build_dir('tmp', 'tmp_before_dtbdeploy')
self.perform_build_test(targets, image_install='')
+ self.check_kernel_deploy(targets)
def test_dtb_deploy_images(self):
"""
@@ -740,6 +755,7 @@ class DtbDeployTest(CIBaseTest):
self.init()
try:
self.perform_build_test(targets, image_install='')
+ self.check_kernel_deploy(targets)
finally:
self.move_in_build_dir('tmp', 'tmp_dtbdeploy')
--
2.34.1
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260703150835.2490688-4-amikan%40ilbers.de.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v12 0/3] Move kernel artifacts deployment from image recipe
2026-07-03 15:08 [PATCH v12 0/3] Move kernel artifacts deployment from image recipe Anton Mikanovich
` (2 preceding siblings ...)
2026-07-03 15:08 ` [PATCH v12 3/3] CI: Check kernel artifacts deployment Anton Mikanovich
@ 2026-07-07 9:52 ` 'Quirin Gylstorff' via isar-users
2026-07-07 11:11 ` 'Jan Kiszka' via isar-users
3 siblings, 1 reply; 8+ messages in thread
From: 'Quirin Gylstorff' via isar-users @ 2026-07-07 9:52 UTC (permalink / raw)
To: isar-users
On 7/3/26 5:08 PM, Anton Mikanovich wrote:
> Image task "do_copy_boot_files" is used to deploy kernel and
> devicetree binaries to deploy directory.
>
> When different images for the same target are built (e.g., "-base" and
> "-debug") in parallel, this causes issues with DTB deployment since
> they have no any image_name-specific stuff in the path, unlike kernel
> and initrd.
>
> Here we move DTB deployment to the kernel recipe (for both distro and
> custom kernel recipes), where DTBs and kernel image are extracted from
> the kernel package.
> For the distro kernels it means we install its deb into the schroot to
> obtain the real artifacts from the filesystem.
>
> To keep the same artifacts path as before "do_copy_boot_files" task is
> now searching for the artifacts and create per-image symlinks.
> In case any naming conflicts (like dtbs with the same name from
> different kernels) the latest built dtb will be pointing by the
> symlink, while both will be deployed to different subdirs without
> conflicting and stay untouched.
> This symlink creation is done in the same way as artifacts conficts are
> handled by Yocto.
>
> The case when different distros (e.g., bullseye/bookworm) are using the
> same name for DTB binaries is covered by the patch 2. It also covers
> the case when image uses different kernel suffix (e.g., "realtime"
> option for the kernel in some downstreams).
In my option we should go back to the design board as multiconfig and
Debian are in conflict in this case.
We should decide what we want to support with multiconfigs and what not.
The technical implementation should come afterwards.
We should also disallow some of the possible use cases of multiconfig.
Quirin
>
> Related topics on maillist:
>
> https://groups.google.com/g/isar-users/c/ht3YmKZcJlM
> https://groups.google.com/g/isar-users/c/4BRDM02xC40
> https://groups.google.com/g/isar-users/c/qAnjahjjfsw
> https://groups.google.com/g/isar-users/c/ZMD4XY4dKWQ
> https://groups.google.com/g/isar-users/c/PSGU_AcdPZ8
> https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA
> https://groups.google.com/g/isar-users/c/ixzytHYnTmI
>
> Changes since v11:
> - Fix commit messages wording.
> - Simplify variables obtain on task creation.
>
> Changes since v10:
> - Rebase on next.
> - Use the same deploy dir with per-kernel subdirs.
> - Allow using different kernels for one machine.
> - Restore paths for downstreams in Yocto way (symlinks).
>
> Changes since v9:
> - Base on v6.
> - Rebase on next.
> - Move also kernel deployment from image recipe.
>
> Changes since v8:
> - Rebase on next.
> - Document how this approach differs from OpenEmbedded.
>
> Changes since v7:
> - Fix SOB.
>
> Changes since v6:
> - Fix isoimage-isohybrid-isar WIC plugin.
> - Split testcases.
> - Update wording.
>
> Changes since v4:
> - Rebase on next.
> - Squash latest patch to first one.
> - Fix deploy paths in initrd tests.
>
> Changes since v3:
> - Rebase on next.
> - Fix target deploy dir path for installer.
>
> Changes since v2:
> - Patch 1: Consider also kernel suffix.
> - Patch 2: Use separate testcase for possible dtb clashes.
> - Minor cosmetic fixes.
>
> Changes since v1:
> - Separate `dtb-files` recipe used instead of linux one
> - Included older "[PATCH] Fix do_copy_boot_files error" into the
> series ( https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA )
>
> Anton Mikanovich (3):
> meta: Fix do_copy_boot_files error for different distros of same
> machine
> meta: Move kernel artifacts deployment from image recipe
> CI: Check kernel artifacts deployment
>
> RECIPE-API-CHANGELOG.md | 28 +++++++++++
> .../installer-add-rootfs.bbclass | 2 +-
> meta/classes-recipe/image.bbclass | 39 ++++++++--------
> meta/classes-recipe/linux-deploy.bbclass | 46 +++++++++++++++++++
> meta/classes-recipe/linux-kernel.bbclass | 3 ++
> meta/conf/bitbake.conf | 2 +-
> meta/recipes-kernel/linux/files/getkernel.sh | 40 ++++++++++++++++
> meta/recipes-kernel/linux/files/rules.tmpl | 12 +++++
> meta/recipes-kernel/linux/linux-distro.bb | 27 +++++++++++
> testsuite/citest.py | 30 ++++++++----
> 10 files changed, 199 insertions(+), 30 deletions(-)
> create mode 100644 meta/classes-recipe/linux-deploy.bbclass
> create mode 100755 meta/recipes-kernel/linux/files/getkernel.sh
> create mode 100644 meta/recipes-kernel/linux/files/rules.tmpl
>
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/cdc0b124-7de1-40f0-84b2-2cb89ee28279%40siemens.com.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v12 0/3] Move kernel artifacts deployment from image recipe
2026-07-07 9:52 ` [PATCH v12 0/3] Move kernel artifacts deployment from image recipe 'Quirin Gylstorff' via isar-users
@ 2026-07-07 11:11 ` 'Jan Kiszka' via isar-users
0 siblings, 0 replies; 8+ messages in thread
From: 'Jan Kiszka' via isar-users @ 2026-07-07 11:11 UTC (permalink / raw)
To: Quirin Gylstorff, isar-users
On 07.07.26 11:52, 'Quirin Gylstorff' via isar-users wrote:
>
>
> On 7/3/26 5:08 PM, Anton Mikanovich wrote:
>> Image task "do_copy_boot_files" is used to deploy kernel and
>> devicetree binaries to deploy directory.
>>
>> When different images for the same target are built (e.g., "-base" and
>> "-debug") in parallel, this causes issues with DTB deployment since
>> they have no any image_name-specific stuff in the path, unlike kernel
>> and initrd.
>>
>> Here we move DTB deployment to the kernel recipe (for both distro and
>> custom kernel recipes), where DTBs and kernel image are extracted from
>> the kernel package.
>> For the distro kernels it means we install its deb into the schroot to
>> obtain the real artifacts from the filesystem.
>>
>> To keep the same artifacts path as before "do_copy_boot_files" task is
>> now searching for the artifacts and create per-image symlinks.
>> In case any naming conflicts (like dtbs with the same name from
>> different kernels) the latest built dtb will be pointing by the
>> symlink, while both will be deployed to different subdirs without
>> conflicting and stay untouched.
>> This symlink creation is done in the same way as artifacts conficts are
>> handled by Yocto.
>>
>> The case when different distros (e.g., bullseye/bookworm) are using the
>> same name for DTB binaries is covered by the patch 2. It also covers
>> the case when image uses different kernel suffix (e.g., "realtime"
>> option for the kernel in some downstreams).
>
> In my option we should go back to the design board as multiconfig and
> Debian are in conflict in this case.
>
> We should decide what we want to support with multiconfigs and what not.
>
> The technical implementation should come afterwards.
>
> We should also disallow some of the possible use cases of multiconfig.
>
Ack. Given that only Isar upstream was apparently bitten by this so far,
none of our various user layers, it is really questionable how much
value we would add, compared to the complications we are causing, both
to our users as well as to Isar itself.
Jan
--
Siemens AG, Foundational Technologies
Linux Expert Center
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/1c64b051-9e6c-4891-904b-8efc578dd904%40siemens.com.
^ permalink raw reply [flat|nested] 8+ messages in thread