Re: [PATCH v11 2/3] meta: Move kernel artifacts deployment from image recipe

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Tue, 2026-06-09 at 11:44 +0300, Anton Mikanovich wrote:
> Task do_copy_boot_files deploys DTB files into the same location for
> different images (e.g., isar-image-base and isar-image-ci). This causes
> a build error.
> 
> do_copy_boot_files is called once for every image recipe while dtb
> files belong to the kernel which is the same for both images. Performing
> dtb deployment once for the same kernel solves the issue.
> 
> Introduce universal `kernel-deploy` bbclass responsible for extracting
> the DTBs and kernel from the linux-image package and deploying them.
> 
> For the distro kernel type its package is now installed into sbuild
> chroot. After that, deb which contains vmlinu[xz] file is extracted
> from the apt cache.
> 
> Fixes test_dtb_deploy_images testcase:
> 
> ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
>   build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
>     (not matched to any task)
> Please verify which recipe should provide the above files.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  RECIPE-API-CHANGELOG.md                      | 13 ++++++
>  meta/classes-recipe/image.bbclass            | 39 +++++++++--------
>  meta/classes-recipe/linux-deploy.bbclass     | 46 ++++++++++++++++++++
>  meta/classes-recipe/linux-kernel.bbclass     |  3 ++
>  meta/recipes-kernel/linux/files/getkernel.sh | 40 +++++++++++++++++
>  meta/recipes-kernel/linux/files/rules.tmpl   | 12 +++++
>  meta/recipes-kernel/linux/linux-distro.bb    | 27 ++++++++++++
>  testsuite/citest.py                          |  2 -
>  8 files changed, 161 insertions(+), 21 deletions(-)
>  create mode 100644 meta/classes-recipe/linux-deploy.bbclass
>  create mode 100755 meta/recipes-kernel/linux/files/getkernel.sh
>  create mode 100644 meta/recipes-kernel/linux/files/rules.tmpl
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index a6ded089..ef9dce00 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -1102,3 +1102,16 @@ To prevent this new path is separated also by distro and kernel values.
>  
>  This change will influence on build artifacts location and should be taken
>  into account by downstreams.
> +
> +### Move kernel and DTBs deployment from image recipe to kernel one
> +
> +To prevent parallel conflicting artifacts deployment from the same kernel
> +used by different images, move original deployment logic from
> +do_copy_boot_files() task of the image recipe to the kernel recipe. As we have
> +two types of kernel (distro and self build), both are affected. With this
> +aproach the deployment of any kernel artifacts will be performed only once per
> +kernel.
> +
> +Also allow using different kernels for one machine and distro without
> +overwritting by storing kernel artifacts in kernel-{KERNEL_NAME} subdirectory
> +of DEPLOY_DIR_IMAGE. Previous image linking names are kept by symlinks.
> diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
> index 26a4ec06..5f34cd4f 100644
> --- a/meta/classes-recipe/image.bbclass
> +++ b/meta/classes-recipe/image.bbclass
> @@ -1,5 +1,7 @@
>  # This software is a part of Isar.
> -# Copyright (C) 2015-2017 ilbers GmbH
> +# Copyright (C) 2015-2026 ilbers GmbH
> +#
> +# SPDX-License-Identifier: MIT
>  
>  # Make workdir and stamps machine-specific without changing common PN target
>  WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
> @@ -9,7 +11,6 @@ STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
>  
>  # Sstate also needs to be machine-specific
>  SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
> -SSTATETASKS += "do_copy_boot_files"
>  
>  IMAGE_INSTALL ?= ""
>  IMAGE_FSTYPES ?= "ext4"
> @@ -381,37 +382,37 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_DEPLOY_FILE}"
>  # only one dtb file supported, pick the first
>  DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"
>  
> -do_copy_boot_files[cleandirs] += "${DEPLOYDIR}"
> -do_copy_boot_files[sstate-inputdirs] = "${DEPLOYDIR}"
> -do_copy_boot_files[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
> -do_copy_boot_files[network] = "${TASK_USE_SUDO}"
> +KERNEL_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}/kernel-${KERNEL_NAME}"
> +KERNEL_WILDCARD = "${@ 'vmlinu[xz]*' if (p := d.getVar('KERNEL_FILE')) == 'vmlinux' else p+'*'}"
> +
> +python() {
> +    if d.getVar('KERNEL_NAME'):
> +        pn = d.getVar('KERNEL_IMAGE_PKG') or ''
> +        task = 'do_deploy_kernel_%s' % (d.getVar('MACHINE').replace('-','_') or '')
> +        d.appendVarFlag("do_copy_boot_files", "depends", f"{pn}:{task}")
> +}
> +
> +# Associate kernel with image by symlinks
> +do_copy_boot_files[dirs] += "${DEPLOY_DIR_IMAGE}"
> +do_copy_boot_files[file-checksums] += "${KERNEL_DEPLOY_DIR}/${KERNEL_WILDCARD}:True"
>  do_copy_boot_files() {
> -    kernel="$(realpath -q '${IMAGE_ROOTFS}'/vmlinu[xz])"
> -    if [ ! -f "$kernel" ]; then
> -        kernel="$(realpath -q '${IMAGE_ROOTFS}'/boot/vmlinu[xz])"
> -    fi
> +    kernel="$(realpath -mq '${KERNEL_DEPLOY_DIR}/'${KERNEL_WILDCARD} | head -n1)"
>      if [ -f "$kernel" ]; then
> -        sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
> +        ln -sfr "$kernel" "${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE}"

These symlinks will be dangling when running bitbake -c clean on the
kernel recipe. This is because the kernel is deployed via sstate cache,
but the symlink is manually created.

I have no idea how to fix this...

>      fi
>  
>      for file in ${DTB_FILES}; do
> -        dtb="$(find '${IMAGE_ROOTFS}/usr/lib' -type f \
> -                    -iwholename '*linux-image-*/'${file} | head -1)"
> +        dtb="${KERNEL_DEPLOY_DIR}/$(basename ${file})"
>  
>          if [ -z "$dtb" -o ! -e "$dtb" ]; then
>              die "${file} not found"
>          fi
>  
> -        cp -f "$dtb" "${DEPLOYDIR}/"
> +        ln -sfr "$dtb" "${DEPLOY_DIR_IMAGE}/$(basename $dtb)"

Same here.

>      done
>  }
>  addtask copy_boot_files before do_rootfs_postprocess after do_rootfs_install
>  
> -python do_copy_boot_files_setscene () {
> -    sstate_setscene(d)
> -}
> -addtask do_copy_boot_files_setscene
> -
>  python do_image_tools() {
>      """Virtual task"""
>      pass
> diff --git a/meta/classes-recipe/linux-deploy.bbclass b/meta/classes-recipe/linux-deploy.bbclass
> new file mode 100644
> index 00000000..cb29065d
> --- /dev/null
> +++ b/meta/classes-recipe/linux-deploy.bbclass
> @@ -0,0 +1,46 @@
> +# This software is a part of Isar.
> +# Copyright (C) 2026 ilbers GmbH
> +#
> +# SPDX-License-Identifier: MIT
> +
> +DEPLOYDIR = "${WORKDIR}/deploy_${@ d.getVar('MACHINE').replace('-','_') or ''}"
> +KERNEL_DEPLOY_TASKNAME ?= "do_deploy_kernel_${@ d.getVar('MACHINE').replace('-','_') or ''}"
> +SSTATETASKS += "${KERNEL_DEPLOY_TASKNAME}"
> +
> +python () {
> +    kernel_name = d.getVar('KERNEL_NAME_PROVIDED') or ''
> +    if "linux-image-"+kernel_name in d.getVar('PROVIDES'):
> +        task = d.getVar('KERNEL_DEPLOY_TASKNAME')
> +        d.setVar(task, d.expand('kernel_deploy'))
> +        d.setVarFlag(task, 'func', '1')
> +        d.setVarFlag(task, 'sstate-inputdirs', d.expand('${DEPLOYDIR}'))
> +        d.setVarFlag(task, 'sstate-outputdirs', d.expand('${KERNEL_DEPLOY_DIR}'))

> +        d.appendVarFlag(task, 'cleandirs', d.expand('${DEPLOYDIR}'))
> +        d.appendVarFlag(task, 'stamp-extra-info', d.expand('${MACHINE}'))

Why not simply d.getVar()?

> +        bb.build.addtask(task, 'do_build', 'do_dpkg_build', d)
> +}
> +
> +KERNEL_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}/kernel-${KERNEL_NAME_PROVIDED}"
> +
> +KERNEL_LOCATION ?= "./boot"
> +KERNEL_DEB ?= "linux-image-${KERNEL_NAME_PROVIDED}_${CHANGELOG_V}_${DISTRO_ARCH}.deb"
> +
> +# Take care the case when requested kernel format doesn't match distro one
> +DEPLOY_WILDCARDS = "'${KERNEL_LOCATION}/${@ 'vmlinu[xz]*' if (p := d.getVar('KERNEL_FILE')) == 'vmlinux' else p+'*'}'"
> +DEPLOY_WILDCARDS += "${@(' '.join("'*%s'" % p for p in (d.getVar('DTB_FILES') or '').split()))}"
> +
> +kernel_deploy() {
> +       case "${PROVIDES}" in
> +               *linux-image-${KERNEL_NAME_PROVIDED}*)
> +                       dpkg --fsys-tarfile ${WORKDIR}/${KERNEL_DEB} | \
> +                               tar xvf - -C "${DEPLOYDIR}" \
> +                                       --transform='s|^.*/||' \
> +                                       --wildcards ${DEPLOY_WILDCARDS}
> +               ;;
> +       esac
> +}
> +
> +python do_copy_boot_files_setscene () {
> +    sstate_setscene(d)
> +}
> +addtask do_copy_boot_files_setscene
> diff --git a/meta/classes-recipe/linux-kernel.bbclass b/meta/classes-recipe/linux-kernel.bbclass
> index e4ae356d..32d10f68 100644
> --- a/meta/classes-recipe/linux-kernel.bbclass
> +++ b/meta/classes-recipe/linux-kernel.bbclass
> @@ -3,6 +3,7 @@
>  # This software is a part of Isar.
>  # Copyright (c) Siemens AG, 2022
>  # Copyright (c) Mentor Graphics, a Siemens business, 2022
> +# Copyright (C) 2022-2026 ilbers GmbH
>  #
>  # SPDX-License-Identifier: MIT
>  
> @@ -337,3 +338,5 @@ do_dpkg_source:prepend() {
>  	dpkg_configure_kernel
>  	get_localversion_auto
>  }
> +
> +inherit linux-deploy
> diff --git a/meta/recipes-kernel/linux/files/getkernel.sh b/meta/recipes-kernel/linux/files/getkernel.sh
> new file mode 100755
> index 00000000..7070dbc0
> --- /dev/null
> +++ b/meta/recipes-kernel/linux/files/getkernel.sh
> @@ -0,0 +1,40 @@
> +#!/bin/bash -e
> +
> +deb_cache="/var/cache/apt/archives"
> +
> +paths="/vmlinu[xz] /boot/vmlinu[xz]"
> +if [ -n "$1" ]; then
> +    paths="/$1 /boot/$1 $paths"
> +fi
> +
> +# Lookup for the kernel file
> +for path in ${paths}; do
> +    kernel="$(realpath -q ${path})"
> +    if [ -f "${kernel}" ]; then
> +        break
> +    fi
> +done
> +
> +# Obtain package name for the kernel file
> +pkg="$(dpkg -S ${kernel} | cut -d':' -f1)"
> +if [ -z "${pkg}" ]; then
> +    >&2 echo "No package providing ${kernel} found!"
> +    exit 1
> +fi
> +
> +# Query for deb filename
> +deb_name=$(dpkg-query -W -f='${Package}_${Version}_${Architecture}.deb\n' ${pkg})
> +
> +# Take care about special symbols
> +deb_name="${deb_name//%/%25}"
> +deb_name="${deb_name//:/%3a}"
> +deb_name="${deb_name//~/%7e}"
> +
> +# Search for deb in cache dir
> +deb_path="$(find ${deb_cache} -name "${deb_name}" 2>/dev/null | head -n1)"
> +if [ ! -f "${deb_path}" ]; then
> +    >&2 echo "Package ${deb_name} not found in ${deb_cache}!"
> +    exit 1
> +fi
> +
> +echo "${deb_path}"
> diff --git a/meta/recipes-kernel/linux/files/rules.tmpl b/meta/recipes-kernel/linux/files/rules.tmpl
> new file mode 100644
> index 00000000..69d79bb5
> --- /dev/null
> +++ b/meta/recipes-kernel/linux/files/rules.tmpl
> @@ -0,0 +1,12 @@
> +#!/usr/bin/make -f
> +
> +KERNEL_PATH := $(shell ./debian/getkernel.sh ${KERNEL_FILE})
> +DEB_NAME := ${KERNEL_DEB}
> +
> +binary:
> +	@[ -z "$(KERNEL_PATH)" ] && { echo "Kernel not found!"; exit 1; } || true
> +	cp "$(KERNEL_PATH)" ../$(DEB_NAME)
> +	echo "$(DEB_NAME) misc optional" > debian/files
> +
> +%:
> +	true
> diff --git a/meta/recipes-kernel/linux/linux-distro.bb b/meta/recipes-kernel/linux/linux-distro.bb
> index 8fc1bcb7..47fe3fb4 100644
> --- a/meta/recipes-kernel/linux/linux-distro.bb
> +++ b/meta/recipes-kernel/linux/linux-distro.bb
> @@ -2,6 +2,7 @@
>  #
>  # This software is a part of Isar.
>  # Copyright (c) Siemens AG, 2018
> +# Copyright (C) 2022-2026 ilbers GmbH
>  #
>  # SPDX-License-Identifier: MIT
>  
> @@ -27,3 +28,29 @@ python() {
>  }
>  
>  inherit multiarch
> +inherit dpkg
> +inherit linux-deploy
> +
> +# Always use target arch for kernel package lookup
> +ISAR_CROSS_COMPILE = "0"
> +
> +MAINTAINER = "isar-users <isar-users@googlegroups.com>"
> +
> +PN .= "-${KERNEL_NAME}"
> +KERNEL_NAME_PROVIDED ??= "${KERNEL_NAME}"
> +DEBIAN_BUILD_DEPENDS ?= "${@d.getVar('KERNEL_IMAGE_PKG') or ('linux-image-' + (d.getVar('KERNEL_NAME') or ''))}"
> +
> +FILESPATH:prepend = "${LAYERDIR_core}/recipes-kernel/linux/files:"
> +
> +SRC_URI = "file://getkernel.sh \
> +           file://rules.tmpl"
> +
> +TEMPLATE_VARS += "KERNEL_FILE KERNEL_DEB"
> +TEMPLATE_FILES = "rules.tmpl"
> +
> +do_prepare_build[cleandirs] += "${S}/debian"
> +do_prepare_build() {
> +    deb_debianize
> +    cp "${WORKDIR}/getkernel.sh" "${S}/debian/"
> +}
> +do_deploy_deb[noexec] = "1"

That is definitely a hack. I would prefer if linux-distro can avoid
inheriting dpkg, as this simply is not generating a package but just
registering dependencies / provides.

Felix

> diff --git a/testsuite/citest.py b/testsuite/citest.py
> index 7d666880..5aa3e799 100644
> --- a/testsuite/citest.py
> +++ b/testsuite/citest.py
> @@ -740,8 +740,6 @@ class DtbDeployTest(CIBaseTest):
>          self.init()
>          try:
>              self.perform_build_test(targets, image_install='')
> -        except exceptions.TestFail:
> -            self.cancel('KFAIL')
>          finally:
>              self.move_in_build_dir('tmp', 'tmp_dtbdeploy')
>  
> -- 
> 2.34.1
> 
> -- 
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260609084422.3948645-3-amikan%40ilbers.de.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/1402cd5ce266312076b38a6896873abd9615efd7.camel%40siemens.com.