Re: [PATCH 2/2] prefix DTB files with PN in deploy dir

["'Jan Kiszka' via isar-users" <isar-users@googlegroups.com>]

On 10.04.26 16:12, Moessbauer, Felix (FT RPD CED OES-DE) wrote:
> On Fri, 2026-04-10 at 15:38 +0200, Jan Kiszka wrote:
>> On 10.04.26 15:22, Felix Moessbauer wrote:
>>> As the changing of the DEPLOY_DIR_IMAGE has proven to be fundamentally
>>> incompatible with custom initrd recipes, this patch was reverted,
>>> re-introducing the do_copy_boot_files error on DTBs that are named
>>> equally but belong to different mc targets.
>>>
>>> To mitigate this limitation without breaking custom initrds, we prefix
>>> all DTB files with ${PN}-${DISTRO} when deploying to DEPLOY_IMAGE_DIR.
>>> On imaging, these prefixes are stripped again by the imager scripts.
>>>
>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
>>> ---
>>>  RECIPE-API-CHANGELOG.md                       | 20 +++++++++++++++++++
>>>  meta/classes-recipe/image.bbclass             |  6 ++++--
>>>  meta/classes-recipe/imagetypes_wic.bbclass    |  2 +-
>>>  .../wic/plugins/source/bootimg-efi-isar.py    |  3 ++-
>>>  4 files changed, 27 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
>>> index 0e6a3172..856da5de 100644
>>> --- a/RECIPE-API-CHANGELOG.md
>>> +++ b/RECIPE-API-CHANGELOG.md
>>> @@ -981,3 +981,23 @@ fragment, this can be specified via adding `${S}/path/to/fragment.cfg` to
>>>  `KERNEL_CONFIG_FRAGMENTS`. If a fragment was checked out into ${WORKDIR} as
>>>  part of a repository, a tarball, or some other directory structure, just
>>>  specify it relative to ${WORKDIR} in `KERNEL_CONFIG_FRAGMENTS`.
>>> +
>>> +Changes in next
>>> +---------------
>>> +
>>> +### Prefix DTB file names when deploying
>>> +
>>> +DTB files are now placed in the ${DEPLOY_DIR_IMAGE} with a prefix of
>>> +${PN}-${DISTRO}. During wic imaging, the prefix is removed again, so no changes
>>> +to downstream wks files are needed (i.e. `dtb=my-device-tree.dtb` is not
>>> +affected by this change). Custom imaging plugins need to be adapted to this
>>> +change by removing the prefix from the filename. For that, the variable
>>> +DTB_PREFIX is exported as bitbake var into wic environment.

Downstream pick-up scripts like
https://gitlab.com/Xenomai/xenomai-images/-/blob/master/scripts/deploy_to_aws.sh?ref_type=heads
will need adjustments as well.

>>> +
>>> +This fixes errors when building different distros with the same machine,
>>> +whereby previously the following error occured:
>>> +
>>> +do_copy_boot_files: The recipe isar-image-base is trying to install
>>> +files into a shared area when those files already exists. It happens
>>> +when some files have the same names (e.g., dtb files) for different
>>> +distros.
>>> diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
>>> index 26a4ec06..9b5dd23e 100644
>>> --- a/meta/classes-recipe/image.bbclass
>>> +++ b/meta/classes-recipe/image.bbclass
>>> @@ -379,7 +379,8 @@ EOF
>>>  KERNEL_IMG = "${PP_DEPLOY}/${KERNEL_IMAGE}"
>>>  INITRD_IMG = "${PP_DEPLOY}/${INITRD_DEPLOY_FILE}"
>>>  # only one dtb file supported, pick the first
>>> -DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"
>>> +DTB_PREFIX = "${PN}-${DISTRO}."
>>> +DTB_IMG = "${PP_DEPLOY}/${DTB_PREFIX}${@os.path.basename((d.getVar('DTB_FILES').split() or [''])[0])}"
>>>  
>>>  do_copy_boot_files[cleandirs] += "${DEPLOYDIR}"
>>>  do_copy_boot_files[sstate-inputdirs] = "${DEPLOYDIR}"
>>> @@ -402,7 +403,8 @@ do_copy_boot_files() {
>>>              die "${file} not found"
>>>          fi
>>>  
>>> -        cp -f "$dtb" "${DEPLOYDIR}/"
>>> +        dtb_name=$(basename "$dtb")
>>> +        cp -f "$dtb" "${DEPLOYDIR}/${DTB_PREFIX}$dtb_name"
>>
>> Let's but the DTBs in prefix-subdirs - will also make referring to them
>> cleaner.
> 
> I explicitly decided against putting them in subdirs to not diverge
> from the names of the other artifacts. However, OE deploys the DTBs via
> the devicetree class to ${IMAGE_DEPLOY_DIR}/devicetree/ , which does
> not help much as we still would get a clash. In addition, it deploys to
> sysroot where other recipes should consume it from [1]. But we could do
> that cleanup while we are at it.

Which of the cleanups? There is no sysroot in Isar. Rather, we would
have to add a package to the chroot of the imager and pick it up from there.

> 
> Another reason for not putting them in a directory are file globs
> commonly used in CI to copy out all files in the deploy dir.

See above, those will have to be adjusted because you often cannot pick
up the mangled DTB file names. That is why I suggested a directory.

> 
> [1]
> https://docs.yoctoproject.org/dev/ref-manual/classes.html#devicetree
> 
> Apart from that, OE anyways discourages direct deploy to
> DEPLOY_DIR_IMAGE. Instead the deploy.bbclass class should be used to
> deploy through the sstate cache.
> 

Then let's try that and see if it helps better.

>>
>>>      done
>>>  }
>>>  addtask copy_boot_files before do_rootfs_postprocess after do_rootfs_install
>>> diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
>>> index dd6c501d..c0813223 100644
>>> --- a/meta/classes-recipe/imagetypes_wic.bbclass
>>> +++ b/meta/classes-recipe/imagetypes_wic.bbclass
>>> @@ -107,7 +107,7 @@ WICVARS += "\
>>>             ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS TRANSLATED_TARGET_ARCH"
>>>  
>>>  # Isar specific vars used in our plugins
>>> -WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE MACHINE"
>>> +WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE MACHINE DTB_PREFIX"
>>>  
>>>  python do_rootfs_wicenv () {
>>>      wicvars = d.getVar('WICVARS')
>>> diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
>>> index 6bc78d42..32b220fa 100644
>>> --- a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
>>> +++ b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
>>> @@ -57,7 +57,8 @@ class BootimgEFIPlugin(SourcePlugin):
>>>          if dtb:
>>>              if ';' in dtb:
>>>                  raise WicError("Only one DTB supported, exiting")
>>> -            cp_cmd = "cp %s/%s %s" % (bootimg_dir, dtb, hdddir)
>>> +            dtb_file = "%s%s" % (get_bitbake_var("DTB_PREFIX"), dtb)
>>> +            cp_cmd = "cp %s/%s %s/%s" % (bootimg_dir, dtb_file, hdddir, dtb)
>>>              exec_cmd(cp_cmd, True)
>>>  
>>>      @classmethod
>>
>> DTBs are the most prominent conflicts with multiconfigs or partial
>> rebuilds. We may mitigate this one, so it's fine, but the fundamental
>> risk will remain. One of the reason why I asked to study OE carefully
>> and try to learn from it first.
> 
> In OE these problems are also not solved. Even the split of the TMPDIR
> is done in downstream layers, not OE-core (at least I found nothing in
> OE core).

Then we are either overusing multiconfig here or are still missing some
other detail, such as indirect deployment.

As the DTB deployment conflict is existing in Isar for many years, only
affecting in practice its own setup, I would suggest to take the revert
quickly into the tree and possibly postpone a real solution after
further research.

I'm also concerned that the pattern applied here will not easily scale
to similar problems around other artifacts we deploy in various
downstream layers. We are producing the same error around its
firmware.bin when rebuilding isar-cip-core for different distros e.g.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/cf057977-bbee-4da6-914e-a7eb287505b1%40siemens.com.