* [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps
@ 2025-08-20 12:41 'Felix Moessbauer' via isar-users
2025-08-20 12:41 ` [PATCH 1/2] dpkg-raw: add files to source package 'Felix Moessbauer' via isar-users
` (2 more replies)
0 siblings, 3 replies; 21+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2025-08-20 12:41 UTC (permalink / raw)
To: isar-users
Cc: jan.kiszka, cedric.hombourger, adriaan.schmidt,
christoph.steiger, Felix Moessbauer
This series fixes an issue with DPKG_ARCH=all packages that have
dependencies to other DPKG_ARCH=all packages. Hereby, the transitive
packages were built for the wrong architecture. The issue became
apparent when testing the SBOM series by Christoph, which builds
packages to be used during the ISAR build (not for the final target).
While fixing this, another issue surfaced where files are missing in
dpkg-raw packages.
Best regards,
Felix
Felix Moessbauer (2):
dpkg-raw: add files to source package
handle DPKG_ARCH=all case for transitive deps
meta/classes/dpkg-raw.bbclass | 4 ++--
meta/classes/multiarch.bbclass | 18 +++++++++++++++++-
2 files changed, 19 insertions(+), 3 deletions(-)
--
2.50.1
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250820124140.281190-1-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH 1/2] dpkg-raw: add files to source package 2025-08-20 12:41 [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users @ 2025-08-20 12:41 ` 'Felix Moessbauer' via isar-users 2025-10-03 11:09 ` Anton Mikanovich 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-10-24 5:34 ` [PATCH 0/2] " 'cedric.hombourger@siemens.com' via isar-users 2 siblings, 1 reply; 21+ messages in thread From: 'Felix Moessbauer' via isar-users @ 2025-08-20 12:41 UTC (permalink / raw) To: isar-users Cc: jan.kiszka, cedric.hombourger, adriaan.schmidt, christoph.steiger, Felix Moessbauer In dpkg-raw, the user can place files in ${D} which are then installed into the resulting binary package. Hereby, ${D} is a directory outside of ${S} (since b19cd25) to not interfere with other data added to ${S}. However, by that the files are not added to the source package. This remained unnoticed, as the directory dh_install installs from is set to absolute path, hence the installed files actually came from the absolute path and not from the extracted source package. In case of ${PN} == ${BPN}, this path was always there as it has been created by previous tasks. However, with the switch in 2ca3a7e5 to only build the source package once, the path is not always there. We fix this by adding the files to the source package (under image) and install from a relative base. We further use a sub-path (image) in ${S} as a temporary location to not run into the issue solved in b19cd25. Fixes: 2ca3a7e5 ("dpkg-source: Build source package only once") Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> --- meta/classes/dpkg-raw.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass index a7bf204a..d4cb7d8a 100644 --- a/meta/classes/dpkg-raw.bbclass +++ b/meta/classes/dpkg-raw.bbclass @@ -5,7 +5,7 @@ inherit dpkg -D = "${WORKDIR}/image" +D = "${S}/image" # Default to creating a binary-indep package DPKG_ARCH ??= "all" @@ -30,6 +30,6 @@ do_prepare_build() { cat <<EOF >> ${S}/debian/rules override_dh_install: - dh_install --sourcedir=${PP}/image + dh_install --sourcedir=image EOF } -- 2.50.1 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250820124140.281190-2-felix.moessbauer%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 1/2] dpkg-raw: add files to source package 2025-08-20 12:41 ` [PATCH 1/2] dpkg-raw: add files to source package 'Felix Moessbauer' via isar-users @ 2025-10-03 11:09 ` Anton Mikanovich 0 siblings, 0 replies; 21+ messages in thread From: Anton Mikanovich @ 2025-10-03 11:09 UTC (permalink / raw) To: Felix Moessbauer, isar-users Cc: jan.kiszka, cedric.hombourger, adriaan.schmidt, christoph.steiger 20/08/2025 15:41, 'Felix Moessbauer' via isar-users wrote: > In dpkg-raw, the user can place files in ${D} which are then installed > into the resulting binary package. Hereby, ${D} is a directory outside > of ${S} (since b19cd25) to not interfere with other data added to ${S}. > However, by that the files are not added to the source package. This > remained unnoticed, as the directory dh_install installs from is set > to absolute path, hence the installed files actually came from the > absolute path and not from the extracted source package. In case of > ${PN} == ${BPN}, this path was always there as it has been created by > previous tasks. However, with the switch in 2ca3a7e5 to only build the > source package once, the path is not always there. > > We fix this by adding the files to the source package (under image) and > install from a relative base. We further use a sub-path (image) in ${S} > as a temporary location to not run into the issue solved in b19cd25. > > Fixes: 2ca3a7e5 ("dpkg-source: Build source package only once") > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> p1 was applied to next, thanks. -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/5da617f1-ea5a-4838-91ff-da5b18c7e14a%40ilbers.de. ^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-08-20 12:41 [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-08-20 12:41 ` [PATCH 1/2] dpkg-raw: add files to source package 'Felix Moessbauer' via isar-users @ 2025-08-20 12:41 ` 'Felix Moessbauer' via isar-users 2025-09-10 11:00 ` Andreas Naumann ` (3 more replies) 2025-10-24 5:34 ` [PATCH 0/2] " 'cedric.hombourger@siemens.com' via isar-users 2 siblings, 4 replies; 21+ messages in thread From: 'Felix Moessbauer' via isar-users @ 2025-08-20 12:41 UTC (permalink / raw) To: isar-users Cc: jan.kiszka, cedric.hombourger, adriaan.schmidt, christoph.steiger, Felix Moessbauer Arch=all packages might build depend on other arch=all packages, hence we need to correctly model the dependency chain. Otherwise the transitive dependencies are built for the distro arch instead of the native arch. We implement this by dispatching the non-native variant of DPKG_ARCH=all packages to the -native variant by adding a dependency. We further replace the non-native do_deploy_dep task with a noop to preserve the dependency chain. Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> --- meta/classes/multiarch.bbclass | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass index babdfbd4..c2bba21f 100644 --- a/meta/classes/multiarch.bbclass +++ b/meta/classes/multiarch.bbclass @@ -29,7 +29,11 @@ python() { d.appendVar('BBCLASSEXTEND', ' compat') # build native separately only when it differs from the target variant - if not archIsAll and archDiffers: + # We must not short-circuit for DPKG_ARCH=all packages, as they might + # have transitive dependencies which need to be built for -native. + # This special handling for DPKG_ARCH=all packages is left to the + # multiarch_virtclass_handler + if archDiffers: d.appendVar('BBCLASSEXTEND', ' native') else: extend_provides(pn, 'native', d) @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { d.setVar(var, ' '.join(multiarch_var)) pn = e.data.getVar('PN') + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') + archIsAll = d.getVar('DPKG_ARCH') == 'all' if pn.endswith('-compat'): e.data.setVar('BPN', pn[:-len('-compat')]) e.data.appendVar('OVERRIDES', ':class-compat') @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { e.data.appendVar('OVERRIDES', ':class-native') fixup_pn_in_vars(e.data) fixup_depends('-native', e.data) + elif archIsAll and archDiffers: + # Arch=all packages might build depend on other arch=all packages, + # hence we need to correctly model the dependency chain. + # We implement this by dispatching the non-native variant to the -native + # variant by adding a dependency. We further replace the non-native + # do_deploy_dep task with a noop to preserve the dependency chain. + e.data.setVar('do_deploy_deb', '') + bb.build.deltask('deploy_deb', e.data) + bb.build.addtask('deploy_deb', 'do_build', '', e.data) + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') } addhandler multiarch_virtclass_handler multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" -- 2.50.1 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250820124140.281190-3-felix.moessbauer%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users @ 2025-09-10 11:00 ` Andreas Naumann 2025-09-11 10:20 ` 'MOESSBAUER, Felix' via isar-users 2025-09-15 17:49 ` Andreas Naumann ` (2 subsequent siblings) 3 siblings, 1 reply; 21+ messages in thread From: Andreas Naumann @ 2025-09-10 11:00 UTC (permalink / raw) To: isar-users, felix.moessbauer Hi Felix, all, Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: > Arch=all packages might build depend on other arch=all packages, hence we > need to correctly model the dependency chain. Otherwise the transitive > dependencies are built for the distro arch instead of the native arch. > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > packages to the -native variant by adding a dependency. We further > replace the non-native do_deploy_dep task with a noop to preserve the > dependency chain. > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > index babdfbd4..c2bba21f 100644 > --- a/meta/classes/multiarch.bbclass > +++ b/meta/classes/multiarch.bbclass > @@ -29,7 +29,11 @@ python() { > d.appendVar('BBCLASSEXTEND', ' compat') > > # build native separately only when it differs from the target variant > - if not archIsAll and archDiffers: > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > + # have transitive dependencies which need to be built for -native. Funny enough i suspected to have the same problem yesterday, but it turned out to be different: When a package is single architecture only, e.g. arm64, it should not be extended as native if HOST_ARCH is amd64. So I added a check to BBCLASSEXTEND only if DPKG_ARCH != HOST_ARCH. > + # This special handling for DPKG_ARCH=all packages is left to the > + # multiarch_virtclass_handler > + if archDiffers: > d.appendVar('BBCLASSEXTEND', ' native') Now, seeing your patch to remove the not-"all" case also, I wonder if that could be used as the sole check before BBCLASSEXTEND. Like e.g. archIsNative = d.getVar('DPKG_ARCH') == d.getVar('HOST_ARCH') if isNative: d.appendVar('BBCLASSEXTEND', ' native') > else: > extend_provides(pn, 'native', d) but now I wonder what the extend_provides is needed for. If HOST_ARCH == DISTRO_ARCH, what's the use of providing a native variant? But back to your Patch: Is it possible that -compat "all" packages also have dependencies on other "all" packages, which in turn have dependencies on architecture specific packages? regards, Andreas > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > d.setVar(var, ' '.join(multiarch_var)) > > pn = e.data.getVar('PN') > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > if pn.endswith('-compat'): > e.data.setVar('BPN', pn[:-len('-compat')]) > e.data.appendVar('OVERRIDES', ':class-compat') > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > e.data.appendVar('OVERRIDES', ':class-native') > fixup_pn_in_vars(e.data) > fixup_depends('-native', e.data) > + elif archIsAll and archDiffers: > + # Arch=all packages might build depend on other arch=all packages, > + # hence we need to correctly model the dependency chain. > + # We implement this by dispatching the non-native variant to the -native > + # variant by adding a dependency. We further replace the non-native > + # do_deploy_dep task with a noop to preserve the dependency chain. > + e.data.setVar('do_deploy_deb', '') > + bb.build.deltask('deploy_deb', e.data) > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') > } > addhandler multiarch_virtclass_handler > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mail info@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/24e7baaa-c347-4327-b06d-25c9a074a494%40emlix.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-09-10 11:00 ` Andreas Naumann @ 2025-09-11 10:20 ` 'MOESSBAUER, Felix' via isar-users 2025-09-12 15:50 ` Andreas Naumann 0 siblings, 1 reply; 21+ messages in thread From: 'MOESSBAUER, Felix' via isar-users @ 2025-09-11 10:20 UTC (permalink / raw) To: Schmidt, Adriaan, anaumann; +Cc: isar-users On Wed, 2025-09-10 at 13:00 +0200, Andreas Naumann wrote: > Hi Felix, all, > > Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: > > Arch=all packages might build depend on other arch=all packages, hence we > > need to correctly model the dependency chain. Otherwise the transitive > > dependencies are built for the distro arch instead of the native arch. > > > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > > packages to the -native variant by adding a dependency. We further > > replace the non-native do_deploy_dep task with a noop to preserve the > > dependency chain. > > > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > > --- > > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > > index babdfbd4..c2bba21f 100644 > > --- a/meta/classes/multiarch.bbclass > > +++ b/meta/classes/multiarch.bbclass > > @@ -29,7 +29,11 @@ python() { > > d.appendVar('BBCLASSEXTEND', ' compat') > > > > # build native separately only when it differs from the target variant > > - if not archIsAll and archDiffers: > > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > > + # have transitive dependencies which need to be built for -native. > > Funny enough i suspected to have the same problem yesterday, but it > turned out to > be different: When a package is single architecture only, e.g. arm64, it > should not be > extended as native if HOST_ARCH is amd64. > > So I added a check to BBCLASSEXTEND only if DPKG_ARCH != HOST_ARCH. Good catch. This case is currently not handled. But I would like to have it in a separate patch, as it tackles another issue. > > > + # This special handling for DPKG_ARCH=all packages is left to the > > + # multiarch_virtclass_handler > > + if archDiffers: > > d.appendVar('BBCLASSEXTEND', ' native') > Now, seeing your patch to remove the not-"all" case also, I wonder if > that could be used > as the sole check before BBCLASSEXTEND. Like e.g. > archIsNative = d.getVar('DPKG_ARCH') == d.getVar('HOST_ARCH') > if isNative: > d.appendVar('BBCLASSEXTEND', ' native') > > else: > > extend_provides(pn, 'native', d) > > but now I wonder what the extend_provides is needed for. If HOST_ARCH == > DISTRO_ARCH, what's > the use of providing a native variant? If we are building for the native arch, we can just add the -native variants to the PROVIDES, so dependencies of packages depending explicitly on <foo>-native can be resolved (without the need for extra recipe variants). > > But back to your Patch: Is it possible that -compat "all" packages also have > dependencies on other "all" packages, which in turn have dependencies on > architecture specific packages? Good question. In general, package dependencies are resolved for the native architecture (or the requested one). AFAIK this selection is propagated down the dependency tree. By that, A:i386 -> B:all -> C:x should always resolve C as either :all or i386 (except if there is a explicit cross-arch dependency specifier like :any, which we don't model in bitbake). Anyways, The A:i386 -> B:all -> C:i386 chain should already be resolved correctly with this patch. Tested by's are welcome :) Felix > > regards, > Andreas > > > > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > > d.setVar(var, ' '.join(multiarch_var)) > > > > pn = e.data.getVar('PN') > > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > > if pn.endswith('-compat'): > > e.data.setVar('BPN', pn[:-len('-compat')]) > > e.data.appendVar('OVERRIDES', ':class-compat') > > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > > e.data.appendVar('OVERRIDES', ':class-native') > > fixup_pn_in_vars(e.data) > > fixup_depends('-native', e.data) > > + elif archIsAll and archDiffers: > > + # Arch=all packages might build depend on other arch=all packages, > > + # hence we need to correctly model the dependency chain. > > + # We implement this by dispatching the non-native variant to the -native > > + # variant by adding a dependency. We further replace the non-native > > + # do_deploy_dep task with a noop to preserve the dependency chain. > > + e.data.setVar('do_deploy_deb', '') > > + bb.build.deltask('deploy_deb', e.data) > > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') > > } > > addhandler multiarch_virtclass_handler > > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" > > -- > Andreas Naumann > > emlix GmbH > Headquarters: Berliner Str. 12, 37073 Goettingen, Germany > Phone +49 (0)551 30664-0, e-mail info@emlix.com > District Court of Goettingen, Registry Number HR B 3160 > Managing Directors: Heike Jordan, Dr. Uwe Kracke > VAT ID No. DE 205 198 055 > Office Berlin: Panoramastr. 1, 10178 Berlin, Germany > Office Bonn: Bachstr. 6, 53115 Bonn, Germany > http://www.emlix.com -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/b1b0bddfda7e982b274e4ebbca509fc1898200ab.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-09-11 10:20 ` 'MOESSBAUER, Felix' via isar-users @ 2025-09-12 15:50 ` Andreas Naumann 2025-09-15 7:55 ` 'MOESSBAUER, Felix' via isar-users 0 siblings, 1 reply; 21+ messages in thread From: Andreas Naumann @ 2025-09-12 15:50 UTC (permalink / raw) To: MOESSBAUER, Felix, Schmidt, Adriaan; +Cc: isar-users Am 11.09.25 um 12:20 schrieb MOESSBAUER, Felix: > On Wed, 2025-09-10 at 13:00 +0200, Andreas Naumann wrote: >> Hi Felix, all, >> >> Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: >>> Arch=all packages might build depend on other arch=all packages, hence we >>> need to correctly model the dependency chain. Otherwise the transitive >>> dependencies are built for the distro arch instead of the native arch. >>> >>> We implement this by dispatching the non-native variant of DPKG_ARCH=all >>> packages to the -native variant by adding a dependency. We further >>> replace the non-native do_deploy_dep task with a noop to preserve the >>> dependency chain. >>> >>> Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> >>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> >>> --- >>> meta/classes/multiarch.bbclass | 18 +++++++++++++++++- >>> 1 file changed, 17 insertions(+), 1 deletion(-) >>> >>> diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass >>> index babdfbd4..c2bba21f 100644 >>> --- a/meta/classes/multiarch.bbclass >>> +++ b/meta/classes/multiarch.bbclass >>> @@ -29,7 +29,11 @@ python() { >>> d.appendVar('BBCLASSEXTEND', ' compat') >>> >>> # build native separately only when it differs from the target variant >>> - if not archIsAll and archDiffers: >>> + # We must not short-circuit for DPKG_ARCH=all packages, as they might >>> + # have transitive dependencies which need to be built for -native. >> Funny enough i suspected to have the same problem yesterday, but it >> turned out to >> be different: When a package is single architecture only, e.g. arm64, it >> should not be >> extended as native if HOST_ARCH is amd64. >> >> So I added a check to BBCLASSEXTEND only if DPKG_ARCH != HOST_ARCH. > Good catch. This case is currently not handled. But I would like to > have it in a separate patch, as it tackles another issue. I thought it's possible to make the nesting of if-clauses easier that way, but after some more thought and fiddling... unfortunately it's the opposite. So yes, separate patch. > >>> + # This special handling for DPKG_ARCH=all packages is left to the >>> + # multiarch_virtclass_handler >>> + if archDiffers: >>> d.appendVar('BBCLASSEXTEND', ' native') >> Now, seeing your patch to remove the not-"all" case also, I wonder if >> that could be used >> as the sole check before BBCLASSEXTEND. Like e.g. >> archIsNative = d.getVar('DPKG_ARCH') == d.getVar('HOST_ARCH') >> if isNative: >> d.appendVar('BBCLASSEXTEND', ' native') >>> else: >>> extend_provides(pn, 'native', d) >> but now I wonder what the extend_provides is needed for. If HOST_ARCH == >> DISTRO_ARCH, what's >> the use of providing a native variant? > If we are building for the native arch, we can just add the -native > variants to the PROVIDES, so dependencies of packages depending > explicitly on <foo>-native can be resolved (without the need for extra > recipe variants). I understand that its a simpler solution in that case. But, maybe I'm not seeing the obvious, what would require a -native package then? If target-arch is host-arch, why would anything else than the base package be in any dependency chain? > >> But back to your Patch: Is it possible that -compat "all" packages also have >> dependencies on other "all" packages, which in turn have dependencies on >> architecture specific packages? > Good question. In general, package dependencies are resolved for the > native architecture (or the requested one). AFAIK this selection is > propagated down the dependency tree. By that, A:i386 -> B:all -> C:x > should always resolve C as either :all or i386 (except if there is a > explicit cross-arch dependency specifier like :any, which we don't > model in bitbake). Anyways, The A:i386 -> B:all -> C:i386 chain should > already be resolved correctly with this patch. The patch doesnt change that -compat is "short-circuited" when archIsAll, doesnt it? Or maybe I dont fully understand how the code works yet. > > Tested by's are welcome :) I'm working with it and keep an eye on it. regards, Andreas > > Felix > >> regards, >> Andreas >> >> >>> @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { >>> d.setVar(var, ' '.join(multiarch_var)) >>> >>> pn = e.data.getVar('PN') >>> + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') >>> + archIsAll = d.getVar('DPKG_ARCH') == 'all' >>> if pn.endswith('-compat'): >>> e.data.setVar('BPN', pn[:-len('-compat')]) >>> e.data.appendVar('OVERRIDES', ':class-compat') >>> @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { >>> e.data.appendVar('OVERRIDES', ':class-native') >>> fixup_pn_in_vars(e.data) >>> fixup_depends('-native', e.data) >>> + elif archIsAll and archDiffers: >>> + # Arch=all packages might build depend on other arch=all packages, >>> + # hence we need to correctly model the dependency chain. >>> + # We implement this by dispatching the non-native variant to the -native >>> + # variant by adding a dependency. We further replace the non-native >>> + # do_deploy_dep task with a noop to preserve the dependency chain. >>> + e.data.setVar('do_deploy_deb', '') >>> + bb.build.deltask('deploy_deb', e.data) >>> + bb.build.addtask('deploy_deb', 'do_build', '', e.data) >>> + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') >>> } >>> addhandler multiarch_virtclass_handler >>> multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" >> -- >> Andreas Naumann >> >> emlix GmbH >> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany >> Phone +49 (0)551 30664-0, e-mail info@emlix.com >> District Court of Goettingen, Registry Number HR B 3160 >> Managing Directors: Heike Jordan, Dr. Uwe Kracke >> VAT ID No. DE 205 198 055 >> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany >> Office Bonn: Bachstr. 6, 53115 Bonn, Germany >> http://www.emlix.com -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mail info@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/f7ac34b9-c96b-4939-aec4-c72bcd471db9%40emlix.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-09-12 15:50 ` Andreas Naumann @ 2025-09-15 7:55 ` 'MOESSBAUER, Felix' via isar-users 0 siblings, 0 replies; 21+ messages in thread From: 'MOESSBAUER, Felix' via isar-users @ 2025-09-15 7:55 UTC (permalink / raw) To: Schmidt, Adriaan, anaumann; +Cc: isar-users On Fri, 2025-09-12 at 17:50 +0200, Andreas Naumann wrote: > Am 11.09.25 um 12:20 schrieb MOESSBAUER, Felix: > > On Wed, 2025-09-10 at 13:00 +0200, Andreas Naumann wrote: > > > Hi Felix, all, > > > > > > Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: > > > > Arch=all packages might build depend on other arch=all packages, hence we > > > > need to correctly model the dependency chain. Otherwise the transitive > > > > dependencies are built for the distro arch instead of the native arch. > > > > > > > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > > > > packages to the -native variant by adding a dependency. We further > > > > replace the non-native do_deploy_dep task with a noop to preserve the > > > > dependency chain. > > > > > > > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > > > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > > > > --- > > > > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > > > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > > > > index babdfbd4..c2bba21f 100644 > > > > --- a/meta/classes/multiarch.bbclass > > > > +++ b/meta/classes/multiarch.bbclass > > > > @@ -29,7 +29,11 @@ python() { > > > > d.appendVar('BBCLASSEXTEND', ' compat') > > > > > > > > # build native separately only when it differs from the target variant > > > > - if not archIsAll and archDiffers: > > > > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > > > > + # have transitive dependencies which need to be built for -native. > > > Funny enough i suspected to have the same problem yesterday, but it > > > turned out to > > > be different: When a package is single architecture only, e.g. arm64, it > > > should not be > > > extended as native if HOST_ARCH is amd64. > > > > > > So I added a check to BBCLASSEXTEND only if DPKG_ARCH != HOST_ARCH. > > Good catch. This case is currently not handled. But I would like to > > have it in a separate patch, as it tackles another issue. > > I thought it's possible to make the nesting of if-clauses easier that > way, but after some more thought and fiddling... unfortunately it's the > opposite. So yes, separate patch. > > > > > > > + # This special handling for DPKG_ARCH=all packages is left to the > > > > + # multiarch_virtclass_handler > > > > + if archDiffers: > > > > d.appendVar('BBCLASSEXTEND', ' native') > > > Now, seeing your patch to remove the not-"all" case also, I wonder if > > > that could be used > > > as the sole check before BBCLASSEXTEND. Like e.g. > > > archIsNative = d.getVar('DPKG_ARCH') == d.getVar('HOST_ARCH') > > > if isNative: > > > d.appendVar('BBCLASSEXTEND', ' native') > > > > else: > > > > extend_provides(pn, 'native', d) > > > but now I wonder what the extend_provides is needed for. If HOST_ARCH == > > > DISTRO_ARCH, what's > > > the use of providing a native variant? > > If we are building for the native arch, we can just add the -native > > variants to the PROVIDES, so dependencies of packages depending > > explicitly on <foo>-native can be resolved (without the need for extra > > recipe variants). > > I understand that its a simpler solution in that case. But, maybe I'm not > seeing the obvious, what would require a -native package then? If > target-arch > is host-arch, why would anything else than the base package be in any > dependency chain? Consider the case where you have a documentation generator in your dependency chain. This should always run in the native arch. In Debian, this build dependency would be annotated with <foo>:any. In ISAR, you need to annotate it with -native, to ensure that always the native version is built, no matter what is DISTRO_ARCH. > > > > > > But back to your Patch: Is it possible that -compat "all" packages also have > > > dependencies on other "all" packages, which in turn have dependencies on > > > architecture specific packages? > > Good question. In general, package dependencies are resolved for the > > native architecture (or the requested one). AFAIK this selection is > > propagated down the dependency tree. By that, A:i386 -> B:all -> C:x > > should always resolve C as either :all or i386 (except if there is a > > explicit cross-arch dependency specifier like :any, which we don't > > model in bitbake). Anyways, The A:i386 -> B:all -> C:i386 chain should > > already be resolved correctly with this patch. > The patch doesnt change that -compat is "short-circuited" when > archIsAll, doesnt it? > Or maybe I dont fully understand how the code works yet. > > > > Tested by's are welcome :) > > I'm working with it and keep an eye on it. Thanks! If there are no other comments, this patchset is ready to be merged. Felix -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/96bbe4d4fa98a89ca8ee07dd8100b1a936b394d5.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-09-10 11:00 ` Andreas Naumann @ 2025-09-15 17:49 ` Andreas Naumann 2025-09-16 7:16 ` 'MOESSBAUER, Felix' via isar-users 2025-10-07 12:31 ` Zhihang Wei 2025-10-24 13:03 ` 'MOESSBAUER, Felix' via isar-users 3 siblings, 1 reply; 21+ messages in thread From: Andreas Naumann @ 2025-09-15 17:49 UTC (permalink / raw) To: isar-users, MOESSBAUER, Felix Hi Felix, Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: > Arch=all packages might build depend on other arch=all packages, hence we > need to correctly model the dependency chain. Otherwise the transitive > dependencies are built for the distro arch instead of the native arch. > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > packages to the -native variant by adding a dependency. We further > replace the non-native do_deploy_dep task with a noop to preserve the > dependency chain. > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > index babdfbd4..c2bba21f 100644 > --- a/meta/classes/multiarch.bbclass > +++ b/meta/classes/multiarch.bbclass > @@ -29,7 +29,11 @@ python() { > d.appendVar('BBCLASSEXTEND', ' compat') > > # build native separately only when it differs from the target variant > - if not archIsAll and archDiffers: > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > + # have transitive dependencies which need to be built for -native. > + # This special handling for DPKG_ARCH=all packages is left to the > + # multiarch_virtclass_handler > + if archDiffers: > d.appendVar('BBCLASSEXTEND', ' native') > else: > extend_provides(pn, 'native', d) > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > d.setVar(var, ' '.join(multiarch_var)) > > pn = e.data.getVar('PN') > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > if pn.endswith('-compat'): > e.data.setVar('BPN', pn[:-len('-compat')]) > e.data.appendVar('OVERRIDES', ':class-compat') > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > e.data.appendVar('OVERRIDES', ':class-native') > fixup_pn_in_vars(e.data) > fixup_depends('-native', e.data) > + elif archIsAll and archDiffers: > + # Arch=all packages might build depend on other arch=all packages, > + # hence we need to correctly model the dependency chain. > + # We implement this by dispatching the non-native variant to the -native > + # variant by adding a dependency. We further replace the non-native > + # do_deploy_dep task with a noop to preserve the dependency chain. > + e.data.setVar('do_deploy_deb', '') > + bb.build.deltask('deploy_deb', e.data) > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') I have now done some more testing and unfortunately find that this causes failures when building the -native variant of some dpkg-raw packages we have. We use them for certain config files and they are intended for the target. I dont know why exactly they fail and I'm sure this could be fixed, but actually there is no need to build those packages in the native environment. So I'm somewhat surprised to see that and could image that this causes confusion for others that dont know about the "all" intricacies as well. Another observation that I see is that dpkg-prebuilt "any" packages, which are probably in the dependency chain of an "all" package, now are always also built as -native, even though they are needed for the target only. best, Andras > } > addhandler multiarch_virtclass_handler > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mail info@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/05c06d5d-5248-4d56-b470-a096411b2844%40emlix.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-09-15 17:49 ` Andreas Naumann @ 2025-09-16 7:16 ` 'MOESSBAUER, Felix' via isar-users 2025-09-19 7:15 ` Andreas Naumann 0 siblings, 1 reply; 21+ messages in thread From: 'MOESSBAUER, Felix' via isar-users @ 2025-09-16 7:16 UTC (permalink / raw) To: isar-users, anaumann On Mon, 2025-09-15 at 19:49 +0200, Andreas Naumann wrote: > Hi Felix, > > Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: > > Arch=all packages might build depend on other arch=all packages, hence we > > need to correctly model the dependency chain. Otherwise the transitive > > dependencies are built for the distro arch instead of the native arch. > > > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > > packages to the -native variant by adding a dependency. We further > > replace the non-native do_deploy_dep task with a noop to preserve the > > dependency chain. > > > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > > --- > > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > > index babdfbd4..c2bba21f 100644 > > --- a/meta/classes/multiarch.bbclass > > +++ b/meta/classes/multiarch.bbclass > > @@ -29,7 +29,11 @@ python() { > > d.appendVar('BBCLASSEXTEND', ' compat') > > > > # build native separately only when it differs from the target variant > > - if not archIsAll and archDiffers: > > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > > + # have transitive dependencies which need to be built for -native. > > + # This special handling for DPKG_ARCH=all packages is left to the > > + # multiarch_virtclass_handler > > + if archDiffers: > > d.appendVar('BBCLASSEXTEND', ' native') > > else: > > extend_provides(pn, 'native', d) > > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > > d.setVar(var, ' '.join(multiarch_var)) > > > > pn = e.data.getVar('PN') > > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > > if pn.endswith('-compat'): > > e.data.setVar('BPN', pn[:-len('-compat')]) > > e.data.appendVar('OVERRIDES', ':class-compat') > > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > > e.data.appendVar('OVERRIDES', ':class-native') > > fixup_pn_in_vars(e.data) > > fixup_depends('-native', e.data) > > + elif archIsAll and archDiffers: > > + # Arch=all packages might build depend on other arch=all packages, > > + # hence we need to correctly model the dependency chain. > > + # We implement this by dispatching the non-native variant to the -native > > + # variant by adding a dependency. We further replace the non-native > > + # do_deploy_dep task with a noop to preserve the dependency chain. > > + e.data.setVar('do_deploy_deb', '') > > + bb.build.deltask('deploy_deb', e.data) > > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') > > I have now done some more testing and unfortunately find that this > causes failures when building the -native variant of some dpkg-raw > packages we have. > Hi, do you have an example for such a package? Just to be clear: There is no "native" variant of a dpkg-raw package. The dpkg-raw packages are always build on the host architecture (in bitbake terms) and of Debian arch=all. > We use them for certain config files and they are > intended for the target. > Do these configs contain data that is not the same on all architectures? If so, DPKG_ARCH="all" is wrong. > I dont know why exactly they fail and I'm sure > this could be fixed, but actually there is no need to build those > packages in the native environment. > There is need to do so, as otherwise we emulate the build process which is super slow. > So I'm somewhat surprised to see > that and could image that this causes confusion for others that dont > know about the "all" intricacies as well. Probably, but in the end we are building a debian system and by that the users should be aware of the debian architecture specifiers. > > Another observation that I see is that dpkg-prebuilt "any" packages, > which are probably in the dependency chain of an "all" package, now are > always also built as -native, even though they are needed for the target > only. Same here, even if bitbake tells you that they are build as "-native", there is no difference in the output, as the binary is simply copied. Best regards, Felix > > best, > Andras > > > } > > addhandler multiarch_virtclass_handler > > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" > > -- > Andreas Naumann > > emlix GmbH > Headquarters: Berliner Str. 12, 37073 Goettingen, Germany > Phone +49 (0)551 30664-0, e-mail info@emlix.com > District Court of Goettingen, Registry Number HR B 3160 > Managing Directors: Heike Jordan, Dr. Uwe Kracke > VAT ID No. DE 205 198 055 > Office Berlin: Panoramastr. 1, 10178 Berlin, Germany > Office Bonn: Bachstr. 6, 53115 Bonn, Germany > http://www.emlix.com -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/9af00c922ab12ee5961d377babb3a87f49efac6c.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-09-16 7:16 ` 'MOESSBAUER, Felix' via isar-users @ 2025-09-19 7:15 ` Andreas Naumann 2025-10-06 14:05 ` 'MOESSBAUER, Felix' via isar-users 0 siblings, 1 reply; 21+ messages in thread From: Andreas Naumann @ 2025-09-19 7:15 UTC (permalink / raw) To: MOESSBAUER, Felix, isar-users, Jan Kiszka, Steiger, Christoph Hi Felix, all Am 16.09.25 um 09:16 schrieb MOESSBAUER, Felix: > On Mon, 2025-09-15 at 19:49 +0200, Andreas Naumann wrote: >> Hi Felix, >> >> Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: >>> Arch=all packages might build depend on other arch=all packages, hence we >>> need to correctly model the dependency chain. Otherwise the transitive >>> dependencies are built for the distro arch instead of the native arch. >>> >>> We implement this by dispatching the non-native variant of DPKG_ARCH=all >>> packages to the -native variant by adding a dependency. We further >>> replace the non-native do_deploy_dep task with a noop to preserve the >>> dependency chain. >>> >>> Co-developed-by: Adriaan Schmidt<adriaan.schmidt@siemens.com> >>> Signed-off-by: Felix Moessbauer<felix.moessbauer@siemens.com> >>> --- >>> meta/classes/multiarch.bbclass | 18 +++++++++++++++++- >>> 1 file changed, 17 insertions(+), 1 deletion(-) >>> >>> diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass >>> index babdfbd4..c2bba21f 100644 >>> --- a/meta/classes/multiarch.bbclass >>> +++ b/meta/classes/multiarch.bbclass >>> @@ -29,7 +29,11 @@ python() { >>> d.appendVar('BBCLASSEXTEND', ' compat') >>> >>> # build native separately only when it differs from the target variant >>> - if not archIsAll and archDiffers: >>> + # We must not short-circuit for DPKG_ARCH=all packages, as they might >>> + # have transitive dependencies which need to be built for -native. >>> + # This special handling for DPKG_ARCH=all packages is left to the >>> + # multiarch_virtclass_handler >>> + if archDiffers: >>> d.appendVar('BBCLASSEXTEND', ' native') >>> else: >>> extend_provides(pn, 'native', d) >>> @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { >>> d.setVar(var, ' '.join(multiarch_var)) >>> >>> pn = e.data.getVar('PN') >>> + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') >>> + archIsAll = d.getVar('DPKG_ARCH') == 'all' >>> if pn.endswith('-compat'): >>> e.data.setVar('BPN', pn[:-len('-compat')]) >>> e.data.appendVar('OVERRIDES', ':class-compat') >>> @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { >>> e.data.appendVar('OVERRIDES', ':class-native') >>> fixup_pn_in_vars(e.data) >>> fixup_depends('-native', e.data) >>> + elif archIsAll and archDiffers: >>> + # Arch=all packages might build depend on other arch=all packages, >>> + # hence we need to correctly model the dependency chain. >>> + # We implement this by dispatching the non-native variant to the -native >>> + # variant by adding a dependency. We further replace the non-native >>> + # do_deploy_dep task with a noop to preserve the dependency chain. >>> + e.data.setVar('do_deploy_deb', '') >>> + bb.build.deltask('deploy_deb', e.data) >>> + bb.build.addtask('deploy_deb', 'do_build', '', e.data) >>> + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') >> I have now done some more testing and unfortunately find that this >> causes failures when building the -native variant of some dpkg-raw >> packages we have. >> > Hi, do you have an example for such a package? Just to be clear: There > is no "native" variant of a dpkg-raw package. The dpkg-raw packages are > always build on the host architecture (in bitbake terms) and of Debian > arch=all. > >> We use them for certain config files and they are >> intended for the target. >> > Do these configs contain data that is not the same on all > architectures? If so, DPKG_ARCH="all" is wrong. After some debugging, it seems to be a combination of your patch and "2ca3a7e dpkg-source: Build source package only once". There is no problem building our dpkg-raw packages explicitly in the non-native form, e.g. in the host environment for the target arch. But through your patch, the package also runs as -native, which then triggers a problem with 2ca3a7e. ... I only now discover that the other part of your patchset "dpkg-raw: add files to source package" fixes this. Ok. However, because of this I went digging on why you introduced the patch. And I wonder why sbom-chroot doesn't just depend on python3-debsbom-native directly? (Which still breaks as long as 2ca3a7e sets the dependency on a function of the non-native package). >> I dont know why exactly they fail and I'm sure >> this could be fixed, but actually there is no need to build those >> packages in the native environment. >> > There is need to do so, as otherwise we emulate the build process which > is super slow. Yes you're right. I meant there is no need to build them *for* the native architecture. >> So I'm somewhat surprised to see >> that and could image that this causes confusion for others that dont >> know about the "all" intricacies as well. > Probably, but in the end we are building a debian system and by that > the users should be aware of the debian architecture specifiers. Yes sure, they were new to me, but no problem, they are not that complicated. But when I deliberately build -native packages (which I assume to be a valid use case according to the doc), and somehow the buildsystem goes back and forth running functions from the native and the base package, it's difficult to follow why that would be necessary. It doesnt make for ease of debugging when working on new packages either. For the use cases I have in sight right now it would be fine if a package build breaks because it cannot be done cross-architecture. I'd hope to being able to fix this by either improving the package or making sure to call it -native only. But if that's not what the expectation is for Debian Crossbuilding I'm eager to learn. best regards, Andreas >> Another observation that I see is that dpkg-prebuilt "any" packages, >> which are probably in the dependency chain of an "all" package, now are >> always also built as -native, even though they are needed for the target >> only. > Same here, even if bitbake tells you that they are build as "-native", > there is no difference in the output, as the binary is simply copied. > > Best regards, > Felix > >> best, >> Andras >> >>> } >>> addhandler multiarch_virtclass_handler >>> multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" >> -- >> Andreas Naumann >> >> emlix GmbH >> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany >> Phone +49 (0)551 30664-0, e-mailinfo@emlix.com >> District Court of Goettingen, Registry Number HR B 3160 >> Managing Directors: Heike Jordan, Dr. Uwe Kracke >> VAT ID No. DE 205 198 055 >> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany >> Office Bonn: Bachstr. 6, 53115 Bonn, Germany >> http://www.emlix.com -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mailinfo@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/45ecefef-bbc3-4506-b9b3-c945d0d77e65%40emlix.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-09-19 7:15 ` Andreas Naumann @ 2025-10-06 14:05 ` 'MOESSBAUER, Felix' via isar-users 2025-10-07 10:57 ` 'Andreas Naumann' via isar-users 0 siblings, 1 reply; 21+ messages in thread From: 'MOESSBAUER, Felix' via isar-users @ 2025-10-06 14:05 UTC (permalink / raw) To: Steiger, Christoph, isar-users, Kiszka, Jan, anaumann On Fri, 2025-09-19 at 09:15 +0200, Andreas Naumann wrote: > Hi Felix, all > > Am 16.09.25 um 09:16 schrieb MOESSBAUER, Felix: > > On Mon, 2025-09-15 at 19:49 +0200, Andreas Naumann wrote: > > > Hi Felix, > > > > > > Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: > > > > Arch=all packages might build depend on other arch=all packages, hence we > > > > need to correctly model the dependency chain. Otherwise the transitive > > > > dependencies are built for the distro arch instead of the native arch. > > > > > > > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > > > > packages to the -native variant by adding a dependency. We further > > > > replace the non-native do_deploy_dep task with a noop to preserve the > > > > dependency chain. > > > > > > > > Co-developed-by: Adriaan Schmidt<adriaan.schmidt@siemens.com> > > > > Signed-off-by: Felix Moessbauer<felix.moessbauer@siemens.com> > > > > --- > > > > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > > > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > > > > index babdfbd4..c2bba21f 100644 > > > > --- a/meta/classes/multiarch.bbclass > > > > +++ b/meta/classes/multiarch.bbclass > > > > @@ -29,7 +29,11 @@ python() { > > > > d.appendVar('BBCLASSEXTEND', ' compat') > > > > > > > > # build native separately only when it differs from the target variant > > > > - if not archIsAll and archDiffers: > > > > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > > > > + # have transitive dependencies which need to be built for -native. > > > > + # This special handling for DPKG_ARCH=all packages is left to the > > > > + # multiarch_virtclass_handler > > > > + if archDiffers: > > > > d.appendVar('BBCLASSEXTEND', ' native') > > > > else: > > > > extend_provides(pn, 'native', d) > > > > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > > > > d.setVar(var, ' '.join(multiarch_var)) > > > > > > > > pn = e.data.getVar('PN') > > > > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > > > > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > > > > if pn.endswith('-compat'): > > > > e.data.setVar('BPN', pn[:-len('-compat')]) > > > > e.data.appendVar('OVERRIDES', ':class-compat') > > > > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > > > > e.data.appendVar('OVERRIDES', ':class-native') > > > > fixup_pn_in_vars(e.data) > > > > fixup_depends('-native', e.data) > > > > + elif archIsAll and archDiffers: > > > > + # Arch=all packages might build depend on other arch=all packages, > > > > + # hence we need to correctly model the dependency chain. > > > > + # We implement this by dispatching the non-native variant to the -native > > > > + # variant by adding a dependency. We further replace the non-native > > > > + # do_deploy_dep task with a noop to preserve the dependency chain. > > > > + e.data.setVar('do_deploy_deb', '') > > > > + bb.build.deltask('deploy_deb', e.data) > > > > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > > > > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') > > > I have now done some more testing and unfortunately find that this > > > causes failures when building the -native variant of some dpkg-raw > > > packages we have. > > > > > Hi, do you have an example for such a package? Just to be clear: There > > is no "native" variant of a dpkg-raw package. The dpkg-raw packages are > > always build on the host architecture (in bitbake terms) and of Debian > > arch=all. > > > > > We use them for certain config files and they are > > > intended for the target. > > > > > Do these configs contain data that is not the same on all > > architectures? If so, DPKG_ARCH="all" is wrong. > > After some debugging, it seems to be a combination of your patch and > "2ca3a7e dpkg-source: Build source package only once". There is no > problem building our dpkg-raw packages explicitly in the non-native > form, e.g. in the host environment for the target arch. > But through your patch, the package also runs as -native, which then > triggers a problem with 2ca3a7e. > > ... I only now discover that the other part of your patchset "dpkg-raw: > add files to source package" fixes this. Ok. Ok, so in general this series is fine. > > However, because of this I went digging on why you introduced the patch. > And I wonder why sbom-chroot doesn't just depend on > python3-debsbom-native directly? (Which still breaks as long as 2ca3a7e > sets the dependency on a function of the non-native package). This is a more general thing: The chroot is architecture specific. By that, the architecture of the packages (DEPENDS) is selected based on that without the need for having -native. The internal logic then takes care of building the packages in the correct variant (e.g. -compat, -native or without any suffix, which refers to the target arch). The 2ca3a7e ("dpkg-source: Build source package only once") just ensures that we only create a single source package, no matter for which (and how many) architectures we build it. > > > > > I dont know why exactly they fail and I'm sure > > > this could be fixed, but actually there is no need to build those > > > packages in the native environment. > > > > > There is need to do so, as otherwise we emulate the build process which > > is super slow. > Yes you're right. I meant there is no need to build them *for* the > native architecture. > > > So I'm somewhat surprised to see > > > that and could image that this causes confusion for others that dont > > > know about the "all" intricacies as well. > > Probably, but in the end we are building a debian system and by that > > the users should be aware of the debian architecture specifiers. > Yes sure, they were new to me, but no problem, they are not that > complicated. > > But when I deliberately build -native packages (which I assume to be a > valid use case according to the doc), and somehow the buildsystem goes > back and forth running functions from the native and the base package, > it's difficult to follow why that would be necessary. It doesnt make for > ease of debugging when working on new packages either. > > For the use cases I have in sight right now it would be fine if a > package build breaks because it cannot be done cross-architecture. I'd > hope to being able to fix this by either improving the package or making > sure to call it -native only. But if that's not what the expectation is > for Debian Crossbuilding I'm eager to learn. If there are no further issues with this patchset, I would like to move forward and see it merged. It is a pre-condition for the SBOM patches for non target!=host arch. Best regards, Felix > > best regards, > Andreas > > > > Another observation that I see is that dpkg-prebuilt "any" packages, > > > which are probably in the dependency chain of an "all" package, now are > > > always also built as -native, even though they are needed for the target > > > only. > > Same here, even if bitbake tells you that they are build as "-native", > > there is no difference in the output, as the binary is simply copied. > > > > Best regards, > > Felix > > > > > best, > > > Andras > > > > > > > } > > > > addhandler multiarch_virtclass_handler > > > > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" > > > -- > > > Andreas Naumann > > > > > > emlix GmbH > > > Headquarters: Berliner Str. 12, 37073 Goettingen, Germany > > > Phone +49 (0)551 30664-0, e-mailinfo@emlix.com > > > District Court of Goettingen, Registry Number HR B 3160 > > > Managing Directors: Heike Jordan, Dr. Uwe Kracke > > > VAT ID No. DE 205 198 055 > > > Office Berlin: Panoramastr. 1, 10178 Berlin, Germany > > > Office Bonn: Bachstr. 6, 53115 Bonn, Germany > > > http://www.emlix.com > > -- > Andreas Naumann > > emlix GmbH > Headquarters: Berliner Str. 12, 37073 Goettingen, Germany > Phone +49 (0)551 30664-0, e-mailinfo@emlix.com > District Court of Goettingen, Registry Number HR B 3160 > Managing Directors: Heike Jordan, Dr. Uwe Kracke > VAT ID No. DE 205 198 055 > Office Berlin: Panoramastr. 1, 10178 Berlin, Germany > Office Bonn: Bachstr. 6, 53115 Bonn, Germany > http://www.emlix.com -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/56507dd43d6b0f0e309563ebbe2d48327353cdc5.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-10-06 14:05 ` 'MOESSBAUER, Felix' via isar-users @ 2025-10-07 10:57 ` 'Andreas Naumann' via isar-users 0 siblings, 0 replies; 21+ messages in thread From: 'Andreas Naumann' via isar-users @ 2025-10-07 10:57 UTC (permalink / raw) To: MOESSBAUER, Felix, Steiger, Christoph, isar-users, Kiszka, Jan Hi Felix, all Am 06.10.25 um 16:05 schrieb MOESSBAUER, Felix: > On Fri, 2025-09-19 at 09:15 +0200, Andreas Naumann wrote: >> Hi Felix, all >> >> Am 16.09.25 um 09:16 schrieb MOESSBAUER, Felix: >>> On Mon, 2025-09-15 at 19:49 +0200, Andreas Naumann wrote: >>>> Hi Felix, >>>> >>>> Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: >>>>> Arch=all packages might build depend on other arch=all packages, hence we >>>>> need to correctly model the dependency chain. Otherwise the transitive >>>>> dependencies are built for the distro arch instead of the native arch. >>>>> >>>>> We implement this by dispatching the non-native variant of DPKG_ARCH=all >>>>> packages to the -native variant by adding a dependency. We further >>>>> replace the non-native do_deploy_dep task with a noop to preserve the >>>>> dependency chain. >>>>> >>>>> Co-developed-by: Adriaan Schmidt<adriaan.schmidt@siemens.com> >>>>> Signed-off-by: Felix Moessbauer<felix.moessbauer@siemens.com> >>>>> --- >>>>> meta/classes/multiarch.bbclass | 18 +++++++++++++++++- >>>>> 1 file changed, 17 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass >>>>> index babdfbd4..c2bba21f 100644 >>>>> --- a/meta/classes/multiarch.bbclass >>>>> +++ b/meta/classes/multiarch.bbclass >>>>> @@ -29,7 +29,11 @@ python() { >>>>> d.appendVar('BBCLASSEXTEND', ' compat') >>>>> >>>>> # build native separately only when it differs from the target variant >>>>> - if not archIsAll and archDiffers: >>>>> + # We must not short-circuit for DPKG_ARCH=all packages, as they might >>>>> + # have transitive dependencies which need to be built for -native. >>>>> + # This special handling for DPKG_ARCH=all packages is left to the >>>>> + # multiarch_virtclass_handler >>>>> + if archDiffers: >>>>> d.appendVar('BBCLASSEXTEND', ' native') >>>>> else: >>>>> extend_provides(pn, 'native', d) >>>>> @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { >>>>> d.setVar(var, ' '.join(multiarch_var)) >>>>> >>>>> pn = e.data.getVar('PN') >>>>> + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') >>>>> + archIsAll = d.getVar('DPKG_ARCH') == 'all' >>>>> if pn.endswith('-compat'): >>>>> e.data.setVar('BPN', pn[:-len('-compat')]) >>>>> e.data.appendVar('OVERRIDES', ':class-compat') >>>>> @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { >>>>> e.data.appendVar('OVERRIDES', ':class-native') >>>>> fixup_pn_in_vars(e.data) >>>>> fixup_depends('-native', e.data) >>>>> + elif archIsAll and archDiffers: >>>>> + # Arch=all packages might build depend on other arch=all packages, >>>>> + # hence we need to correctly model the dependency chain. >>>>> + # We implement this by dispatching the non-native variant to the -native >>>>> + # variant by adding a dependency. We further replace the non-native >>>>> + # do_deploy_dep task with a noop to preserve the dependency chain. >>>>> + e.data.setVar('do_deploy_deb', '') >>>>> + bb.build.deltask('deploy_deb', e.data) >>>>> + bb.build.addtask('deploy_deb', 'do_build', '', e.data) >>>>> + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') >>>> I have now done some more testing and unfortunately find that this >>>> causes failures when building the -native variant of some dpkg-raw >>>> packages we have. >>>> >>> Hi, do you have an example for such a package? Just to be clear: There >>> is no "native" variant of a dpkg-raw package. The dpkg-raw packages are >>> always build on the host architecture (in bitbake terms) and of Debian >>> arch=all. >>> >>>> We use them for certain config files and they are >>>> intended for the target. >>>> >>> Do these configs contain data that is not the same on all >>> architectures? If so, DPKG_ARCH="all" is wrong. >> After some debugging, it seems to be a combination of your patch and >> "2ca3a7e dpkg-source: Build source package only once". There is no >> problem building our dpkg-raw packages explicitly in the non-native >> form, e.g. in the host environment for the target arch. >> But through your patch, the package also runs as -native, which then >> triggers a problem with 2ca3a7e. >> >> ... I only now discover that the other part of your patchset "dpkg-raw: >> add files to source package" fixes this. Ok. > Ok, so in general this series is fine. ACK, to get the sbom series going. However, I have proposed and would like to invite you to comment on my [RFC 0/5] Improving multiarch support for arch-incompatible packages. I think it provides a straightforward, generic solution that works for "all" and non-all packages alike. >> However, because of this I went digging on why you introduced the patch. >> And I wonder why sbom-chroot doesn't just depend on >> python3-debsbom-native directly? (Which still breaks as long as 2ca3a7e >> sets the dependency on a function of the non-native package). > This is a more general thing: The chroot is architecture specific. By > that, the architecture of the packages (DEPENDS) is selected based on > that without the need for having -native. The internal logic then takes > care of building the packages in the correct variant (e.g. -compat, > -native or without any suffix, which refers to the target arch). I'm not sure I understand what you mean. If python3-debsbom was an "any" package, how would the logic guess that it's needed for the host? best regards, Andreas > > The 2ca3a7e ("dpkg-source: Build source package only once") just > ensures that we only create a single source package, no matter for > which (and how many) architectures we build it. > >> >>>> I dont know why exactly they fail and I'm sure >>>> this could be fixed, but actually there is no need to build those >>>> packages in the native environment. >>>> >>> There is need to do so, as otherwise we emulate the build process which >>> is super slow. >> Yes you're right. I meant there is no need to build them *for* the >> native architecture. >>>> So I'm somewhat surprised to see >>>> that and could image that this causes confusion for others that dont >>>> know about the "all" intricacies as well. >>> Probably, but in the end we are building a debian system and by that >>> the users should be aware of the debian architecture specifiers. >> Yes sure, they were new to me, but no problem, they are not that >> complicated. >> >> But when I deliberately build -native packages (which I assume to be a >> valid use case according to the doc), and somehow the buildsystem goes >> back and forth running functions from the native and the base package, >> it's difficult to follow why that would be necessary. It doesnt make for >> ease of debugging when working on new packages either. >> >> For the use cases I have in sight right now it would be fine if a >> package build breaks because it cannot be done cross-architecture. I'd >> hope to being able to fix this by either improving the package or making >> sure to call it -native only. But if that's not what the expectation is >> for Debian Crossbuilding I'm eager to learn. > If there are no further issues with this patchset, I would like to move > forward and see it merged. It is a pre-condition for the SBOM patches > for non target!=host arch. > > Best regards, > Felix > >> best regards, >> Andreas >> >>>> Another observation that I see is that dpkg-prebuilt "any" packages, >>>> which are probably in the dependency chain of an "all" package, now are >>>> always also built as -native, even though they are needed for the target >>>> only. >>> Same here, even if bitbake tells you that they are build as "-native", >>> there is no difference in the output, as the binary is simply copied. >>> >>> Best regards, >>> Felix >>> >>>> best, >>>> Andras >>>> >>>>> } >>>>> addhandler multiarch_virtclass_handler >>>>> multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" >>>> -- >>>> Andreas Naumann >>>> >>>> emlix GmbH >>>> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany >>>> Phone +49 (0)551 30664-0, e-mailinfo@emlix.com >>>> District Court of Goettingen, Registry Number HR B 3160 >>>> Managing Directors: Heike Jordan, Dr. Uwe Kracke >>>> VAT ID No. DE 205 198 055 >>>> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany >>>> Office Bonn: Bachstr. 6, 53115 Bonn, Germany >>>> http://www.emlix.com >> -- >> Andreas Naumann >> >> emlix GmbH >> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany >> Phone +49 (0)551 30664-0, e-mailinfo@emlix.com >> District Court of Goettingen, Registry Number HR B 3160 >> Managing Directors: Heike Jordan, Dr. Uwe Kracke >> VAT ID No. DE 205 198 055 >> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany >> Office Bonn: Bachstr. 6, 53115 Bonn, Germany >> http://www.emlix.com -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mail info@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/6c272cfc-ca61-4ec9-8913-7497cf43001e%40emlix.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-09-10 11:00 ` Andreas Naumann 2025-09-15 17:49 ` Andreas Naumann @ 2025-10-07 12:31 ` Zhihang Wei 2025-10-20 12:07 ` 'cedric.hombourger@siemens.com' via isar-users 2025-10-24 13:03 ` 'MOESSBAUER, Felix' via isar-users 3 siblings, 1 reply; 21+ messages in thread From: Zhihang Wei @ 2025-10-07 12:31 UTC (permalink / raw) To: Felix Moessbauer, isar-users Cc: jan.kiszka, cedric.hombourger, adriaan.schmidt, christoph.steiger p2 was applied to next, thanks. On 8/20/25 14:41, 'Felix Moessbauer' via isar-users wrote: > Arch=all packages might build depend on other arch=all packages, hence we > need to correctly model the dependency chain. Otherwise the transitive > dependencies are built for the distro arch instead of the native arch. > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > packages to the -native variant by adding a dependency. We further > replace the non-native do_deploy_dep task with a noop to preserve the > dependency chain. > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > index babdfbd4..c2bba21f 100644 > --- a/meta/classes/multiarch.bbclass > +++ b/meta/classes/multiarch.bbclass > @@ -29,7 +29,11 @@ python() { > d.appendVar('BBCLASSEXTEND', ' compat') > > # build native separately only when it differs from the target variant > - if not archIsAll and archDiffers: > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > + # have transitive dependencies which need to be built for -native. > + # This special handling for DPKG_ARCH=all packages is left to the > + # multiarch_virtclass_handler > + if archDiffers: > d.appendVar('BBCLASSEXTEND', ' native') > else: > extend_provides(pn, 'native', d) > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > d.setVar(var, ' '.join(multiarch_var)) > > pn = e.data.getVar('PN') > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > if pn.endswith('-compat'): > e.data.setVar('BPN', pn[:-len('-compat')]) > e.data.appendVar('OVERRIDES', ':class-compat') > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > e.data.appendVar('OVERRIDES', ':class-native') > fixup_pn_in_vars(e.data) > fixup_depends('-native', e.data) > + elif archIsAll and archDiffers: > + # Arch=all packages might build depend on other arch=all packages, > + # hence we need to correctly model the dependency chain. > + # We implement this by dispatching the non-native variant to the -native > + # variant by adding a dependency. We further replace the non-native > + # do_deploy_dep task with a noop to preserve the dependency chain. > + e.data.setVar('do_deploy_deb', '') > + bb.build.deltask('deploy_deb', e.data) > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') > } > addhandler multiarch_virtclass_handler > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/9edc443a-88d8-4cae-88b3-d2963b40c1f6%40ilbers.de. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-10-07 12:31 ` Zhihang Wei @ 2025-10-20 12:07 ` 'cedric.hombourger@siemens.com' via isar-users 2025-10-20 12:23 ` 'Jan Kiszka' via isar-users 0 siblings, 1 reply; 21+ messages in thread From: 'cedric.hombourger@siemens.com' via isar-users @ 2025-10-20 12:07 UTC (permalink / raw) To: isar-users, wzh, MOESSBAUER, Felix Cc: Steiger, Christoph, Schmidt, Adriaan, Kiszka, Jan On Tue, 2025-10-07 at 14:31 +0200, Zhihang Wei wrote: > p2 was applied to next, thanks. this appears to cause a regression with dpkg-customization packages. Not yet sure why this wasn't caught by dpkg-customization test cases that Chris had added when dpkg-customization was introduced. ERROR: Multiple .bb files are due to be built which each provide hostname-customization: mc:iot2050-bookworm:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb A list of tasks depending on these providers is shown and may help explain where the dependency comes from. mc:iot2050-bookworm:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb has unique dependees: mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb:do_dpkg_build mc:iot2050-bookworm:/work/build/../../repo/meta/recipes- core/images/development-image.bb:do_image_tools mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb:do_fetch_common_source mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb has unique dependees: mc:iot2050-bookworm:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb:do_deploy_deb It could be that one recipe provides something the other doesn't and should. The following provider and runtime provider differences may be helpful. mc:iot2050-bookworm:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb has unique provides: bookworm-hostname-customization-iot2050 mc:iot2050-bookworm:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb has unique rprovides: hostname-customization bookworm-hostname-customization-iot2050 mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb has unique provides: bookworm-hostname-customization-iot2050-native mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- support/customizations/hostname-customization.bb has unique rprovides: bookworm-hostname-customization-iot2050-native > > On 8/20/25 14:41, 'Felix Moessbauer' via isar-users wrote: > > Arch=all packages might build depend on other arch=all packages, > > hence we > > need to correctly model the dependency chain. Otherwise the > > transitive > > dependencies are built for the distro arch instead of the native > > arch. > > > > We implement this by dispatching the non-native variant of > > DPKG_ARCH=all > > packages to the -native variant by adding a dependency. We further > > replace the non-native do_deploy_dep task with a noop to preserve > > the > > dependency chain. > > > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > > --- > > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > diff --git a/meta/classes/multiarch.bbclass > > b/meta/classes/multiarch.bbclass > > index babdfbd4..c2bba21f 100644 > > --- a/meta/classes/multiarch.bbclass > > +++ b/meta/classes/multiarch.bbclass > > @@ -29,7 +29,11 @@ python() { > > d.appendVar('BBCLASSEXTEND', ' compat') > > > > # build native separately only when it differs from the > > target variant > > - if not archIsAll and archDiffers: > > + # We must not short-circuit for DPKG_ARCH=all packages, as > > they might > > + # have transitive dependencies which need to be built for - > > native. > > + # This special handling for DPKG_ARCH=all packages is left to > > the > > + # multiarch_virtclass_handler > > + if archDiffers: > > d.appendVar('BBCLASSEXTEND', ' native') > > else: > > extend_provides(pn, 'native', d) > > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > > d.setVar(var, ' '.join(multiarch_var)) > > > > pn = e.data.getVar('PN') > > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > > if pn.endswith('-compat'): > > e.data.setVar('BPN', pn[:-len('-compat')]) > > e.data.appendVar('OVERRIDES', ':class-compat') > > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > > e.data.appendVar('OVERRIDES', ':class-native') > > fixup_pn_in_vars(e.data) > > fixup_depends('-native', e.data) > > + elif archIsAll and archDiffers: > > + # Arch=all packages might build depend on other arch=all > > packages, > > + # hence we need to correctly model the dependency chain. > > + # We implement this by dispatching the non-native variant > > to the -native > > + # variant by adding a dependency. We further replace the > > non-native > > + # do_deploy_dep task with a noop to preserve the > > dependency chain. > > + e.data.setVar('do_deploy_deb', '') > > + bb.build.deltask('deploy_deb', e.data) > > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}- > > native:do_deploy_deb') > > } > > addhandler multiarch_virtclass_handler > > multiarch_virtclass_handler[eventmask] = > > "bb.event.RecipePreFinalise" > -- Cedric Hombourger Siemens AG www.siemens.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/5fb19237cc4f329c157a9ac8a1b07d3bd3813734.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-10-20 12:07 ` 'cedric.hombourger@siemens.com' via isar-users @ 2025-10-20 12:23 ` 'Jan Kiszka' via isar-users 2025-10-20 12:25 ` 'cedric.hombourger@siemens.com' via isar-users 0 siblings, 1 reply; 21+ messages in thread From: 'Jan Kiszka' via isar-users @ 2025-10-20 12:23 UTC (permalink / raw) To: Hombourger, Cedric (FT FDS CES LX), isar-users, wzh, Moessbauer, Felix (FT RPD CED OES-DE) Cc: Steiger, Christoph (FT RPD CED OES-DE), Schmidt, Adriaan (FT RPD CED EDC-DE) On 20.10.25 14:07, Hombourger, Cedric (FT FDS CES LX) wrote: > On Tue, 2025-10-07 at 14:31 +0200, Zhihang Wei wrote: >> p2 was applied to next, thanks. > > this appears to cause a regression with dpkg-customization packages. > Not yet sure why this wasn't caught by dpkg-customization test cases > that Chris had added when dpkg-customization was introduced. > > ERROR: Multiple .bb files are due to be built which each provide > hostname-customization: > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb > mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb > A list of tasks depending on these providers is shown and may help > explain where the dependency comes from. > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb has unique dependees: > mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb:do_dpkg_build > mc:iot2050-bookworm:/work/build/../../repo/meta/recipes- > core/images/development-image.bb:do_image_tools > mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb:do_fetch_common_source > mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb has unique dependees: > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb:do_deploy_deb > It could be that one recipe provides something the other doesn't and > should. The following provider and runtime provider differences may be > helpful. > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb has unique provides: > bookworm-hostname-customization-iot2050 > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb has unique rprovides: > hostname-customization > bookworm-hostname-customization-iot2050 > mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb has unique provides: > bookworm-hostname-customization-iot2050-native > mc:iot2050-bookworm:virtual:native:/work/build/../isar/meta/recipes- > support/customizations/hostname-customization.bb has unique rprovides: > bookworm-hostname-customization-iot2050-native > Will be fixed once https://patchwork.isar-build.org/project/isar/patch/20251017085344.2647058-1-felix.moessbauer@siemens.com/ is merged - hopefully rather sooner than later. Jan -- Siemens AG, Foundational Technologies Linux Expert Center -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/d1a2da82-2af3-42df-bf79-11b4085a3276%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-10-20 12:23 ` 'Jan Kiszka' via isar-users @ 2025-10-20 12:25 ` 'cedric.hombourger@siemens.com' via isar-users 0 siblings, 0 replies; 21+ messages in thread From: 'cedric.hombourger@siemens.com' via isar-users @ 2025-10-20 12:25 UTC (permalink / raw) To: isar-users, Kiszka, Jan, wzh, MOESSBAUER, Felix Cc: Steiger, Christoph, Schmidt, Adriaan On Mon, 2025-10-20 at 14:23 +0200, Jan Kiszka wrote: > On 20.10.25 14:07, Hombourger, Cedric (FT FDS CES LX) wrote: > > On Tue, 2025-10-07 at 14:31 +0200, Zhihang Wei wrote: > > > p2 was applied to next, thanks. > > > > this appears to cause a regression with dpkg-customization > > packages. > > Not yet sure why this wasn't caught by dpkg-customization test > > cases > > that Chris had added when dpkg-customization was introduced. > > > > ERROR: Multiple .bb files are due to be built which each provide > > hostname-customization: > > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb > > mc:iot2050- > > bookworm:virtual:native:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb > > A list of tasks depending on these providers is shown and may help > > explain where the dependency comes from. > > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb has unique > > dependees: > > mc:iot2050- > > bookworm:virtual:native:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb:do_dpkg_build > > mc:iot2050-bookworm:/work/build/../../repo/meta/recipes- > > core/images/development-image.bb:do_image_tools > > mc:iot2050- > > bookworm:virtual:native:/work/build/../isar/meta/recipes- > > support/customizations/hostname- > > customization.bb:do_fetch_common_source > > mc:iot2050- > > bookworm:virtual:native:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb has unique > > dependees: > > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb:do_deploy_deb > > It could be that one recipe provides something the other doesn't > > and > > should. The following provider and runtime provider differences may > > be > > helpful. > > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb has unique > > provides: > > bookworm-hostname-customization-iot2050 > > mc:iot2050-bookworm:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb has unique > > rprovides: > > hostname-customization > > bookworm-hostname-customization-iot2050 > > mc:iot2050- > > bookworm:virtual:native:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb has unique > > provides: > > bookworm-hostname-customization-iot2050-native > > mc:iot2050- > > bookworm:virtual:native:/work/build/../isar/meta/recipes- > > support/customizations/hostname-customization.bb has unique > > rprovides: > > bookworm-hostname-customization-iot2050-native > > > > Will be fixed once > https://patchwork.isar-build.org/project/isar/patch/20251017085344.2647058-1-felix.moessbauer@siemens.com/ > is merged - hopefully rather sooner than later. Ah thanks Jan! I am reworking the customization patch without that patch hoping to catch the same failure. Will then apply the suggested patch and hopefully confirm the fix. I would then post the patch for the testsuite. > > Jan > -- Cedric Hombourger Siemens AG http://www.siemens.com/ -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/304522592621eaf678f47b711604c65eccfe29da.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users ` (2 preceding siblings ...) 2025-10-07 12:31 ` Zhihang Wei @ 2025-10-24 13:03 ` 'MOESSBAUER, Felix' via isar-users 3 siblings, 0 replies; 21+ messages in thread From: 'MOESSBAUER, Felix' via isar-users @ 2025-10-24 13:03 UTC (permalink / raw) To: isar-users Cc: Steiger, Christoph, Schmidt, Adriaan, Kiszka, Jan, cedric.hombourger On Wed, 2025-08-20 at 14:41 +0200, Felix Moessbauer wrote: > Arch=all packages might build depend on other arch=all packages, hence we > need to correctly model the dependency chain. Otherwise the transitive > dependencies are built for the distro arch instead of the native arch. > > We implement this by dispatching the non-native variant of DPKG_ARCH=all > packages to the -native variant by adding a dependency. We further > replace the non-native do_deploy_dep task with a noop to preserve the > dependency chain. This patch unfortunately creates some downstream fallout, but a simple revert is also not the solution. The patch solves the issue where a DPKG_ARCH=all package depends on other DPKG_ARCH=all packages, however it does break the case where a DPKG_ARCH=all package depends on DPKG_ARCH=any packages. A workaround is to use DPKG_ARCH="any" for the main package, but a proper solution is needed. I need to think about a proper solution, but likely that will be by explicitly modeling dependencies to -all packages by annotating them with -all (instead of -native). Best regards, Felix > > Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass > index babdfbd4..c2bba21f 100644 > --- a/meta/classes/multiarch.bbclass > +++ b/meta/classes/multiarch.bbclass > @@ -29,7 +29,11 @@ python() { > d.appendVar('BBCLASSEXTEND', ' compat') > > # build native separately only when it differs from the target variant > - if not archIsAll and archDiffers: > + # We must not short-circuit for DPKG_ARCH=all packages, as they might > + # have transitive dependencies which need to be built for -native. > + # This special handling for DPKG_ARCH=all packages is left to the > + # multiarch_virtclass_handler > + if archDiffers: > d.appendVar('BBCLASSEXTEND', ' native') > else: > extend_provides(pn, 'native', d) > @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { > d.setVar(var, ' '.join(multiarch_var)) > > pn = e.data.getVar('PN') > + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') > + archIsAll = d.getVar('DPKG_ARCH') == 'all' > if pn.endswith('-compat'): > e.data.setVar('BPN', pn[:-len('-compat')]) > e.data.appendVar('OVERRIDES', ':class-compat') > @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { > e.data.appendVar('OVERRIDES', ':class-native') > fixup_pn_in_vars(e.data) > fixup_depends('-native', e.data) > + elif archIsAll and archDiffers: > + # Arch=all packages might build depend on other arch=all packages, > + # hence we need to correctly model the dependency chain. > + # We implement this by dispatching the non-native variant to the -native > + # variant by adding a dependency. We further replace the non-native > + # do_deploy_dep task with a noop to preserve the dependency chain. > + e.data.setVar('do_deploy_deb', '') > + bb.build.deltask('deploy_deb', e.data) > + bb.build.addtask('deploy_deb', 'do_build', '', e.data) > + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') > } > addhandler multiarch_virtclass_handler > multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" > -- > 2.50.1 -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/f401397eb2ed67464ce2ce01a1ac0ac6c353b200.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 2025-08-20 12:41 [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-08-20 12:41 ` [PATCH 1/2] dpkg-raw: add files to source package 'Felix Moessbauer' via isar-users 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users @ 2025-10-24 5:34 ` 'cedric.hombourger@siemens.com' via isar-users 2025-10-24 7:36 ` 'MOESSBAUER, Felix' via isar-users 2025-10-27 13:35 ` 'Andreas Naumann' via isar-users 2 siblings, 2 replies; 21+ messages in thread From: 'cedric.hombourger@siemens.com' via isar-users @ 2025-10-24 5:34 UTC (permalink / raw) To: isar-users, MOESSBAUER, Felix Cc: Steiger, Christoph, Schmidt, Adriaan, Kiszka, Jan On Wed, 2025-08-20 at 14:41 +0200, Felix Moessbauer wrote: > This series fixes an issue with DPKG_ARCH=all packages that have > dependencies to other DPKG_ARCH=all packages. Hereby, the transitive > packages were built for the wrong architecture. The issue became > apparent when testing the SBOM series by Christoph, which builds > packages to be used during the ISAR build (not for the final target). > > While fixing this, another issue surfaced where files are missing in > dpkg-raw packages. In my testing with downstream layers, this change caused some packages to be built for architectures they were not targeted for. For instance, I am now seeing the k3-rti-wdt firmware package [1] being built for both amd64 and arm64. I also had a dpkg-prebuilt recipe for a package supported on arm64 and amd64: it is now tried for armhf/i386. Then there was this issue with dpkg-customization packages now leading to a "multiple .bb files are due to be built" error. My work-around was to add DPKG_ARCH=any to the dpkg-customization class since customization packages are anyhow decorated with both DISTRO and MACHINE (it is therefore ok and possibly better to have them explicitly built for the arch of the MACHINE). Note: Jan had suggested that I also pull [2] but that did not help I evidently need to provide you more details. I have had a short discussion with Felix and we suspect that multiconfig builds may be causing these issues. Anyhow, I hope that I can create minimal reproducers and get them added to the CI test suite. > > Best regards, > Felix > > Felix Moessbauer (2): > dpkg-raw: add files to source package > handle DPKG_ARCH=all case for transitive deps > > meta/classes/dpkg-raw.bbclass | 4 ++-- > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > 2 files changed, 19 insertions(+), 3 deletions(-) > -- Cedric Hombourger Siemens AG www.siemens.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/7b572ffbbd160f51fd15a737ff81c5278422e9ca.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 2025-10-24 5:34 ` [PATCH 0/2] " 'cedric.hombourger@siemens.com' via isar-users @ 2025-10-24 7:36 ` 'MOESSBAUER, Felix' via isar-users 2025-10-27 13:35 ` 'Andreas Naumann' via isar-users 1 sibling, 0 replies; 21+ messages in thread From: 'MOESSBAUER, Felix' via isar-users @ 2025-10-24 7:36 UTC (permalink / raw) To: isar-users, cedric.hombourger Cc: Steiger, Christoph, Schmidt, Adriaan, Kiszka, Jan On Fri, 2025-10-24 at 05:34 +0000, Hombourger, Cedric (FT FDS CES LX) wrote: > On Wed, 2025-08-20 at 14:41 +0200, Felix Moessbauer wrote: > > This series fixes an issue with DPKG_ARCH=all packages that have > > dependencies to other DPKG_ARCH=all packages. Hereby, the transitive > > packages were built for the wrong architecture. The issue became > > apparent when testing the SBOM series by Christoph, which builds > > packages to be used during the ISAR build (not for the final target). > > > > While fixing this, another issue surfaced where files are missing in > > dpkg-raw packages. > > In my testing with downstream layers, this change caused some packages > to be built for architectures they were not targeted for. For instance, > I am now seeing the k3-rti-wdt firmware package [1] being built for > both amd64 and arm64. I also had a dpkg-prebuilt recipe for a package > supported on arm64 and amd64: it is now tried for armhf/i386. Where is [1]? The question hereby is if the dependency chains are actually modeled correctly or if the patch is not behaving as expected. Without in-tree reproducers that's really hard to check. > > Then there was this issue with dpkg-customization packages now leading > to a "multiple .bb files are due to be built" error. My work-around was > to add DPKG_ARCH=any to the dpkg-customization class since > customization packages are anyhow decorated with both DISTRO and > MACHINE (it is therefore ok and possibly better to have them explicitly > built for the arch of the MACHINE). Apart from if we have a bug there or not, these customization packages should be arch specific (so DPKG_ARCH=any is the right choice). > > Note: Jan had suggested that I also pull [2] but that did not help > > I evidently need to provide you more details. I have had a short > discussion with Felix and we suspect that multiconfig builds may be > causing these issues. Likely. We don't use mc in isar-cip-core and the isar layer itself seems to be too simple to surface the remaining bugs. > > Anyhow, I hope that I can create minimal reproducers and get them added > to the CI test suite. That would be great. Felix > > > > > Best regards, > > Felix > > > > Felix Moessbauer (2): > > dpkg-raw: add files to source package > > handle DPKG_ARCH=all case for transitive deps > > > > meta/classes/dpkg-raw.bbclass | 4 ++-- > > meta/classes/multiarch.bbclass | 18 +++++++++++++++++- > > 2 files changed, 19 insertions(+), 3 deletions(-) > > > > -- > Cedric Hombourger > Siemens AG > www.siemens.com -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/8f24b7a52dd2fb330ba677850343ea4ed6f37d7f.camel%40siemens.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 2025-10-24 5:34 ` [PATCH 0/2] " 'cedric.hombourger@siemens.com' via isar-users 2025-10-24 7:36 ` 'MOESSBAUER, Felix' via isar-users @ 2025-10-27 13:35 ` 'Andreas Naumann' via isar-users 1 sibling, 0 replies; 21+ messages in thread From: 'Andreas Naumann' via isar-users @ 2025-10-27 13:35 UTC (permalink / raw) To: cedric.hombourger, isar-users, MOESSBAUER, Felix Cc: Steiger, Christoph, Schmidt, Adriaan, Kiszka, Jan Hi all, Am 24.10.25 um 07:34 schrieb 'cedric.hombourger@siemens.com' via isar-users: > On Wed, 2025-08-20 at 14:41 +0200, Felix Moessbauer wrote: >> This series fixes an issue with DPKG_ARCH=all packages that have >> dependencies to other DPKG_ARCH=all packages. Hereby, the transitive >> packages were built for the wrong architecture. The issue became >> apparent when testing the SBOM series by Christoph, which builds >> packages to be used during the ISAR build (not for the final target). >> >> While fixing this, another issue surfaced where files are missing in >> dpkg-raw packages. > In my testing with downstream layers, this change caused some packages > to be built for architectures they were not targeted for. For instance, > I am now seeing the k3-rti-wdt firmware package [1] being built for > both amd64 and arm64. I also had a dpkg-prebuilt recipe for a package > supported on arm64 and amd64: it is now tried for armhf/i386. Please have a look at the commit message of Patch 1 of my RFC I sent a while ago: [PATCH 1/5] rootfs: Do not recursively deploy every dependent package which I have now rephrased as "rootfs: Do not recursively build unneeded packages". I think it's very likely that you're hit by the exact problem i describe there. Later in the RFC i have suggested to drop the optimization part of this patch. Having reworked the series since, it's possible to even keep it. If you want, I can send the reworked version. regards, Andreas > > Then there was this issue with dpkg-customization packages now leading > to a "multiple .bb files are due to be built" error. My work-around was > to add DPKG_ARCH=any to the dpkg-customization class since > customization packages are anyhow decorated with both DISTRO and > MACHINE (it is therefore ok and possibly better to have them explicitly > built for the arch of the MACHINE). > > Note: Jan had suggested that I also pull [2] but that did not help > > I evidently need to provide you more details. I have had a short > discussion with Felix and we suspect that multiconfig builds may be > causing these issues. > > Anyhow, I hope that I can create minimal reproducers and get them added > to the CI test suite. > >> Best regards, >> Felix >> >> Felix Moessbauer (2): >> dpkg-raw: add files to source package >> handle DPKG_ARCH=all case for transitive deps >> >> meta/classes/dpkg-raw.bbclass | 4 ++-- >> meta/classes/multiarch.bbclass | 18 +++++++++++++++++- >> 2 files changed, 19 insertions(+), 3 deletions(-) >> -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mail info@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/027334d6-a5ae-4893-a7ff-f6b9dcb7fc55%40emlix.com. ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2025-10-27 13:36 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-08-20 12:41 [PATCH 0/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-08-20 12:41 ` [PATCH 1/2] dpkg-raw: add files to source package 'Felix Moessbauer' via isar-users 2025-10-03 11:09 ` Anton Mikanovich 2025-08-20 12:41 ` [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps 'Felix Moessbauer' via isar-users 2025-09-10 11:00 ` Andreas Naumann 2025-09-11 10:20 ` 'MOESSBAUER, Felix' via isar-users 2025-09-12 15:50 ` Andreas Naumann 2025-09-15 7:55 ` 'MOESSBAUER, Felix' via isar-users 2025-09-15 17:49 ` Andreas Naumann 2025-09-16 7:16 ` 'MOESSBAUER, Felix' via isar-users 2025-09-19 7:15 ` Andreas Naumann 2025-10-06 14:05 ` 'MOESSBAUER, Felix' via isar-users 2025-10-07 10:57 ` 'Andreas Naumann' via isar-users 2025-10-07 12:31 ` Zhihang Wei 2025-10-20 12:07 ` 'cedric.hombourger@siemens.com' via isar-users 2025-10-20 12:23 ` 'Jan Kiszka' via isar-users 2025-10-20 12:25 ` 'cedric.hombourger@siemens.com' via isar-users 2025-10-24 13:03 ` 'MOESSBAUER, Felix' via isar-users 2025-10-24 5:34 ` [PATCH 0/2] " 'cedric.hombourger@siemens.com' via isar-users 2025-10-24 7:36 ` 'MOESSBAUER, Felix' via isar-users 2025-10-27 13:35 ` 'Andreas Naumann' via isar-users
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox