Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps

[Andreas Naumann <anaumann@emlix.com>]

Am 11.09.25 um 12:20 schrieb MOESSBAUER, Felix:
> On Wed, 2025-09-10 at 13:00 +0200, Andreas Naumann wrote:
>> Hi Felix, all,
>>
>> Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users:
>>> Arch=all packages might build depend on other arch=all packages, hence we
>>> need to correctly model the dependency chain. Otherwise the transitive
>>> dependencies are built for the distro arch instead of the native arch.
>>>
>>> We implement this by dispatching the non-native variant of DPKG_ARCH=all
>>> packages to the -native variant by adding a dependency. We further
>>> replace the non-native do_deploy_dep task with a noop to preserve the
>>> dependency chain.
>>>
>>> Co-developed-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
>>> ---
>>>    meta/classes/multiarch.bbclass | 18 +++++++++++++++++-
>>>    1 file changed, 17 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass
>>> index babdfbd4..c2bba21f 100644
>>> --- a/meta/classes/multiarch.bbclass
>>> +++ b/meta/classes/multiarch.bbclass
>>> @@ -29,7 +29,11 @@ python() {
>>>                d.appendVar('BBCLASSEXTEND', ' compat')
>>>    
>>>        # build native separately only when it differs from the target variant
>>> -    if not archIsAll and archDiffers:
>>> +    # We must not short-circuit for DPKG_ARCH=all packages, as they might
>>> +    # have transitive dependencies which need to be built for -native.
>> Funny enough i suspected to have the same problem yesterday, but it
>> turned out to
>> be different: When a package is single architecture only, e.g. arm64, it
>> should not be
>> extended as native if HOST_ARCH is amd64.
>>
>> So I added a check to BBCLASSEXTEND only if DPKG_ARCH != HOST_ARCH.
> Good catch. This case is currently not handled. But I would like to
> have it in a separate patch, as it tackles another issue.

I thought it's possible to make the nesting of if-clauses easier that
way, but after some more thought and fiddling... unfortunately it's the
opposite. So yes, separate patch.

>
>>> +    # This special handling for DPKG_ARCH=all packages is left to the
>>> +    # multiarch_virtclass_handler
>>> +    if archDiffers:
>>>            d.appendVar('BBCLASSEXTEND', ' native')
>> Now, seeing your patch to remove the not-"all" case also, I wonder if
>> that could be used
>> as the sole check before BBCLASSEXTEND. Like e.g.
>> archIsNative = d.getVar('DPKG_ARCH') == d.getVar('HOST_ARCH')
>> if isNative:
>>      d.appendVar('BBCLASSEXTEND', ' native')
>>>        else:
>>>            extend_provides(pn, 'native', d)
>> but now I wonder what the extend_provides is needed for. If HOST_ARCH ==
>> DISTRO_ARCH, what's
>> the use of providing a native variant?
> If we are building for the native arch, we can just add the -native
> variants to the PROVIDES, so dependencies of packages depending
> explicitly on <foo>-native can be resolved (without the need for extra
> recipe variants).

I understand that its a simpler solution in that case. But, maybe I'm not
seeing the obvious, what would require a -native package then? If 
target-arch
is host-arch, why would anything else than the base package be in any
dependency chain?

>
>> But back to your Patch: Is it possible that -compat "all" packages also have
>> dependencies on other "all" packages, which in turn have dependencies on
>> architecture specific packages?
> Good question. In general, package dependencies are resolved for the
> native architecture (or the requested one). AFAIK this selection is
> propagated down the dependency tree. By that, A:i386 -> B:all -> C:x
> should always resolve C as either :all or i386 (except if there is a
> explicit cross-arch dependency specifier like :any, which we don't
> model in bitbake). Anyways, The A:i386 -> B:all -> C:i386 chain should
> already be resolved correctly with this patch.
The patch doesnt change that -compat is "short-circuited" when 
archIsAll, doesnt it?
Or maybe I dont fully understand how the code works yet.
>
> Tested by's are welcome :)

I'm working with it and keep an eye on it.

regards,
Andreas

>
> Felix
>
>> regards,
>> Andreas
>>
>>
>>> @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() {
>>>                d.setVar(var, ' '.join(multiarch_var))
>>>    
>>>        pn = e.data.getVar('PN')
>>> +    archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH')
>>> +    archIsAll = d.getVar('DPKG_ARCH') == 'all'
>>>        if pn.endswith('-compat'):
>>>            e.data.setVar('BPN', pn[:-len('-compat')])
>>>            e.data.appendVar('OVERRIDES', ':class-compat')
>>> @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() {
>>>            e.data.appendVar('OVERRIDES', ':class-native')
>>>            fixup_pn_in_vars(e.data)
>>>            fixup_depends('-native', e.data)
>>> +    elif archIsAll and archDiffers:
>>> +        # Arch=all packages might build depend on other arch=all packages,
>>> +        # hence we need to correctly model the dependency chain.
>>> +        # We implement this by dispatching the non-native variant to the -native
>>> +        # variant by adding a dependency. We further replace the non-native
>>> +        # do_deploy_dep task with a noop to preserve the dependency chain.
>>> +        e.data.setVar('do_deploy_deb', '')
>>> +        bb.build.deltask('deploy_deb', e.data)
>>> +        bb.build.addtask('deploy_deb', 'do_build', '', e.data)
>>> +        e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb')
>>>    }
>>>    addhandler multiarch_virtclass_handler
>>>    multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
>> -- 
>> Andreas Naumann
>>
>> emlix GmbH
>> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany
>> Phone +49 (0)551 30664-0, e-mail info@emlix.com
>> District Court of Goettingen, Registry Number HR B 3160
>> Managing Directors: Heike Jordan, Dr. Uwe Kracke
>> VAT ID No. DE 205 198 055
>> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany
>> Office Bonn: Bachstr. 6, 53115 Bonn, Germany
>> http://www.emlix.com

-- 
Andreas Naumann

emlix GmbH
Headquarters: Berliner Str. 12, 37073 Goettingen, Germany
Phone +49 (0)551 30664-0, e-mail info@emlix.com
District Court of Goettingen, Registry Number HR B 3160
Managing Directors: Heike Jordan, Dr. Uwe Kracke
VAT ID No. DE 205 198 055
Office Berlin: Panoramastr. 1, 10178 Berlin, Germany
Office Bonn: Bachstr. 6, 53115 Bonn, Germany
http://www.emlix.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/f7ac34b9-c96b-4939-aec4-c72bcd471db9%40emlix.com.