From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Tue, 07 Oct 2025 12:57:59 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f61.google.com (mail-wm1-f61.google.com [209.85.128.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 597AvwKM001301 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 7 Oct 2025 12:57:58 +0200 Received: by mail-wm1-f61.google.com with SMTP id 5b1f17b1804b1-46e3ef2dd66sf30629225e9.1 for ; Tue, 07 Oct 2025 03:57:58 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1759834673; cv=pass; d=google.com; s=arc-20240605; b=kfMGnFORzyziScNb70KZNi7DunPB4rIH9F4MdCJ6HSitNZeKxa8jrXWWZjzXJQtZbv 399sWlSpIE4afRgLmWUxPjOINduHMC1zK8mNfRAnTpp+k+xIiDYeUKH50pBNv8eaAwPR 1LQjFyJR4W+0n37SIy9gB6gYFabNQuRyTR8BfxW7ijZgYA3ft+rBSDhBdKV1NIYM/GsT Viwh62TxGfJMD4InASrV5OomPDSskfwois8gNeXS0TSMxAOrmhaOjhwk3pxPS0I1Nma2 vyb5PByOVyNSBWe0LnpWFk2YJdHrqY6XTVrSDlKBxYB/5u56TfvV82r9OEWc34ARTp7h fOeg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:in-reply-to:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:dkim-signature; bh=O7aoXI5uxLzUeyxtbKXmDnogzu1UAc6jaX9OuyhtPiw=; fh=SRS6am9TWzCf+j9xS91xozI3cnnLtAEYVkdhee5fQiU=; b=btP9b+YHNeiQsSALuDZ7pAxLuMii9inAFQVQu0hhv7OM11LZHfioAsZmHa+ebO7cH0 2aPBDpx6tbrmHEAwXSD/wU9Ic59jnOqURzTxmMiROiug0Yf87Aj1zGIiSduWFHGYPUp+ JQ7G9BF6X4K11oHeto03ozcjGQWR0lKyLHK4VU8jPICEAvmrS1KDEe0y6EtE4r76gjEy qpMNVLFfWnjzd1ESUBG+EQn3SvA0DGadeEjN5pQ/xdh5Gtlht7jzFNrXJoeo/Ke5DlOA k18ZzuKRsP1or7VBloUDrwq71+f1kK1luvIqolGayItWwbqC92XpCdlwMsgQihYcOIV7 sDuQ==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@emlix.com header.s=20250930 header.b=TPERubIh; dkim=pass header.i=@emlix.com header.s=20250930 header.b=TPERubIh; spf=pass (google.com: domain of anaumann@emlix.com designates 178.63.209.131 as permitted sender) smtp.mailfrom=anaumann@emlix.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=emlix.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1759834673; x=1760439473; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:in-reply-to :from:content-language:references:to:subject:user-agent:mime-version :date:message-id:from:to:cc:subject:date:message-id:reply-to; bh=O7aoXI5uxLzUeyxtbKXmDnogzu1UAc6jaX9OuyhtPiw=; b=FInxxV9o3W2Ed1MhkXTMiC2vgg/k11f1//OoV9vuh/iwcfqtxWnXKK/3n0eeS0to+x OTjoetrXOkw85cGav6rsWH6f02lfGbSslQivqUgmiJts6Ty+MEjh0Ydtp7qUS9/7qW/0 As0iYbL76Egh/WzySkKDigAtx/ESHz85NxbfH3L8G85+n+mqZTQE5OGN+CMWybNpdPRS GJkiqZYaaLB4sJJCEPhttqUoqJ5AAGN3AQUFuHsuYzhCUtlHbQn93o3ep7TP/ylE6AMg EIrq2xlzAdg1ifXnTnrMfai/whWfiFkp341med5LMulmHXKtyixNrNtueubs/SdqQlfr LgYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759834673; x=1760439473; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:in-reply-to :from:content-language:references:to:subject:user-agent:mime-version :date:message-id:x-beenthere:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=O7aoXI5uxLzUeyxtbKXmDnogzu1UAc6jaX9OuyhtPiw=; b=AgnPxYx7TBFWrBpnOlvQU6G9nldOwFcF/lT/edyNbm5kXma+XAICHBcwdyBYN9Mw7g t2PJ8/N2iP4FX9KF6AM5cFRUKcAJmHCqINRst05G8kfYcY9pGB5ON7zsX3r39vxOog72 5SAfm5Kd+oXj3jdj6iReTfrqUfRX6hYEE1VeBEe5mPFosIjQTv2HRvQzTazRo6PpzfEc U9Libtzat+CyWkqC0FseAU9oNZVhmx2pyKXtSLmlH9KOqMq09wufTl3LME1bui/RIq+A KY6ZQcZzQa5tswapv0grPrOV8mKEZXrqXDBBHB9/qli5KLxUiHAbpx+9mkmIv4kg54r6 G5mQ== X-Forwarded-Encrypted: i=2; AJvYcCVwi8ZFvmH+H5DLTgzC6IdiLRkaEJA5XqCpSrdGhXbCqyzMuponB6iAXkzaw6D9745Pnnm/@ilbers.de X-Gm-Message-State: AOJu0Ywl0PGhQNEwBsqG/20WSOqORnAfevi2LhBcE2+OQIP+mU7KWkRO 0ZNTZ4FVGk4JliFB0KG3nTZ490wLpVHHnt9jtnk7OGonTzquAEVglFzh X-Google-Smtp-Source: AGHT+IFYdjRxv8XQkSO2mJxiLsmuFj0uv7jTAydw+nliIrP+6tmW680ocafHpX+8z3EjO2yOdNjggA== X-Received: by 2002:a05:600c:a105:b0:46d:83e7:45ec with SMTP id 5b1f17b1804b1-46fa29e1e0amr13805025e9.11.1759834672952; Tue, 07 Oct 2025 03:57:52 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd6gXETnGNc8Q2JG4rYum4ju4jlwLMWLrQ0D3gVuQMbhOQ==" Received: by 2002:a05:600c:8a16:20b0:46e:2277:de67 with SMTP id 5b1f17b1804b1-46e689e7e2als13353405e9.0.-pod-prod-00-eu; Tue, 07 Oct 2025 03:57:50 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXdzQTAXgsUyfPd7PMIORjK3ed/TC8WVkAAlS5qDNDUFdxhZeQgPCdSwhpXMqWVmnJ5ut0mznoUPaHj@googlegroups.com X-Received: by 2002:a05:600c:c0d6:b0:46e:1b9d:ac6c with SMTP id 5b1f17b1804b1-46fa29ffeebmr14314425e9.17.1759834670152; Tue, 07 Oct 2025 03:57:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1759834670; cv=none; d=google.com; s=arc-20240605; b=AyWOTJ8Voo2tKakvQsxTAufnqOPJJEhDvrTTac/aANY9wrYLxNsqSAlSVALahxWxFf n4r3+O3ojMc7wMeMEg8kqrQ74Ja4Agb4Mo8beURPxIvoqatS94tRO+oC/JZBhdhyqsVy z0U2L4b3yjQSHEqopc7y7BZVGj97mNMQ7IGYq7xH+fHxN0gsG5IWJexBmaOBzLFGkEKQ eqDFB4HKnDHQ0O54MxiSEr4Y9puVvnn307dtVZa7hr1vnso/hs2nz8vD/pw+SWkB6SSi JXTJbWfBq6QXE5RMX925TNesPB/fbjsXw3t/zCGjTQuZoH+4VzAvfrzdlpMvxoMo8ZGp XQxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :dkim-signature:dkim-signature; bh=8KGPQ+KDhFKcnGnO7ptEoBEQw02v/yfixVEHybKxULM=; fh=dAUR+lF0BC55N/OTDJkDh94ke1QeUZyJ14RzwNCC3BI=; b=WQAoUrXjkM0pJ3vmYPxZ8ld9nFZoXg509CXcqanx1xutCJ3XA70FK4HlO+jWbF0ge9 T8DzdhfvtbVaHuFmSvaYQAiW7pcIzSqUnwEScvDED//RXzyMMWuP2VO39SA/70wqpm8m /auUNrtHAs3bJfR5sdCE9Qr9IulRQYl0hhdvJwM/kSFIDqfqFYY/R9iyhJzfplCrNS/6 ttoZPof8Ja6S4aQLjVKBWvuj7k4H5PrxgQpRuknnbL3wa4CkH8YNS/lK4suk5u1MpPyc xVTYz0Hdk0YiAR5hhhKiUb1t1EZTEuF20WHd1PZRFDC6LC4Iv2ftoFzdxETJ7bxjZUPZ bBtQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@emlix.com header.s=20250930 header.b=TPERubIh; dkim=pass header.i=@emlix.com header.s=20250930 header.b=TPERubIh; spf=pass (google.com: domain of anaumann@emlix.com designates 178.63.209.131 as permitted sender) smtp.mailfrom=anaumann@emlix.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=emlix.com Received: from mx1.emlix.com (mx1.emlix.com. [178.63.209.131]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-46e6918b245si2997725e9.1.2025.10.07.03.57.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Oct 2025 03:57:50 -0700 (PDT) Received-SPF: pass (google.com: domain of anaumann@emlix.com designates 178.63.209.131 as permitted sender) client-ip=178.63.209.131; Received: from mx1.emlix.com (localhost [127.0.0.1]) by mx1.emlix.com (Postfix) with ESMTP id B77EB5F953; Tue, 7 Oct 2025 12:57:49 +0200 (CEST) Received: from mailer.emlix.com (p5098be52.dip0.t-ipconnect.de [80.152.190.82]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.emlix.com (Postfix) with ESMTPS id 9A5BC5F774; Tue, 7 Oct 2025 12:57:49 +0200 (CEST) Message-ID: <6c272cfc-ca61-4ec9-8913-7497cf43001e@emlix.com> Date: Tue, 7 Oct 2025 12:57:47 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/2] handle DPKG_ARCH=all case for transitive deps To: "MOESSBAUER, Felix" , "Steiger, Christoph" , "isar-users@googlegroups.com" , "Kiszka, Jan" References: <20250820124140.281190-1-felix.moessbauer@siemens.com> <20250820124140.281190-3-felix.moessbauer@siemens.com> <05c06d5d-5248-4d56-b470-a096411b2844@emlix.com> <9af00c922ab12ee5961d377babb3a87f49efac6c.camel@siemens.com> <45ecefef-bbc3-4506-b9b3-c945d0d77e65@emlix.com> <56507dd43d6b0f0e309563ebbe2d48327353cdc5.camel@siemens.com> Content-Language: en-US From: "'Andreas Naumann' via isar-users" In-Reply-To: <56507dd43d6b0f0e309563ebbe2d48327353cdc5.camel@siemens.com> Content-Type: text/plain; charset="UTF-8"; format=flowed X-Virus-Scanned: ClamAV using ClamSMTP X-Original-Sender: anaumann@emlix.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@emlix.com header.s=20250930 header.b=TPERubIh; dkim=pass header.i=@emlix.com header.s=20250930 header.b=TPERubIh; spf=pass (google.com: domain of anaumann@emlix.com designates 178.63.209.131 as permitted sender) smtp.mailfrom=anaumann@emlix.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=emlix.com X-Original-From: Andreas Naumann Reply-To: Andreas Naumann Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: tXWt4eTGjeQ7 Hi Felix, all Am 06.10.25 um 16:05 schrieb MOESSBAUER, Felix: > On Fri, 2025-09-19 at 09:15 +0200, Andreas Naumann wrote: >> Hi Felix, all >> >> Am 16.09.25 um 09:16 schrieb MOESSBAUER, Felix: >>> On Mon, 2025-09-15 at 19:49 +0200, Andreas Naumann wrote: >>>> Hi Felix, >>>> >>>> Am 20.08.25 um 14:41 schrieb 'Felix Moessbauer' via isar-users: >>>>> Arch=all packages might build depend on other arch=all packages, hence we >>>>> need to correctly model the dependency chain. Otherwise the transitive >>>>> dependencies are built for the distro arch instead of the native arch. >>>>> >>>>> We implement this by dispatching the non-native variant of DPKG_ARCH=all >>>>> packages to the -native variant by adding a dependency. We further >>>>> replace the non-native do_deploy_dep task with a noop to preserve the >>>>> dependency chain. >>>>> >>>>> Co-developed-by: Adriaan Schmidt >>>>> Signed-off-by: Felix Moessbauer >>>>> --- >>>>> meta/classes/multiarch.bbclass | 18 +++++++++++++++++- >>>>> 1 file changed, 17 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass >>>>> index babdfbd4..c2bba21f 100644 >>>>> --- a/meta/classes/multiarch.bbclass >>>>> +++ b/meta/classes/multiarch.bbclass >>>>> @@ -29,7 +29,11 @@ python() { >>>>> d.appendVar('BBCLASSEXTEND', ' compat') >>>>> >>>>> # build native separately only when it differs from the target variant >>>>> - if not archIsAll and archDiffers: >>>>> + # We must not short-circuit for DPKG_ARCH=all packages, as they might >>>>> + # have transitive dependencies which need to be built for -native. >>>>> + # This special handling for DPKG_ARCH=all packages is left to the >>>>> + # multiarch_virtclass_handler >>>>> + if archDiffers: >>>>> d.appendVar('BBCLASSEXTEND', ' native') >>>>> else: >>>>> extend_provides(pn, 'native', d) >>>>> @@ -86,6 +90,8 @@ python multiarch_virtclass_handler() { >>>>> d.setVar(var, ' '.join(multiarch_var)) >>>>> >>>>> pn = e.data.getVar('PN') >>>>> + archDiffers = d.getVar('HOST_ARCH') != d.getVar('DISTRO_ARCH') >>>>> + archIsAll = d.getVar('DPKG_ARCH') == 'all' >>>>> if pn.endswith('-compat'): >>>>> e.data.setVar('BPN', pn[:-len('-compat')]) >>>>> e.data.appendVar('OVERRIDES', ':class-compat') >>>>> @@ -96,6 +102,16 @@ python multiarch_virtclass_handler() { >>>>> e.data.appendVar('OVERRIDES', ':class-native') >>>>> fixup_pn_in_vars(e.data) >>>>> fixup_depends('-native', e.data) >>>>> + elif archIsAll and archDiffers: >>>>> + # Arch=all packages might build depend on other arch=all packages, >>>>> + # hence we need to correctly model the dependency chain. >>>>> + # We implement this by dispatching the non-native variant to the -native >>>>> + # variant by adding a dependency. We further replace the non-native >>>>> + # do_deploy_dep task with a noop to preserve the dependency chain. >>>>> + e.data.setVar('do_deploy_deb', '') >>>>> + bb.build.deltask('deploy_deb', e.data) >>>>> + bb.build.addtask('deploy_deb', 'do_build', '', e.data) >>>>> + e.data.setVarFlag('do_deploy_deb', 'depends', f'{pn}-native:do_deploy_deb') >>>> I have now done some more testing and unfortunately find that this >>>> causes failures when building the -native variant of some dpkg-raw >>>> packages we have. >>>> >>> Hi, do you have an example for such a package? Just to be clear: There >>> is no "native" variant of a dpkg-raw package. The dpkg-raw packages are >>> always build on the host architecture (in bitbake terms) and of Debian >>> arch=all. >>> >>>> We use them for certain config files and they are >>>> intended for the target. >>>> >>> Do these configs contain data that is not the same on all >>> architectures? If so, DPKG_ARCH="all" is wrong. >> After some debugging, it seems to be a combination of your patch and >> "2ca3a7e dpkg-source: Build source package only once". There is no >> problem building our dpkg-raw packages explicitly in the non-native >> form, e.g. in the host environment for the target arch. >> But through your patch, the package also runs as -native, which then >> triggers a problem with 2ca3a7e. >> >> ... I only now discover that the other part of your patchset "dpkg-raw: >> add files to source package" fixes this. Ok. > Ok, so in general this series is fine. ACK, to get the sbom series going. However, I have proposed and would like to invite you to comment on my [RFC 0/5] Improving multiarch support for arch-incompatible packages. I think it provides a straightforward, generic solution that works for "all" and non-all packages alike. >> However, because of this I went digging on why you introduced the patch. >> And I wonder why sbom-chroot doesn't just depend on >> python3-debsbom-native directly? (Which still breaks as long as 2ca3a7e >> sets the dependency on a function of the non-native package). > This is a more general thing: The chroot is architecture specific. By > that, the architecture of the packages (DEPENDS) is selected based on > that without the need for having -native. The internal logic then takes > care of building the packages in the correct variant (e.g. -compat, > -native or without any suffix, which refers to the target arch). I'm not sure I understand what you mean. If python3-debsbom was an "any" package, how would the logic guess that it's needed for the host? best regards, Andreas > > The 2ca3a7e ("dpkg-source: Build source package only once") just > ensures that we only create a single source package, no matter for > which (and how many) architectures we build it. > >> >>>> I dont know why exactly they fail and I'm sure >>>> this could be fixed, but actually there is no need to build those >>>> packages in the native environment. >>>> >>> There is need to do so, as otherwise we emulate the build process which >>> is super slow. >> Yes you're right. I meant there is no need to build them *for* the >> native architecture. >>>> So I'm somewhat surprised to see >>>> that and could image that this causes confusion for others that dont >>>> know about the "all" intricacies as well. >>> Probably, but in the end we are building a debian system and by that >>> the users should be aware of the debian architecture specifiers. >> Yes sure, they were new to me, but no problem, they are not that >> complicated. >> >> But when I deliberately build -native packages (which I assume to be a >> valid use case according to the doc), and somehow the buildsystem goes >> back and forth running functions from the native and the base package, >> it's difficult to follow why that would be necessary. It doesnt make for >> ease of debugging when working on new packages either. >> >> For the use cases I have in sight right now it would be fine if a >> package build breaks because it cannot be done cross-architecture. I'd >> hope to being able to fix this by either improving the package or making >> sure to call it -native only. But if that's not what the expectation is >> for Debian Crossbuilding I'm eager to learn. > If there are no further issues with this patchset, I would like to move > forward and see it merged. It is a pre-condition for the SBOM patches > for non target!=host arch. > > Best regards, > Felix > >> best regards, >> Andreas >> >>>> Another observation that I see is that dpkg-prebuilt "any" packages, >>>> which are probably in the dependency chain of an "all" package, now are >>>> always also built as -native, even though they are needed for the target >>>> only. >>> Same here, even if bitbake tells you that they are build as "-native", >>> there is no difference in the output, as the binary is simply copied. >>> >>> Best regards, >>> Felix >>> >>>> best, >>>> Andras >>>> >>>>> } >>>>> addhandler multiarch_virtclass_handler >>>>> multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" >>>> -- >>>> Andreas Naumann >>>> >>>> emlix GmbH >>>> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany >>>> Phone +49 (0)551 30664-0, e-mailinfo@emlix.com >>>> District Court of Goettingen, Registry Number HR B 3160 >>>> Managing Directors: Heike Jordan, Dr. Uwe Kracke >>>> VAT ID No. DE 205 198 055 >>>> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany >>>> Office Bonn: Bachstr. 6, 53115 Bonn, Germany >>>> http://www.emlix.com >> -- >> Andreas Naumann >> >> emlix GmbH >> Headquarters: Berliner Str. 12, 37073 Goettingen, Germany >> Phone +49 (0)551 30664-0, e-mailinfo@emlix.com >> District Court of Goettingen, Registry Number HR B 3160 >> Managing Directors: Heike Jordan, Dr. Uwe Kracke >> VAT ID No. DE 205 198 055 >> Office Berlin: Panoramastr. 1, 10178 Berlin, Germany >> Office Bonn: Bachstr. 6, 53115 Bonn, Germany >> http://www.emlix.com -- Andreas Naumann emlix GmbH Headquarters: Berliner Str. 12, 37073 Goettingen, Germany Phone +49 (0)551 30664-0, e-mail info@emlix.com District Court of Goettingen, Registry Number HR B 3160 Managing Directors: Heike Jordan, Dr. Uwe Kracke VAT ID No. DE 205 198 055 Office Berlin: Panoramastr. 1, 10178 Berlin, Germany Office Bonn: Bachstr. 6, 53115 Bonn, Germany http://www.emlix.com -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/6c272cfc-ca61-4ec9-8913-7497cf43001e%40emlix.com.