From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 28 Nov 2024 07:04:11 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-wm1-f59.google.com (mail-wm1-f59.google.com [209.85.128.59]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 4AS64Atq017552 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 28 Nov 2024 07:04:10 +0100 Received: by mail-wm1-f59.google.com with SMTP id 5b1f17b1804b1-434a90febb8sf1910975e9.1 for ; Wed, 27 Nov 2024 22:04:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1732773845; cv=pass; d=google.com; s=arc-20240605; b=Y7e7Qe2i+eMvmdNcdZuY/x3ElqIpyVdR3+B/skLt1QtXLzcRabuKE7XgYsYQxSsWnf bCYBsIVVSxQiTI1UGEHfJ2owpn5X3ClQ52rdkgXO+UHbNdHG/ZCsJJmn0bQ98z63jnX3 sHLzQRMf61l4E1rTWxCaJ/yz/NPhC9kzQPCliQz+G+tDxVu1VdB7iqd4GoRtDfpf7hWM vpdRf3KZ/8m54CsaM1yhDWRIQTT1iAjobHudI0j/Jmp9+Z6mIBDVDOQ05lnhFeb/r1Gv /wEJahyx+BiJxV0Pizb/xjbRz08TSKyOlhWGaCxIOHTnt0B6+fOXSn56NbXCdlxvIaUk F+bQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:user-agent :content-transfer-encoding:references:in-reply-to:date:to:from :subject:message-id:sender:dkim-signature; bh=HA7B2+t68vAYtiQ1yXgkxHSVNQjP2D0xd1/QLd24Z8s=; fh=W5MZgCHWLu8meaWIlsBcI2+C3qcj9j1NkZZZUvhThkY=; b=ffxOQ4mtH42IppcU0A11VvINyMagoTx3Mg+CBK8HSGzU/+a2eRX4LyHCjT4sQc936G gT1QhzCqB3ngFOLVzV/fSOkhCVvzW3xLvZDA+HvyLGTQ06Q0k2p7p5BGtHDyu6mSQJjt 42TkotNB6ZBDVTDNBrrDoZccyQ8m4wdbBCLF6CeROQpCQciBmP1gGBIs4p4h31zZU/I3 HK7idlPCaKHZKg3H3HI2pFow2PWRwYJbX+G6ZzWuiuNTvC9xkfKdm+GBUiktych8+ntm 3fn2dg0MA/MMa5e+v2O6B/+Q1V+bZfjRf2MWgR75scuE5l42L4VbVPquhwVr/do21HpJ /Acg==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1732773845; x=1733378645; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:user-agent:content-transfer-encoding :references:in-reply-to:date:to:from:subject:message-id:sender:from :to:cc:subject:date:message-id:reply-to; bh=HA7B2+t68vAYtiQ1yXgkxHSVNQjP2D0xd1/QLd24Z8s=; b=s7B7irkIxkvtVpghoQQQ4ni+1QDULhwjgGb2uYwqyVgXWaLPI7739yKY/hYGQdV4C4 f4n4ch5oMMU7XsgpAh824d4u53IPCub7CRgS12y6FUfSpC854vEJ91ft+uz/WDOBzsbB m0VydqTkaa++8gycDoi875WjKHVxicVdpCgnEjoG8rc8QXcXc/JMmfm14Nqv0NZXxqg0 V6YEu3RkU8FJwCRi8TyOokNhe34IdMW86M36JC7nLqB+3IHSfPa7MBdiFAGdAg/N7vIk HykvjyRaCNyjcy1BKGQvVTblw5jK3qywPyLgFYVRKTXfKiLcwTctSxst6ISnpZrxBNW1 S2Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732773845; x=1733378645; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :user-agent:content-transfer-encoding:references:in-reply-to:date:to :from:subject:message-id:x-beenthere:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=HA7B2+t68vAYtiQ1yXgkxHSVNQjP2D0xd1/QLd24Z8s=; b=CajPnxbCcThz2HcsmgUzJMHPdasDFozzU5+JhAIm5I59LVY3ZIJqRqVEnd9OxgJA8w xZKAkfHaH5oes1vttXRQtjL46A1Od88TJqny8O6HfAGFaTWcuNWw5cGiYKGcq8qWxw4L riqNJJCGV0gMg3u2CW3EhYdb0GMAMOkWEM0kbNzwW4iSwS6wZzEznaVzvLRW6NEclhbs 1XuqMtQEpSlSaOCBXkfhT9mu4PZKV48APV4pvR8pIeKmkU0cBLiFir+Q/F5ZV9o8MM4A Krbc8bcPH0afpp9BJk8zhV0JWI3yv0ZBAVs6AlAvOpA1coMhfn4zh7OPhdWo6H3G8Pfd /0WA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCWBvNFCD+SBczYaJm0iSqjcYLPFmTa2YoGWuKX5Vi60/i0KA4fgh4HVXyCKq+1kMyRe9/IV@ilbers.de X-Gm-Message-State: AOJu0YyWoewGDSjB4ygjhVo/c8azFNtBrTLnF6XzJrMP4RMmWrvxL1VG U6K/+nQ8mt41xz4uEIq8fDBL92rTwRRPOde1ZcYAnguMEHJ+ZG1I X-Google-Smtp-Source: AGHT+IHYsnpWP2V6cCRTY6eXiIERm9n2QCqT9x0BpfKwJqc4YxgljZKfGX6yNaE7VnTccXgIZ6Jc2g== X-Received: by 2002:a05:600c:b87:b0:434:a1e7:27b0 with SMTP id 5b1f17b1804b1-434a9dc3d08mr49962035e9.11.1732773844403; Wed, 27 Nov 2024 22:04:04 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:3549:b0:434:a131:813b with SMTP id 5b1f17b1804b1-434afc3f10bls2552275e9.0.-pod-prod-01-eu; Wed, 27 Nov 2024 22:04:02 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUzHXR25pthsQMbCsdddu1mOKzmX1rsWtrh0y4ojsw6b9arFCSfttbEYojCt1r2yslo5ZDTyIXtnwK9@googlegroups.com X-Received: by 2002:a05:600c:4f46:b0:434:a929:42bb with SMTP id 5b1f17b1804b1-434a9dd0cfamr57384185e9.18.1732773841978; Wed, 27 Nov 2024 22:04:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1732773841; cv=none; d=google.com; s=arc-20240605; b=e/9MDrJqZRF5vTOJNM/9bQ/pLlBiNvkFqpj7UMoYhfwS3AB31ofiVYtwofNl+F2qiU MbMVD/+oWQFTlO6ORDZWwcgVK4X5C0K9+YhKpgnmoqIoktacEBjV2lc5nMwtDsuoj+Hn VbBvY8Ftbw9rTIxYjiMNC/L+WFoVJL1MVEZ8CP9iSF3xm4xafTmNcwdQ+y68zEyg5uGx YOdwkkf8M+rqN9WrbO5ro2+QB8ReLat3Z/cT6obdhMYORmmVQQOU1VR7KImFo6vv+uyK MQLeo92HqYqgP++6OinO+7bvKG0UaXI7mS2RIt9KRyzi3WWsb+pfW/zKdLekSZLs9Haj mDfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id; bh=D3omwpzR4aTll7BOjldPFdpqb6+44bCSJss9M07B9eQ=; fh=IxbeHM4vWxalf1XNf2rnGcQL+r1v0Lr0C6ICsGNNkYw=; b=dbDAySMtChvrsJ4oH9fyBh++g9mC0aT9vsgOx97NbLl3rvix1SNfD1yQp8MFUS4u0a pWFgel+E+v7GosznuFfrwctaJdD8bJHF9KyYK29Q5zlLnPGAKVTvz10Xcqr19or0jm2+ pcZcPvYTTC7huKXKQQeTQJKHHOgryPWe+woQULVlHP80F7kMseAp+DgucgqvOkjESr0k CYQxi9zOqdfzzFyFBMrfhm2CTC+7RZZZHhI3cuuTncPq3MrOlJHFPHFS1HbOeO0a+DaR 4APkn26F8lmh+gd5RqFnUH2SV7Ny1JC7WukHDyzQBkMqi+E0wrH1dnE5OoE9JJgnRd1R O8PA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-434a5d605aesi2309235e9.0.2024.11.27.22.04.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 27 Nov 2024 22:04:01 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 4AS63xCr017543 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 28 Nov 2024 07:04:00 +0100 Message-ID: Subject: Re: [PATCH v2] Convert apt source fetcher into native bitbake variant From: Uladzimir Bely To: Jan Kiszka , isar-users Date: Thu, 28 Nov 2024 09:03:59 +0300 In-Reply-To: <245eca35-04a6-4b72-b528-15b33d661a40@siemens.com> References: <9d1ebee6fe3ffb4040a1c5ea09c2f19700418d64.camel@ilbers.de> <245eca35-04a6-4b72-b528-15b33d661a40@siemens.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.54.1 (by Flathub.org) MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ubely@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: zn2zVzZg0qTk On Thu, 2024-11-28 at 12:55 +0800, Jan Kiszka wrote: > On 27.11.24 22:07, Uladzimir Bely wrote: > > On Fri, 2024-11-15 at 17:40 +0100, Jan Kiszka wrote: > > > From: Jan Kiszka > > >=20 > > > There is no major functional difference, but we no longer have to > > > manipulate SRC_URI by registering an official fetcher for apt://. > > >=20 > > > As the fetching no longer takes place in separate tasks, do_fetch > > > and > > > do_unpack need to gain the extra flags that were so far assigned > > > to > > > apt_fetch and apt_unpack. That happens conditionally, i.e. only > > > if > > > SRC_URI actually contains an apt URL. > > >=20 > > > One difference to the original version is the possibility - even > > > if > > > practically of minor relevance - to unpack multiple apt sources > > > into > > > S. > > > The old version contained a loop but was directing dpkg-source to > > > a > > > pre-existing dir which would have failed on the second iteration. > > > The > > > new version now folds the results together after each step. > > >=20 > > > Another minor difference is that unversioned fetches put their > > > results > > > into the same subfolder in DL_DIR, also when specifying a distro > > > codename. Only versioned fetches get dedicated folders (and .done > > > stamps). > > >=20 > > > There is no progress report realized because dpkg-source > > > unfortunately > > > does not provide information upfront to make this predictable, > > > thus > > > expressible in form of percentage. > > >=20 > > > Signed-off-by: Jan Kiszka > > > --- > > >=20 > > > Changes in v2: > > > =C2=A0- rebased, including the removal of isar-apt sources in > > > apt_unpack > > >=20 > > > I'm carefully optimistic that this change also resolves the > > > previously=20 > > > seen issue in CI. > > >=20 > > > =C2=A0meta/classes/dpkg-base.bbclass | 104 ++++----------------------= - > > > ---- > > > -- > > > =C2=A0meta/lib/aptsrc_fetcher.py=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 93 > > > +++++++++++++++++++++++++++++ > > > =C2=A02 files changed, 104 insertions(+), 93 deletions(-) > > > =C2=A0create mode 100644 meta/lib/aptsrc_fetcher.py > > >=20 > > > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg- > > > base.bbclass > > > index b4ea8e17..c02c07a8 100644 > > > --- a/meta/classes/dpkg-base.bbclass > > > +++ b/meta/classes/dpkg-base.bbclass > > > @@ -79,110 +79,28 @@ do_adjust_git[lockfiles] +=3D > > > "${DL_DIR}/git/isar.lock" > > > =C2=A0inherit patch > > > =C2=A0addtask patch after do_adjust_git > > > =C2=A0 > > > -SRC_APT ?=3D "" > > > - > > > -# filter out all "apt://" URIs out of SRC_URI and stick them > > > into > > > SRC_APT > > > =C2=A0python() { > > > -=C2=A0=C2=A0=C2=A0 src_uri =3D (d.getVar('SRC_URI', False) or "").sp= lit() > > > +=C2=A0=C2=A0=C2=A0 from bb.fetch2 import methods > > > =C2=A0 > > > -=C2=A0=C2=A0=C2=A0 prefix =3D "apt://" > > > -=C2=A0=C2=A0=C2=A0 src_apt =3D [] > > > -=C2=A0=C2=A0=C2=A0 for u in src_uri: > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if u.startswith(prefix): > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s= rc_apt.append(u[len(prefix) :]) > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d= .setVar('SRC_URI:remove', u) > > > +=C2=A0=C2=A0=C2=A0 # apt-src fetcher > > > +=C2=A0=C2=A0=C2=A0 import aptsrc_fetcher > > > +=C2=A0=C2=A0=C2=A0 methods.append(aptsrc_fetcher.AptSrc()) > > > =C2=A0 > > > -=C2=A0=C2=A0=C2=A0 d.prependVar('SRC_APT', ' '.join(src_apt)) > > > +=C2=A0=C2=A0=C2=A0 src_uri =3D (d.getVar('SRC_URI', False) or "").sp= lit() > > > +=C2=A0=C2=A0=C2=A0 for u in src_uri: > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if u.startswith("apt://")= : > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d= .appendVarFlag('do_fetch', 'depends', > > > d.getVar('SCHROOT_DEP')) > > > =C2=A0 > > > -=C2=A0=C2=A0=C2=A0 if len(d.getVar('SRC_APT').strip()) > 0: > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bb.build.addtask('apt_unp= ack', 'do_patch', '', d) > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bb.build.addtask('cleanal= l_apt', 'do_cleanall', '', d) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d= .appendVarFlag('do_unpack', 'cleandirs', > > > d.getVar('S')) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d= .setVarFlag('do_unpack', 'network', > > > d.getVar('TASK_USE_SUDO')) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 b= reak > > > =C2=A0 > > > =C2=A0=C2=A0=C2=A0=C2=A0 # container docker fetcher > > > =C2=A0=C2=A0=C2=A0=C2=A0 import container_fetcher > > > -=C2=A0=C2=A0=C2=A0 from bb.fetch2 import methods > > > =C2=A0 > > > =C2=A0=C2=A0=C2=A0=C2=A0 methods.append(container_fetcher.Container()= ) > > > =C2=A0} > > > =C2=A0 > > > -do_apt_fetch() { > > > -=C2=A0=C2=A0=C2=A0 E=3D"${@ isar_export_proxies(d)}" > > > -=C2=A0=C2=A0=C2=A0 schroot_create_configs > > > - > > > -=C2=A0=C2=A0=C2=A0 session_id=3D$(schroot -q -b -c ${SBUILD_CHROOT}) > > > -=C2=A0=C2=A0=C2=A0 echo "Started session: ${session_id}" > > > - > > > -=C2=A0=C2=A0=C2=A0 schroot_cleanup() { > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 schroot -q -f -e -c ${ses= sion_id} > /dev/null 2>&1 > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 schroot_delete_configs > > > -=C2=A0=C2=A0=C2=A0 } > > > -=C2=A0=C2=A0=C2=A0 trap 'exit 1' INT HUP QUIT TERM ALRM USR1 > > > -=C2=A0=C2=A0=C2=A0 trap 'schroot_cleanup' EXIT > > > - > > > -=C2=A0=C2=A0=C2=A0 schroot -r -c ${session_id} -d / -u root -- \ > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rm /etc/apt/sources.list.= d/isar-apt.list > > > /etc/apt/preferences.d/isar-apt > > > -=C2=A0=C2=A0=C2=A0 schroot -r -c ${session_id} -d / -- \ > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sh -c ' > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s= et -e > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 f= or uri in $2; do > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 mkdir -p /downloads/deb-src/"$1"/${uri} > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 cd /downloads/deb-src/"$1"/${uri} > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 apt-get -y --download-only --only-source source > > > ${uri} > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d= one' \ > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 my_script "${BASE_DISTRO}- > > > ${BASE_DISTRO_CODENAME}" > > > "${SRC_APT}" > > > - > > > -=C2=A0=C2=A0=C2=A0 schroot -e -c ${session_id} > > > -=C2=A0=C2=A0=C2=A0 schroot_delete_configs > > > -} > > > - > > > -addtask apt_fetch > > > -do_apt_fetch[lockfiles] +=3D "${REPO_ISAR_DIR}/isar.lock" > > > -do_apt_fetch[network] =3D "${TASK_USE_NETWORK_AND_SUDO}" > > > - > > > -# Add dependency from the correct schroot: host or target > > > -do_apt_fetch[depends] +=3D "${SCHROOT_DEP}" > > > - > > > -do_apt_unpack() { > > > -=C2=A0=C2=A0=C2=A0 rm -rf ${S} > > > -=C2=A0=C2=A0=C2=A0 schroot_create_configs > > > - > > > -=C2=A0=C2=A0=C2=A0 session_id=3D$(schroot -q -b -c ${SBUILD_CHROOT}) > > > -=C2=A0=C2=A0=C2=A0 echo "Started session: ${session_id}" > > > - > > > -=C2=A0=C2=A0=C2=A0 schroot_cleanup() { > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 schroot -q -f -e -c ${ses= sion_id} > /dev/null 2>&1 > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 schroot_delete_configs > > > -=C2=A0=C2=A0=C2=A0 } > > > -=C2=A0=C2=A0=C2=A0 trap 'exit 1' INT HUP QUIT TERM ALRM USR1 > > > -=C2=A0=C2=A0=C2=A0 trap 'schroot_cleanup' EXIT > > > - > > > -=C2=A0=C2=A0=C2=A0 schroot -r -c ${session_id} -d / -u root -- \ > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rm /etc/apt/sources.list.= d/isar-apt.list > > > /etc/apt/preferences.d/isar-apt > > > -=C2=A0=C2=A0=C2=A0 schroot -r -c ${session_id} -d / -- \ > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sh -c ' > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s= et -e > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 f= or uri in $2; do > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 dscfile=3D"$(apt-get -y -qq --print-uris --only- > > > source > > > source $uri | cut -d " " -f2 | grep -E "*.dsc")" > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 cd ${PP} > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 cp /downloads/deb-src/"${1}"/${uri}/* ${PP} > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 dpkg-source -x "${dscfile}" "${PPS}" > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 d= one' \ > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 my_script "${BASE_DISTRO}- > > > ${BASE_DISTRO_CODENAME}" > > > "${SRC_APT}" > > > - > > > -=C2=A0=C2=A0=C2=A0 schroot -e -c ${session_id} > > > -=C2=A0=C2=A0=C2=A0 schroot_delete_configs > > > -} > > > -do_apt_unpack[network] =3D "${TASK_USE_SUDO}" > > > - > > > -addtask apt_unpack after do_apt_fetch > > > - > > > -do_cleanall_apt[nostamp] =3D "1" > > > -do_cleanall_apt() { > > > -=C2=A0=C2=A0=C2=A0 for uri in "${SRC_APT}"; do > > > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rm -rf "${DEBSRCDIR}/${BA= SE_DISTRO}- > > > ${BASE_DISTRO_CODENAME}/$uri" > > > -=C2=A0=C2=A0=C2=A0 done > > > -} > > > - > > > =C2=A0def get_package_srcdir(d): > > > =C2=A0=C2=A0=C2=A0=C2=A0 s =3D os.path.abspath(d.getVar("S")) > > > =C2=A0=C2=A0=C2=A0=C2=A0 workdir =3D os.path.abspath(d.getVar("WORKDI= R")) > > > diff --git a/meta/lib/aptsrc_fetcher.py > > > b/meta/lib/aptsrc_fetcher.py > > > new file mode 100644 > > > index 00000000..ee726202 > > > --- /dev/null > > > +++ b/meta/lib/aptsrc_fetcher.py > > > @@ -0,0 +1,93 @@ > > > +# This software is a part of ISAR. > > > +# Copyright (c) Siemens AG, 2024 > > > +# > > > +# SPDX-License-Identifier: MIT > > > + > > > +from bb.fetch2 import FetchError > > > +from bb.fetch2 import FetchMethod > > > +from bb.fetch2 import logger > > > +from bb.fetch2 import runfetchcmd > > > + > > > +class AptSrc(FetchMethod): > > > +=C2=A0=C2=A0=C2=A0 def supports(self, ud, d): > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return ud.type in ['apt'] > > > + > > > +=C2=A0=C2=A0=C2=A0 def urldata_init(self, ud, d): > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ud.src_package =3D ud.url= [len('apt://'):] > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ud.host =3D ud.host.repla= ce('=3D', '_') > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 base_distro =3D d.getVar(= 'BASE_DISTRO') > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # For these distros we kn= ow that the same version means > > > the > > > same > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # source package, also ac= ross distro releases. > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 distro_suffix =3D '' if b= ase_distro in ['debian', > > > 'ubuntu'] > > > else \ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 '= -' + d.getVar('BASE_DISTRO_CODENAME') > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ud.localfile=3D'deb-src/'= + base_distro + distro_suffix + > > > '/' > > > + ud.host > > > + > > > +=C2=A0=C2=A0=C2=A0 def download(self, ud, d): > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bb.utils.exec_flat_python= _func('isar_export_proxies', d) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bb.build.exec_func('schro= ot_create_configs', d) > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sbuild_chroot =3D d.getVa= r('SBUILD_CHROOT') > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 session_id =3D runfetchcm= d(f'schroot -q -b -c > > > {sbuild_chroot}', d).strip() > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 logger.info(f'Started ses= sion: {session_id}') > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 repo_isar_dir =3D d.getVa= r('REPO_ISAR_DIR') > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lockfile =3D > > > bb.utils.lockfile(f'{repo_isar_dir}/isar.lock') > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 try: > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r= unfetchcmd(f''' > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 set -e > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 schroot -r -c {session_id} -d / -u root -- \ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rm /etc/apt/sources.list.d= /isar-apt.list > > > /etc/apt/preferences.d/isar-apt > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 schroot -r -c {session_id} -d / -- \ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sh -c ' > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 se= t -e > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mk= dir -p /downloads/{ud.localfile} > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cd= /downloads/{ud.localfile} > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ap= t-get -y --download-only --only-source > > > source {ud.src_package} > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ' > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 ''', d) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 except (OSError, FetchErr= or): > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r= aise > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 finally: > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 b= b.utils.unlockfile(lockfile) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r= unfetchcmd(f'schroot -q -f -e -c {session_id}', d) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 b= b.build.exec_func('schroot_delete_configs', d) > > > + > > > +=C2=A0=C2=A0=C2=A0 def unpack(self, ud, rootdir, d): > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bb.build.exec_func('schro= ot_create_configs', d) > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sbuild_chroot =3D d.getVa= r('SBUILD_CHROOT') > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 session_id =3D runfetchcm= d(f'schroot -q -b -c > > > {sbuild_chroot}', d).strip() > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 logger.info(f'Started ses= sion: {session_id}') > > > + > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pp =3D d.getVar('PP') > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pps =3D d.getVar('PPS') > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 try: > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r= unfetchcmd(f''' > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 set -e > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 schroot -r -c {session_id} -d / -u root -- \ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rm /etc/apt/sources.list.d= /isar-apt.list > > > /etc/apt/preferences.d/isar-apt > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 schroot -r -c {session_id} -d / -- \ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sh -c ' > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 se= t -e > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ds= cfile=3D$(apt-get -y -qq --print-uris -- > > > only-source source {ud.src_package} | \ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cut -d " " -f2 | = grep -E > > > "\.dsc") > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cp= /downloads/{ud.localfile}/* {pp} > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cd= {pp} > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mv= -f {pps} {pps}.prev > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dp= kg-source -x "$dscfile" {pps} > >=20 > > Hello. > >=20 > > This still fails in CI, but this time I had some time to find the > > root > > cause. > >=20 > > The problem is that buster(bullseye) and bookworm(trixie) provide > > different versions of "hello" package. > >=20 > > If we first build e.g. `mc:qemuamd64-bookworm:hello`, hello_2.10- > > 3.dsc > > is downloaded and the whole "downloads/deb-src/debian/hello/" is > > considered finished with "downloads/deb-src/debian/hello.done" > > flag.=C2=A0 > >=20 > > So, when e.g. `mc:qemuamd64-bullseye:hello` build follows, it > > doesn't > > download hello_2.10-2.dsc an results in dpkg-source error. > >=20 > > It doesn't matter if we build both targets in parallel or > > sequentially, > > the latest always fails. > >=20 >=20 > Thanks for the analysis. I'll check if I can reproduce und understand > to > root cause. >=20 > Jan >=20 The easy way to reproduce: ./kas/kas-container menu # select e.g. qemuamd64-bookworm, save & exit ./kas/kas-container shell -c 'bitbake hello' ./kas/kas-container menu # select e.g. qemuamd64-bullseye, save & exit ./kas/kas-container shell -c 'bitbake hello' --=20 Best regards, Uladzimir. --=20 You received this message because you are subscribed to the Google Groups "= isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/= d656bc9d61be9adf3181ccbf333dcf8829675b5f.camel%40ilbers.de.