From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7291284813103759360 X-Received: by 2002:a05:600c:35ce:b0:40c:6593:627e with SMTP id r14-20020a05600c35ce00b0040c6593627emr585776wmq.161.1703238194729; Fri, 22 Dec 2023 01:43:14 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:468e:b0:40d:2bc7:eb4 with SMTP id p14-20020a05600c468e00b0040d2bc70eb4ls362998wmo.1.-pod-prod-06-eu; Fri, 22 Dec 2023 01:43:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IEzB73l5PwRskoY0KbTU+q7OVhbLGdyrSDYPwHRd6+oeuFWP/ksJB83FZK2Y9Tb/Bd2gc4W X-Received: by 2002:a05:600c:6b0d:b0:40d:4a58:f8d1 with SMTP id jn13-20020a05600c6b0d00b0040d4a58f8d1mr48081wmb.172.1703238192642; Fri, 22 Dec 2023 01:43:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703238192; cv=none; d=google.com; s=arc-20160816; b=iMJh+CnXWsIcl2noDFsEOxznAdiv416mCfpPggWrNx2MMAXxKKpdSa84U9Qz9Uns6S D722q6K07wU8fuKB76ZwGgwr7KF17rbboDCJdd9+6jXPNMj6fzhi2h7yCf5hiMarmlHy gAlx4oS6ftL+f/Ehi8K0r7osOVEYvwfV9fNd7wLnKUZEZgHe6bHQizRO2iNcjdc7Z8Mi +/zgp4VWTGf2hSGdt2fP1Q+Ym13XtxdBR5S7yjwjrL7vtRVDXQPe+bryyUrRfQWz5xsH rGa2K6mkHLmUH/HMM3Vf6RmyT/6hZ87hQp7H5qu+NEW8qmNUqcedPkbwREnTkcbR6tfx Mk9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id; bh=Wl2Ptt3taHty1AnsDq13G/cClWU3Or5c47Ga6EFDyCA=; fh=IxbeHM4vWxalf1XNf2rnGcQL+r1v0Lr0C6ICsGNNkYw=; b=BOi4F03oMGsdFwwYww2R1swHRp1CIdY9Vh5PYkAol0Unu4G4O4xRNsIql6diwDlIfB 7/PvhdsKm9z41Wz6djpqlLjStL2UwU7i03BtNZaX0IWu71jmOkchtHuKsp6ZoELznNpS l94tQrenAdVZvFNiEm5fih0AMWcHMKTbzSTqQF9SIhaH8OIfUDohzlJbf0dZxJ1BN8UU iWF7KNP0OhLLhkS2aKBvHvkJllP/QztjH8rmyU4ATubc/sIFoc/iFBADAOFh6wD1sQgQ CyispHIHU1ZhLGvMXI4hRnD5S5WHAHp7REz1b8d1i2yEcuQNzxHQJKk9p54tn4/e8ftK az9g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id m30-20020a05600c3b1e00b0040c69a269fesi150863wms.2.2023.12.22.01.43.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 22 Dec 2023 01:43:12 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 3BM9hAqD005294 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Dec 2023 10:43:11 +0100 Message-ID: <1328db3c77765c8b91e48db94cc23eb7bbf74a8a.camel@ilbers.de> Subject: Re: [PATCH v2] dpkg: Account for empty local apt cache after package build From: Uladzimir Bely To: Jan Kiszka , isar-users Date: Fri, 22 Dec 2023 12:43:10 +0300 In-Reply-To: <47a6c8a9-cbb5-4ef2-ba47-c7a320f9da6e@siemens.com> References: <47a6c8a9-cbb5-4ef2-ba47-c7a320f9da6e@siemens.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.1 (by Flathub.org) MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: ZEthtK2lp2vP On Thu, 2023-10-19 at 06:39 +0200, 'Jan Kiszka' via isar-users wrote: > From: Jan Kiszka >=20 > A build dependency-free package combined with sstate caching and > cleared > deb cache can trigger >=20 > > cp -Ln --no-preserve=3Downer /var/cache/apt/archives/*.deb -t > > /home/builder/example-raw/rootfs/var/cache/apt/archives/ > > ------------------------------------------------------------------- > > ------------------------------------------------- > >=20 > > cp: cannot stat '/var/cache/apt/archives/*.deb': No such file or > > directory > >=20 > > E: Command 'cp -Ln --no-preserve=3Downer > > /var/cache/apt/archives/*.deb -t /home/builder/example- > > raw/rootfs/var/cache/apt/archives/' failed to run. >=20 > Account for that. >=20 > Signed-off-by: Jan Kiszka > --- >=20 > Changes in v2: > =C2=A0- avoid returning error via negativ [ -n ] result >=20 > =C2=A0meta/classes/dpkg.bbclass | 2 +- > =C2=A01 file changed, 1 insertion(+), 1 deletion(-) >=20 > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > index d89b0fd9..d61e9377 100644 > --- a/meta/classes/dpkg.bbclass > +++ b/meta/classes/dpkg.bbclass > @@ -109,7 +109,7 @@ dpkg_runbuild() { > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --chroot-setup-commands= =3D"mkdir -p ${deb_dir}" \ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --chroot-setup-commands= =3D"ln -sf ${ext_deb_dir}/*.deb -t > ${deb_dir}/" \ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --finished-build-command= s=3D"rm -f ${deb_dir}/sbuild-build- > depends-main-dummy_*.deb" \ > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --finished-build-commands=3D"= cp -Ln --no-preserve=3Downer > ${deb_dir}/*.deb -t ${ext_deb_dir}/" \ > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --finished-build-commands=3D"= [ -z "$(find ${deb_dir} -maxdepth > 1 -name '*.deb' -print -quit)" ] || cp -Ln --no-preserve=3Downer > ${deb_dir}/*.deb -t ${ext_deb_dir}/" \ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --finished-build-command= s=3D"cp /var/log/dpkg.log > ${ext_root}/dpkg_partial.log" \ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --debbuildopts=3D"--sour= ce-option=3D-I" \ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 --build-dir=3D${WORKDIR}= --dist=3D"isar" ${DSC_FILE} > --=20 > 2.35.3 >=20 When experimenting with base-apt, I found that this patch causes build issues on second (cached) build. Debug shows that new logic prevents exporting packages downloaded by sbuild from internal layer to external directory. It's reproducible for me when I use kas-container. ``` isar $./kas/kas-container menu #Just save default config here isar $ ./kas/kas-container shell builder@6026019cd1fb:/build$ bitbake -v hello ... I: Finished running 'rm -f /var/cache/apt/archives/sbuild-build- depends-main-dummy_*.deb'. [ -z ] || cp -Ln --no-preserve=3Downer /var/cache/apt/archives/*.deb -t /home/builder/hello/rootfs/var/cache/apt/archives/ I: Finished running '[ -z ] || cp -Ln --no-preserve=3Downer /var/cache/apt/archives/*.deb -t /home/builder/hello/rootfs/var/cache/apt/archives/'. ... ``` The problem is in empty condition `[ -z ]`. I added some "--finished-build-commands" before existing lines for debugging: ``` --finished-build-commands=3D"echo =3D-=3D-=3D" \ --finished-build-commands=3D"find ${deb_dir} -maxdepth 1 -name '*.deb' -print -quit" \ --finished-build-commands=3D"echo $(find ${deb_dir} -maxdepth 1 - name '*.deb' -print -quit)" \ ``` And that's what I see: ``` +---------------------------------------------------------------------- --------+ | Finished Timed Build Commands =20 | +---------------------------------------------------------------------- --------+ echo =3D-=3D-=3D ---------- =3D-=3D-=3D I: Finished running 'echo =3D-=3D-=3D'. find /var/cache/apt/archives -maxdepth 1 -name '*.deb' -print -quit ------------------------------------------------------------------- /var/cache/apt/archives/libgcc-s1_12.2.0-14_amd64.deb I: Finished running 'find /var/cache/apt/archives -maxdepth 1 -name '*.deb' -print -quit'. echo=20 ----- I: Finished running 'echo '. rm -f /var/cache/apt/archives/sbuild-build-depends-main-dummy_*.deb ------------------------------------------------------------------- I: Finished running 'rm -f /var/cache/apt/archives/sbuild-build- depends-main-dummy_*.deb'. [ -z ] || cp -Ln --no-preserve=3Downer /var/cache/apt/archives/*.deb -t /home/builder/hello/rootfs/var/cache/apt/archives/ ----------------------------------------------------------------------- ------------------------------------------------- I: Finished running '[ -z ] || cp -Ln --no-preserve=3Downer /var/cache/apt/archives/*.deb -t /home/builder/hello/rootfs/var/cache/apt/archives/'. ``` E.g., "find" itself works (libgcc-s1_12.2.0-14_amd64.deb is found), but in form of $(find ) output becomes empty for some reasons, and no packages are exported. The issue is not reproducable in simple bullseye chroot (that's why it was not caught by CI), but it's reproducable under kas container. Even if I enter kas-container shell and then manually clone isar inside it in order to try common (`. isar-init-build-env`) way of building isar.