From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7197029299818332160 X-Received: by 2002:a67:fe81:0:b0:411:bcfa:6666 with SMTP id b1-20020a67fe81000000b00411bcfa6666mr1986795vsr.82.1676035656449; Fri, 10 Feb 2023 05:27:36 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac5:c776:0:b0:3e1:cb2e:e15d with SMTP id c22-20020ac5c776000000b003e1cb2ee15dls1013975vkn.1.-pod-prod-gmail; Fri, 10 Feb 2023 05:27:35 -0800 (PST) X-Google-Smtp-Source: AK7set/RvhuHhf7Kx/nLNEgMpCKLQw1ZkyUFWwQ5Bvx+eGgce4E8aDrM0FNgaXAW3Lf+3eGsd1Tm X-Received: by 2002:a05:6122:793:b0:3ea:3403:9d23 with SMTP id k19-20020a056122079300b003ea34039d23mr11759465vkr.6.1676035655443; Fri, 10 Feb 2023 05:27:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676035655; cv=none; d=google.com; s=arc-20160816; b=sAejbgom9zZule/MCdcasawgCsRrGFy7X9HpuScPTWyav6dwG011msjOaNnL7KR0pX bEGRuti6/mpg9bkQdp2whe1LJeK8Y/9Z96FQPUARsD2zaFygVBFahoFz93KgBTkPjjBl LxDRelIdII9JKfL6W3WaUdk9+oyzdlSA5VB/D/v9LwU/hHPfJVuVrXiN3gB1umrB9X1u YUDn6RpzCy8NZkgSFGeX+oE7gVMyhNg1av/e7IBsFVt5YaV98YZY/5fmZveUyezZ6e76 vy/TrSlPZPhBneuyeTBRI3a8OYtVNi3/4uS8T9e0XV5OlrTI+sOXANGoTGYOw+bOcMtE G8OA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=lrj4GRPg09pM9vVNyH/c9gcd3+HCBQK1tU9aLtSsaGE=; b=a/9yFxCzmnKr3jQebthBPSqGSl7yDKUJMnBZONd0fzQDqlfWuOkDRy6SpBx+gNn9Fk qM3D78/IhOrIJPqmRKgn9QtMC8BeDbNvcqgkk7pO+KAOVma0mqWLruZ8tq7SKwIKhOfD LhJdFmlR+LpNDmzgaO+/d9lsOCl5Fs9/xe1kjoF7O3H4G9w5m3LMfoWdnjfB3dIdv4fn g0WolcdVfdX/YEOimArYC8fRtuJ80wshOq2hxzsdCbVyjH83CaLLKYPgLyvYxVgELbPU VBY/3cqqjOX5H3d0AUaLN5e/GTstAO9TniurAcVb2iNWMcc3nhevrXNnwQSX+DtZKzcV jMgw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id m4-20020a1f3f04000000b003ea4c59a0c7si420029vka.2.2023.02.10.05.27.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 10 Feb 2023 05:27:35 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 31ADRWMO024355 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 10 Feb 2023 14:27:33 +0100 From: Uladzimir Bely To: Henning Schild Cc: isar-users@googlegroups.com, roberto.foglietta@linuxteam.org Subject: Re: [PATCH v4] deb-dl-dir class rework to use faster ln -P or fallback to cp Date: Fri, 10 Feb 2023 16:27:30 +0300 Message-ID: <9866221.tdPhlSkOF2@home> In-Reply-To: <20230210134803.00fe46b3@md1za8fc.ad001.siemens.net> References: <20230210110908.1507520-1-roberto.foglietta@linuxteam.org> <20230210134803.00fe46b3@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: ZPxHdCH9oC+M In the email from Friday, 10 February 2023 15:48:03 +03 user Henning Schild= =20 wrote: > Am Fri, 10 Feb 2023 12:09:08 +0100 >=20 > schrieb roberto.foglietta@linuxteam.org: > > From: "Roberto A. Foglietta" > >=20 > > deb-dl-dir, feature: faster when using ln -P otherwise fallback to cp > >=20 > > The original class functions deb_dl_dir_import/export were using cp to > > copy debian package to the target rootfs but this approach is quite > > slow while using hard link does not work if the destination and > > source dirs are not lying on the same filesystem. Thus, ln -P should > > fallback to cp when it does not work (which is different from > > complaining on stderr). > >=20 > > Moreover, these two functions have been reworked to reach a straight > > forward and more compact form. In particular, export function was > > using bashism to do some kind of comparison which after all is useless > > because copying back without overwriting just fulfills that part. > >=20 > > More rework using sudo in a different way plus a corner case > > addressingi, in case the spia file exists for some other reasons. > >=20 > > Rebased on the current next and bugfix about checking the destination > > not the source file. > >=20 > > Signed-off-by: Roberto A. Foglietta > > --- > >=20 > > meta/classes/deb-dl-dir.bbclass | 53 > >=20 > > +++++++++++++++++++-------------- 1 file changed, 30 insertions(+), > > 23 deletions(-) > >=20 > > diff --git a/meta/classes/deb-dl-dir.bbclass > > b/meta/classes/deb-dl-dir.bbclass index 7db25251..2a3c7508 100644 > > --- a/meta/classes/deb-dl-dir.bbclass > > +++ b/meta/classes/deb-dl-dir.bbclass > > @@ -78,39 +78,46 @@ debsrc_download() { > >=20 > > deb_dl_dir_import() { > > =20 > > export pc=3D"${DEBDIR}/${2}" > >=20 > > - export rootfs=3D"${1}" > > - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ > > + export sc=3D"${1}/var/cache/apt/archives/" > > + sudo mkdir -p "${sc}" > >=20 > > [ ! -d "${pc}" ] && return 0 > >=20 > > - flock -s "${pc}".lock -c ' > > + export tf=3D$(cd "${pc}"; ls -1 *.deb | head -n1) > > + [ ! -e "${pc}/${tf}" ] && return 0 > > + flock -Fs "${pc}".lock sudo -Es << 'EOFSUDO' >=20 > Just curious ... what would EOFSUDO stand for? We have EOSUDO in > many (all?) other places and every author can bring in their own style. >=20 > But EndOfSudo is clear to me ... EndOfFileSudo seems weird. >=20 > But maybe whoever used EOSUDO first meant something totally different > with the acronym. And it does not mean EndOfSudo ... >=20 > Henning >=20 I'm also playing with this part of code and found one more funny thing (at= =20 least, I didn't know about it before). I'm saying about taking EOSUDO into quotes. The behaviour is different with= =20 and without quotes: =46or example, this code will print "/" contents: sudo -Es << "EOSUDO" ls -1 / | while read p; do echo $p done =E2=80=8BEOSUDO But this one won't print it: sudo -Es << EOSUDO ls -1 / | while read p; do echo $p done =E2=80=8BEOSUDO Just few empty lines printed. We appear to have to backslash \$p... But it= =20 works well with exported (before) variables... > > set -e > > printenv | grep -q BB_VERBOSE_LOGS && set -x > >=20 > > - sudo find "${pc}" -type f -iname "*\.deb" -exec \ > > - ln -Pf -t "${rootfs}"/var/cache/apt/archives/ {} + > > - ' > > + rm -f "${sc}/${tf}" > > + ln -Pf -t "${sc}" "${pc}/${tf}" 2>/dev/null ||: > > + if [ -r "${sc}/${tf}" ]; then > > + find "${pc}" -type f -iname "*\.deb" -exec \ > > + ln -Pf -t "${sc}" {} + > > + else > > + find "${pc}" -type f -iname "*\.deb" -exec \ > > + cp -np owner --reflink=3Dauto -t "${sc}" {} + > > + fi > > +EOFSUDO > >=20 > > } > > =20 > > deb_dl_dir_export() { > > =20 > > export pc=3D"${DEBDIR}/${2}" > >=20 > > - export rootfs=3D"${1}" > > + export sc=3D"${1}/var/cache/apt/archives/" > >=20 > > mkdir -p "${pc}" > >=20 > > - flock "${pc}".lock -c ' > > + export tf=3D$(cd "${sc}"; ls -1 *.deb | head -n1) > > + [ ! -e "${sc}/${tf}" ] && return 0 > > + flock -F "${pc}".lock sudo -Es << 'EOFSUDO' > >=20 > > set -e > > printenv | grep -q BB_VERBOSE_LOGS && set -x > >=20 > > - find "${rootfs}"/var/cache/apt/archives/ \ > > - -maxdepth 1 -type f -iname '*\.deb' |\ > > - while read p; do > > - # skip files from a previous export > > - [ -f "${pc}/${p##*/}" ] && continue > > - # can not reuse bitbake function here, this is basically > > - # "repo_contains_package" > > - package=3D$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name > > ${p##*/}) > > - if [ -n "$package" ]; then > > - cmp --silent "$package" "$p" && continue > > - fi > > - sudo ln -Pf "${p}" "${pc}" > > - done > > - sudo chown -R $(id -u):$(id -g) "${pc}" > > - ' > > + rm -f "${pc}/${tf}" > > + ln -Pf -t "${pc}" "${sc}/${tf}" 2>/dev/null ||: > > + if [ -r "${pc}/${tf}" ]; then > > + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ > > + -exec ln -P -t "${pc}" {} + 2>/dev/null ||: > > + else > > + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ > > + -exec cp -n --reflink=3Dauto -t "${pc}" {} + > > + fi > > + chown -R $(id -u):$(id -g) "${pc}" 2 Roberto: This also won't work as expected. Since sudo moved to the upper= =20 level, owner is 0:0 now instead of actual user:group, as it was before. I'm now testing v3 of my version of the fix. It works on per-file basis and= =20 seems to have passed all my local thorough tests... at last. > > +EOFSUDO > >=20 > > }