From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7197029299818332160 X-Received: by 2002:a17:902:7c8c:b0:193:648:821d with SMTP id y12-20020a1709027c8c00b001930648821dmr451833pll.10.1675751565900; Mon, 06 Feb 2023 22:32:45 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:3e01:b0:230:c6b0:bf8c with SMTP id j1-20020a17090a3e0100b00230c6b0bf8cls5302397pjc.0.-pod-control-gmail; Mon, 06 Feb 2023 22:32:45 -0800 (PST) X-Google-Smtp-Source: AK7set/g7UhoCmjRXxskxZXXOMz/IaJvZ+fcxmRkmGykAskOqMFjj+TaJYYGrwKHCcMRbPJM0mCw X-Received: by 2002:a05:6a21:3386:b0:bc:3523:13c6 with SMTP id yy6-20020a056a21338600b000bc352313c6mr2741039pzb.29.1675751564914; Mon, 06 Feb 2023 22:32:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675751564; cv=none; d=google.com; s=arc-20160816; b=kL4EZvv60unu/sM/5qAIlhCA4bXiLfOfNQpetkAkrheRQaSEH3Eg309l1Qj3jlYb82 kFvyuC1dJZblb3C/V50gZ1FjtxswpADbRdtkkn679bbIWBBIpq2EEaqtqkVwNzA/gYiJ twAtYL4SpGH/c95PYbCgUYhJ3RwPLAKSPCnrbUq7mMsUFopq37i3F8wvCevcqqnlOnOI Iy/4IjbqQxkMf3zkWe+EejgpW35qwKgXgUk6VKqAZjSG0xh9Oc8w1qNVvmZ0u2buptIJ xAl58z8vwxD6UR61KAceE69v1FgkKsDcfRUYoZ5Nm/l9+iJDY31ixC02HCe1h5dTecsm gwGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=Wf6gTh4bJMVD3YpFp/CcBTaY9vai3SfBtpbRg7WKPNc=; b=yhJJj6Obl+9/EQgyb5VLZsfLjmhilSK1M9kzCAYszwDYevjjbK8gLqMkQEwkQKNUw9 4KlAe+o10m0J7orfiJvkJM8N+zsvL3r9fRjdWdHshLMMMBSMnh78lcMJyl1i3y1aQ0w5 Jr5shBm5eJDS65vxSD6iGYP3krqrwFgJoONcb583TwLVSlEPI3Ox+pK1eQxHzDSYjBWW 4SuGLs3xFaMBui5KSHWjSOp7g9KkCiv/g05Mez5wwNzhJjL70nGBt9tHbf+RsLhVwvbd ODb3QcJKa0Knjm9b4BsoBJO5b/8Iyw1vukFY3yzMVgTpO7vUpLIb95AYz1dQt/TYe/H1 lhrg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id x200-20020a6331d1000000b004ac6ba951f1si1000218pgx.2.2023.02.06.22.32.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 06 Feb 2023 22:32:44 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 3176Wfe5009060 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 7 Feb 2023 07:32:42 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: Re: [PATCH] deb-dl-dir class rework to use faster ln -P or fallback to cp Date: Tue, 07 Feb 2023 09:32:38 +0300 Message-ID: <6850064.9J7NaK4W3v@home> In-Reply-To: <20230206130919.327653-1-roberto.foglietta@linuxteam.org> References: <20230206130919.327653-1-roberto.foglietta@linuxteam.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 0yJMO0OidJ29 In the email from =D0=BF=D0=BE=D0=BD=D0=B5=D0=B4=D0=B5=D0=BB=D1=8C=D0=BD=D0= =B8=D0=BA, 6 =D1=84=D0=B5=D0=B2=D1=80=D0=B0=D0=BB=D1=8F 2023 =D0=B3. 16:09:= 19 +03 user=20 roberto.foglietta@linuxteam.org wrote: > From: "Roberto A. Foglietta" >=20 > deb-dl-dir, feature: faster when using ln -P otherwise fallback to cp >=20 > The original class functions deb_dl_dir_import/export were using cp to > copy debian package to the target rootfs but this approach is quite slow > while using hard link does not work if the destination and source dirs > are not lying on the same filesystem. Thus, ln -P should fallback to cp > when it does not work (which is different from complaining on stderr). >=20 > Moreover, these two functions have been reworked to reach a straight > forward and more compact form. In particular, export function was using > bashism to do some kind of comparison which after all is useless > because copying back without overwriting just fulfills that part. >=20 > Signed-off-by: Roberto A. Foglietta > --- > meta/classes/deb-dl-dir.bbclass | 41 ++++++++++++++++++--------------- > 1 file changed, 23 insertions(+), 18 deletions(-) >=20 > diff --git a/meta/classes/deb-dl-dir.bbclass > b/meta/classes/deb-dl-dir.bbclass index 3b1517dc..22de7bf6 100644 > --- a/meta/classes/deb-dl-dir.bbclass > +++ b/meta/classes/deb-dl-dir.bbclass > @@ -78,39 +78,44 @@ debsrc_download() { >=20 > deb_dl_dir_import() { > export pc=3D"${DEBDIR}/${2}" > - export rootfs=3D"${1}" > - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ > + export sc=3D"${1}/var/cache/apt/archives/" > + sudo mkdir -p "${sc}" > [ ! -d "${pc}" ] && return 0 > + export tf=3D$(ls -1 "${pc}"/*.deb | head -n1) > + [ ! -e "${tf}" ] && return 0 > flock -s "${pc}".lock -c ' > set -e > printenv | grep -q BB_VERBOSE_LOGS && set -x >=20 > - sudo find "${pc}" -type f -iname "*\.deb" -exec \ > - cp -n --no-preserve=3Downer -t > "${rootfs}"/var/cache/apt/archives/ {} + + sudo ln -Pf -t "${sc}" > "${tf}" 2>/dev/null ||: I'm not sure that assumption "if one hardlink failed, other also will fail"= =20 (and vice versa) would always be true. The solution in "[PATCH v2] deb-dl-dir: Fallback to copying when import- export" looks simpler and works in per-file basis. > + if [ -r "${tf}" ]; then > + sudo find "${pc}" -type f -iname "*\.deb" -exec \ > + ln -Pf -t "${sc}" {} + > + else > + sudo find "${pc}" -type f -iname "*\.deb" -exec \ > + cp -np owner --reflink=3Dauto -t "${sc}" {} + > + fi > ' > } >=20 > deb_dl_dir_export() { > export pc=3D"${DEBDIR}/${2}" > - export rootfs=3D"${1}" > + export sc=3D"${1}/var/cache/apt/archives/" > mkdir -p "${pc}" > + export tf=3D$(ls -1 "${sc}"/*.deb | head -n1) > + [ ! -e "${tf}" ] && return 0 > flock "${pc}".lock -c ' > set -e > printenv | grep -q BB_VERBOSE_LOGS && set -x >=20 > - find "${rootfs}"/var/cache/apt/archives/ \ > - -maxdepth 1 -type f -iname '*\.deb' |\ > - while read p; do > - # skip files from a previous export > - [ -f "${pc}/${p##*/}" ] && continue > - # can not reuse bitbake function here, this is basically > - # "repo_contains_package" > - package=3D$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name ${p##*= /}) > - if [ -n "$package" ]; then > - cmp --silent "$package" "$p" && continue > - fi > - sudo cp -n "${p}" "${pc}" > - done > + sudo ln -Pf -t "${pc}" "${tf}" 2>/dev/null ||: > + if [ -r "${tf}" ]; then > + sudo find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ > + -exec ln -P -t "${pc}" {} + 2>/dev/null ||: > + else > + sudo find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ > + -exec cp -n --reflink=3Dauto -t "${pc}" {} + > + fi > sudo chown -R $(id -u):$(id -g) "${pc}" > ' > }