From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7197029299818332160 X-Received: by 2002:a05:651c:2324:b0:292:b759:af85 with SMTP id bi36-20020a05651c232400b00292b759af85mr280743ljb.81.1675755469925; Mon, 06 Feb 2023 23:37:49 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3d1f:b0:4d1:8575:2d31 with SMTP id d31-20020a0565123d1f00b004d185752d31ls2281516lfv.0.-pod-prod-gmail; Mon, 06 Feb 2023 23:37:48 -0800 (PST) X-Google-Smtp-Source: AK7set9eqA3yrxP6vyuPss5CvD/OVoUEhYvoW23iJW1fsWVREMTZxolC/IQ9jYfpn9AmYq3iQYCN X-Received: by 2002:ac2:5e8d:0:b0:4b5:aa85:3874 with SMTP id b13-20020ac25e8d000000b004b5aa853874mr545077lfq.68.1675755468199; Mon, 06 Feb 2023 23:37:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675755468; cv=none; d=google.com; s=arc-20160816; b=RwrRDA5sOcpaX6EHcM6JyEyDHg7RctND96SPNS9P6VZkqR0qTrAO9IoRuRfG5fVj8H gOYluwqsKNRgnmvBMQ5jIx8KdxtrFKCjIPuMGwZxyPpOfg+0HYNKYulDFuwSR5gWPYd8 9O9Ua8Nii7f8cc+FzYDtG7lwA6SIkO1DACZOhGKT2gbkNJevfQZREWvypjOYGXG8Yw8E 7qUglmlOTQ9RWoybgYCkBwQrpx9zKuEDjFFwdBnnr3I7umEBvrWdfZ74WAfYFGCHH5Iu veJf7z0C2dZTH2Q6aAwh1sGrO0XRrSJ5BsfxN8RP3AnllG+XG2u+WgMP1CFrjBeFdHOZ hXVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=ZDiAA8F8XVGqDzTZubEttHLM/8H5GUNzWaFEH756/l8=; b=WygCUW1wKo350ycOYCO3NG0OotKeI3JX6qe0hP9elL5iyJoFbXn3GZwk7ytcYNGeF1 IdEEcLMCRzyzbZd8U9D8UJPS7h2lE/bfzmTJeEVc0qdEtYG1qFFDMLhEiPHU4SCVGO91 746RNP1Xv4E6w3Ude+zU3fo59V3xxJoexzdSXakvcyJmn2isorcAnIwbn19Bgfy5OfdM t2sFBqFkv4DIDJIm4XTDeTBBcV8KoDntr+lCV8N596VoHL6Bz/uH+7LyQQ4bKm32q1fI jqYjhWosB2jegprTu8k//Hu/VWOoPUz8/aNfIQKaI/nGZyUsOxg6NKGu3XzF/bkFzDXy ZRKA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 2001:4b7a:2000:18::169 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Return-Path: Received: from relay08.th.seeweb.it (relay08.th.seeweb.it. [2001:4b7a:2000:18::169]) by gmr-mx.google.com with ESMTPS id k31-20020a0565123d9f00b004cb0f0982f3si605756lfv.4.2023.02.06.23.37.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Feb 2023 23:37:47 -0800 (PST) Received-SPF: pass (google.com: domain of roberto.foglietta@linuxteam.org designates 2001:4b7a:2000:18::169 as permitted sender) client-ip=2001:4b7a:2000:18::169; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 2001:4b7a:2000:18::169 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Received: from localhost.localdomain (unknown [IPv6:2a02:8071:3187:7b80:fccb:83:3c8e:9265]) by m-r2.th.seeweb.it (Postfix) with ESMTPA id 9C7413F1EB; Tue, 7 Feb 2023 08:37:45 +0100 (CET) From: roberto.foglietta@linuxteam.org To: isar-users@googlegroups.com Cc: roberto.foglietta@gmail.com Subject: [PATCH v2] deb-dl-dir class rework to use faster ln -P or fallback to cp Date: Tue, 7 Feb 2023 08:37:40 +0100 Message-Id: <20230207073740.389604-1-roberto.foglietta@linuxteam.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: ZFSch/tNzNQc From: "Roberto A. Foglietta" deb-dl-dir, feature: faster when using ln -P otherwise fallback to cp The original class functions deb_dl_dir_import/export were using cp to copy debian package to the target rootfs but this approach is quite slow while using hard link does not work if the destination and source dirs are not lying on the same filesystem. Thus, ln -P should fallback to cp when it does not work (which is different from complaining on stderr). Moreover, these two functions have been reworked to reach a straight forward and more compact form. In particular, export function was using bashism to do some kind of comparison which after all is useless because copying back without overwriting just fulfills that part. More rework using sudo in a different way plus a corner case addressing, in case the spia file exists for some other reasons. Signed-off-by: Roberto A. Foglietta --- meta/classes/deb-dl-dir.bbclass | 53 +++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 3b1517dc..1fe9d40c 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -78,39 +78,46 @@ debsrc_download() { deb_dl_dir_import() { export pc="${DEBDIR}/${2}" - export rootfs="${1}" - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ + export sc="${1}/var/cache/apt/archives/" + sudo mkdir -p "${sc}" [ ! -d "${pc}" ] && return 0 - flock -s "${pc}".lock -c ' + export tf=$(ls -1 "${pc}"/*.deb | head -n1) + [ ! -e "${tf}" ] && return 0 + flock -Fs "${pc}".lock sudo -Es << 'EOFSUDO' set -e printenv | grep -q BB_VERBOSE_LOGS && set -x - sudo find "${pc}" -type f -iname "*\.deb" -exec \ - cp -n --no-preserve=owner -t "${rootfs}"/var/cache/apt/archives/ {} + - ' + rm -f "${sc}/"$(basename "${tf}") + ln -Pf -t "${sc}" "${tf}" 2>/dev/null ||: + if [ -r "${tf}" ]; then + find "${pc}" -type f -iname "*\.deb" -exec \ + ln -Pf -t "${sc}" {} + + else + find "${pc}" -type f -iname "*\.deb" -exec \ + cp -np owner --reflink=auto -t "${sc}" {} + + fi +EOFSUDO } deb_dl_dir_export() { export pc="${DEBDIR}/${2}" - export rootfs="${1}" + export sc="${1}/var/cache/apt/archives/" mkdir -p "${pc}" - flock "${pc}".lock -c ' + export tf=$(ls -1 "${sc}"/*.deb | head -n1) + [ ! -e "${tf}" ] && return 0 + flock -F "${pc}".lock sudo -Es << 'EOFSUDO' set -e printenv | grep -q BB_VERBOSE_LOGS && set -x - find "${rootfs}"/var/cache/apt/archives/ \ - -maxdepth 1 -type f -iname '*\.deb' |\ - while read p; do - # skip files from a previous export - [ -f "${pc}/${p##*/}" ] && continue - # can not reuse bitbake function here, this is basically - # "repo_contains_package" - package=$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name ${p##*/}) - if [ -n "$package" ]; then - cmp --silent "$package" "$p" && continue - fi - sudo cp -n "${p}" "${pc}" - done - sudo chown -R $(id -u):$(id -g) "${pc}" - ' + rm -f "${pc}/"$(basename "${tf}") + ln -Pf -t "${pc}" "${tf}" 2>/dev/null ||: + if [ -r "${tf}" ]; then + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ + -exec ln -P -t "${pc}" {} + 2>/dev/null ||: + else + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ + -exec cp -n --reflink=auto -t "${pc}" {} + + fi + chown -R $(id -u):$(id -g) "${pc}" +EOFSUDO } -- 2.34.1