From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7194774660571987968 X-Received: by 2002:aa7:9842:0:b0:5a8:91d2:7bb with SMTP id n2-20020aa79842000000b005a891d207bbmr677432pfq.53.1676522702178; Wed, 15 Feb 2023 20:45:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6a00:148c:b0:5a8:16ad:51aa with SMTP id v12-20020a056a00148c00b005a816ad51aals233151pfu.11.-pod-prod-gmail; Wed, 15 Feb 2023 20:45:01 -0800 (PST) X-Google-Smtp-Source: AK7set8KeoTBCko8oWJJIEZw5qkVUT3+1pXiJFhBFrK3ny2AvpcRNJ4hET0uYlwGYn6anjKy056H X-Received: by 2002:a62:17c5:0:b0:5a8:aa1d:30e9 with SMTP id 188-20020a6217c5000000b005a8aa1d30e9mr3387619pfx.18.1676522701197; Wed, 15 Feb 2023 20:45:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676522701; cv=none; d=google.com; s=arc-20160816; b=kPSLyUyU1gJIxf8IR9HLdpRJo3vvrubaO3MH2+frtmHmvvXJtDo+3cXmNXJcJSSb6N vmFjx9GKTDwLrnnZ84l0dUJvR/j8Yknb1qLF1fw78ggKqrF9To75q0rzxZSOUcTV5Xje X5HYOEpdy5h+Wx0F/eiHHU/UzHMsYzT2qO0Obb9YGtrOB9D/r0rvv+UztjUVzSiER4M6 FRZxmNTVuchxBbRqeRHKlsLj6KqZJ2ato13Fv3KSQGM3pTSJIUncRfnevQLpWIlq1bz/ 8kRj3cS4tRvkd+CG6NkYrv+0LZVMZq0cvDUJc06cc7QJeOla/m/M1WF+T5Z3D1U1T96h g2XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=uDD9wDn9cJmFR2J0WWV5qDk79bY0a/z/GWz/+9ya17w=; b=P67btTEiHZ9LTmFqXYOy6uYoqvLzyKkkEvGfIFXD3Oc+IxxSKVzKyLfSpAnyOlmEQd kMKF6fpdNBwav6NHTknPsTuVL4mSRgHWyd7oA2NLqrYrdMnqqn2Vr9PjW1T3+9mahW8r NWxFVso3Hj6UMvoDXBL9GJbV6h4i9BS+pw0EM4GqqaAvs8wmwCnSh79giOZna+LiU/x3 frlCTlkEOU8WNO5sW6SHgAyW8mSIXGh3Cky97Ah+u7CSu+nJrjtJY5gytZ5dIbZWNoOj QUrHDofjM7RvtxXVxTxOOGdYDaqdbAW2ygE+dyPL2PWlLR7UACkWMLOgxC02lQIK9hos 1d3A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id y12-20020a62640c000000b00593910fa1d3si23098pfb.6.2023.02.15.20.45.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Feb 2023 20:45:01 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 31G4iwCP016862 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 16 Feb 2023 05:44:58 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: Re: [PATCH v3] deb-dl-dir: Fallback to copying when import-export Date: Thu, 16 Feb 2023 07:44:58 +0300 Message-ID: <9479096.eNJFYEL58v@home> In-Reply-To: <20230213074439.23410-1-ubely@ilbers.de> References: <20230213074439.23410-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: GxRkMIzBQNB4 In the email from Monday, 13 February 2023 10:44:39 +03 user Uladzimir Bely wrote: > By default, hardlinks are used when doing DL_DIR import and export. > This fails if DL_DIR's filesystem differs from TMPDIR's one. > > Fallback to file copying (with lower performance and huge disk usage) > in this case. > > Also rework sudo using when import/export by preferring "sudo/while" > approach instead of "while/sudo" (less spam in system logs). > > Signed-off-by: Uladzimir Bely > --- > meta/classes/deb-dl-dir.bbclass | 21 +++++++++++++-------- > 1 file changed, 13 insertions(+), 8 deletions(-) > > Changes since v2: > - Reworked sudo usage approach from "while/sudo" to "sudo/while". > > Changes since v1: > - Try create handrlink first and then copy if failed. It should better > handle hardlink issues that may happen even on the same partition. > > diff --git a/meta/classes/deb-dl-dir.bbclass > b/meta/classes/deb-dl-dir.bbclass index 7db25251..ca2a1ee2 100644 > --- a/meta/classes/deb-dl-dir.bbclass > +++ b/meta/classes/deb-dl-dir.bbclass > @@ -81,20 +81,24 @@ deb_dl_dir_import() { > export rootfs="${1}" > sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ > [ ! -d "${pc}" ] && return 0 > - flock -s "${pc}".lock -c ' > + flock -s "${pc}".lock sudo -Es << 'EOSUDO' > set -e > printenv | grep -q BB_VERBOSE_LOGS && set -x > > - sudo find "${pc}" -type f -iname "*\.deb" -exec \ > - ln -Pf -t "${rootfs}"/var/cache/apt/archives/ {} + > - ' > + find "${pc}" -type f -iname "*\.deb" |\ > + while read p; do > + ln -Pf -t "${rootfs}"/var/cache/apt/archives/ "$p" 2>/dev/null > || + cp -n --no-preserve=owner -t > "${rootfs}"/var/cache/apt/archives/ "$p" + done > +EOSUDO > } > > deb_dl_dir_export() { > export pc="${DEBDIR}/${2}" > export rootfs="${1}" > + export owner=$(id -u):$(id -g) > mkdir -p "${pc}" > - flock "${pc}".lock -c ' > + flock "${pc}".lock sudo -Es << 'EOSUDO' > set -e > printenv | grep -q BB_VERBOSE_LOGS && set -x > > @@ -109,8 +113,9 @@ deb_dl_dir_export() { > if [ -n "$package" ]; then > cmp --silent "$package" "$p" && continue > fi > - sudo ln -Pf "${p}" "${pc}" > + ln -Pf "${p}" "${pc}" 2>/dev/null || > + cp -n "${p}" "${pc}" > done > - sudo chown -R $(id -u):$(id -g) "${pc}" > - ' > + chown -R ${owner} "${pc}" > +EOSUDO > } Applied to next