From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:adf:e4cf:: with SMTP id v15mr9940642wrm.237.1584696475572; Fri, 20 Mar 2020 02:27:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:4ecf:: with SMTP id s15ls2251274wrv.0.gmail; Fri, 20 Mar 2020 02:27:55 -0700 (PDT) X-Google-Smtp-Source: ADFU+vvZsLCfMzGc1UMlu2CHtKsqkPmn9B0BK7H/sRtcwbTrfAt6jII0o1E5r2mbDXsjT9jcPsoi X-Received: by 2002:a5d:498f:: with SMTP id r15mr10297997wrq.81.1584696475021; Fri, 20 Mar 2020 02:27:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1584696475; cv=none; d=google.com; s=arc-20160816; b=XpTvfiOKrInc44PfxvJPtOl+Y9NCQheSjkVi5Faym6T+6Po11+xfypfWKh6TiB8OxL J6cVYZ+BKyLYdwEIyNM05vdN56YVfC8pZjdS3Ka3PFH02pch0ee5rMmNPCGlAFb/9I6c F3oRD26QGTOqJ8QyukfBSr0jyOourCCRHGE/Es7HqNrnWDAxyzgeMm18QNXKpYJmMo0c 4PNVVYGhM6TF6zv3FKArsZ7MxWo87AE5CAhKFAMoWtXV0LOuEr0pl6FOKVRalDQqer4z UN/Bmj9xAcDEGUmOkVVwIInsYiPA+4Df6ZlKq7c1UIV5cqjR6GAj6RULy3DDJ5BAifz5 Hqgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=FgR+1xywx+gir81XIbXxFFrzFjwqCfTo/CvAfjnonBc=; b=A8bajX2F09QoXM5yXpk9lJgseM1Nb4rgXG9uySgJEAMCkPXk7uD2M4l/Wrh41ABIDp X7tKbDxT+/I2ewkkBdbsMUYpkgFlftll4/gNhI883LkLgDCo6fdkZZtXrSUbj6I38/tv csSe0r29n+xSAToqGOgCzvIaYlwws78XZ3Ons6DpuH1/NHNCgI74gFkf6Ec3IKQ8Ec4o ovr1fqcnwlSrpfeVaowK2Ke96zRq9hqNV1M5vmShMyxRyDQBb/Q4nuHCTHGdJm/nFxL/ srhYx4EzvX1vc4iPmO9flhSguhx7R4/8N2ft8qbJ4jq434iFA4FcN48NxYQzpDN9erHw KY3g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id n9si226573wmi.0.2020.03.20.02.27.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2020 02:27:54 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 02K9Rrut009310 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 20 Mar 2020 10:27:53 +0100 Received: from md1za8fc.ad001.siemens.net ([167.87.43.60]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 02K9Rq4d021189; Fri, 20 Mar 2020 10:27:52 +0100 Date: Fri, 20 Mar 2020 10:27:46 +0100 From: Henning Schild To: Cc: Vijai Kumar K , Baurzhan Ismagulov , Jan Kiszka Subject: Re: [PATCHv5 25/29] meta: deb-dl-dir: only export newly downloaded files Message-ID: <20200320102746.7b7dc38a@md1za8fc.ad001.siemens.net> In-Reply-To: <20200318085943.1fad7c46@md1za8fc.ad001.siemens.net> References: <20200310215631.24859-1-henning.schild@siemens.com> <20200310215631.24859-26-henning.schild@siemens.com> <20200318085943.1fad7c46@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: cGkKO+PxRwNA Hi, i think i have a working replacement for this one now. In order to prepare a new q i still lack the whitespace fixes or the review on what needed fixing. Henning On Wed, 18 Mar 2020 08:59:43 +0100 "[ext] Henning Schild" wrote: > On Tue, 10 Mar 2020 22:56:27 +0100 > Henning Schild wrote: > > > From: Henning Schild > > > > Remember the time we imported the previously downloaded files, and > > only copy out new ones when exporting again. That saved pointless > > copies and time. > > > > Signed-off-by: Henning Schild > > --- > > meta/classes/deb-dl-dir.bbclass | 13 ++++++++++--- > > 1 file changed, 10 insertions(+), 3 deletions(-) > > > > diff --git a/meta/classes/deb-dl-dir.bbclass > > b/meta/classes/deb-dl-dir.bbclass index 011dd717..55c4a839 100644 > > --- a/meta/classes/deb-dl-dir.bbclass > > +++ b/meta/classes/deb-dl-dir.bbclass > > @@ -8,24 +8,31 @@ inherit repository > > deb_dl_dir_import() { > > export pc="${DEBDIR}/${DISTRO}/" > > export rootfs="${1}" > > + rm -f "${T}"/deb_dl_dir_import.stamp > > + touch "${T}"/deb_dl_dir_import.stamp > > Creating the stamp outside the flock is a race. We could create a new > stamp and get stuck on the lock while a reader would skip new packages > that have not been imported yet. > > Henning > > > [ ! -d "${pc}" ] && return 0 > > + [ ! -d "${rootfs}"/var/cache/apt/archives/ ] && return 0 > > flock -s "${pc}".lock -c ' \ > > sudo find "${pc}" -type f -iname '*\.deb' -exec \ > > - cp -f --no-preserve=owner -t > > "${rootfs}"/var/cache/apt/archives/ '{}' + > > + cp -n --no-preserve=owner -t > > "${rootfs}"/var/cache/apt/archives/ '{}' + ' > > } > > > > deb_dl_dir_export() { > > export pc="${DEBDIR}/${DISTRO}/" > > export rootfs="${1}" > > + export T > > mkdir -p "${pc}" > > flock "${pc}".lock -c ' \ > > - find "${rootfs}"/var/cache/apt/archives/ -type f -iname > > '*\.deb' |\ > > + find "${rootfs}"/var/cache/apt/archives/ \ > > + -cnewer "${T}"/deb_dl_dir_import.stamp \ > > + -type f -iname '*\.deb' |\ > > while read p; do > > repo_contains_package "${REPO_ISAR_DIR}"/"${DISTRO}" > > "${p}" && \ continue > > - sudo cp -f "${p}" "${pc}" > > + sudo cp -n "${p}" "${pc}" > > done > > sudo chown -R $(id -u):$(id -g) "${pc}" > > ' > > + rm -f "${T}"/deb_dl_dir_import.stamp > > } >