From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:a50:9f6e:: with SMTP id b101mr2621815edf.372.1584518389905; Wed, 18 Mar 2020 00:59:49 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aa7:de0a:: with SMTP id h10ls10883979edv.11.gmail; Wed, 18 Mar 2020 00:59:49 -0700 (PDT) X-Google-Smtp-Source: ADFU+vtPOzTOFGLl3A6Z1WBG0Hku8NsIpp63a0nzv1taGmafCM1+khvdiw2i6x+qHUZnTaM+TWSG X-Received: by 2002:aa7:c9cb:: with SMTP id i11mr2719381edt.320.1584518389374; Wed, 18 Mar 2020 00:59:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1584518389; cv=none; d=google.com; s=arc-20160816; b=rsxxBcTxi+YaP+y4bCxR2+Rf7qNp61mXGw+ydgYbn6nCrsffF6u5dbf2R6cSzrBeD4 6rIGsN81EYs1QRyCvYCbYrn6koUJlyh2FGaRwDT2fFlzRopZnO5k3q8004t+brTBkpG8 Ra5n9vF4ObTORAyNl+PluavdKSBuW1Kmpe1f+SAc0HxwzzhLC1r/iLIvL5HpMvwoHX20 hxAYj3qE5cosyPdvOi+oSakSLrd85WpmYhmzw+UkGk2dYXe3glKu2kwBeCtG+kWzrag1 dh/3COAJUb0SQ2fINnDUcAjCzKp6ayQaYn2znHsOvS4mV8kdM6ETNsVJFhgkDurstAGD G7kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=B1GA/0pL/RIQ3XI7+fv14VWQHwu2ZG/aAOetdt07rnw=; b=Lv7RuOcAtBoAe4tUXyyshP0f/I0f60hReXDj0aLUIXrTKEG4P5zNZmfLf71wCPY6Mi Eq8Bzslekmu6bGv2rYp7yPfe4ECoa+Cv/TktNdAWBeG7eHvqZ2tIVRGsCCOSEapmOHLk wUcuss09eNetR4jklAQVC4pgbsX/YuJ77AjUqpgyBSgRkrzIEWm/bp35wNv+llejsALB qo94MdHIvwmC2EiYndtzCvayb2+XqDu2bz28l78aShDMTtm1AqaliKB5R3lFeDJi0tq4 ghnU00jF1jeESTkEDvGPZmDHU0iWhO40oxKd1xc4ccKCZ+cbo8R6GxulOoPYQEGZV2ok t1Qg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id k9si109061eds.1.2020.03.18.00.59.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Mar 2020 00:59:49 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 02I7xmgT022276 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 18 Mar 2020 08:59:48 +0100 Received: from md1za8fc.ad001.siemens.net ([167.87.5.68]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 02I7xlcs018033; Wed, 18 Mar 2020 08:59:47 +0100 Date: Wed, 18 Mar 2020 08:59:43 +0100 From: Henning Schild To: Cc: Vijai Kumar K , Baurzhan Ismagulov , Jan Kiszka Subject: Re: [PATCHv5 25/29] meta: deb-dl-dir: only export newly downloaded files Message-ID: <20200318085943.1fad7c46@md1za8fc.ad001.siemens.net> In-Reply-To: <20200310215631.24859-26-henning.schild@siemens.com> References: <20200310215631.24859-1-henning.schild@siemens.com> <20200310215631.24859-26-henning.schild@siemens.com> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: Z3hqP2aNNOPA On Tue, 10 Mar 2020 22:56:27 +0100 Henning Schild wrote: > From: Henning Schild > > Remember the time we imported the previously downloaded files, and > only copy out new ones when exporting again. That saved pointless > copies and time. > > Signed-off-by: Henning Schild > --- > meta/classes/deb-dl-dir.bbclass | 13 ++++++++++--- > 1 file changed, 10 insertions(+), 3 deletions(-) > > diff --git a/meta/classes/deb-dl-dir.bbclass > b/meta/classes/deb-dl-dir.bbclass index 011dd717..55c4a839 100644 > --- a/meta/classes/deb-dl-dir.bbclass > +++ b/meta/classes/deb-dl-dir.bbclass > @@ -8,24 +8,31 @@ inherit repository > deb_dl_dir_import() { > export pc="${DEBDIR}/${DISTRO}/" > export rootfs="${1}" > + rm -f "${T}"/deb_dl_dir_import.stamp > + touch "${T}"/deb_dl_dir_import.stamp Creating the stamp outside the flock is a race. We could create a new stamp and get stuck on the lock while a reader would skip new packages that have not been imported yet. Henning > [ ! -d "${pc}" ] && return 0 > + [ ! -d "${rootfs}"/var/cache/apt/archives/ ] && return 0 > flock -s "${pc}".lock -c ' \ > sudo find "${pc}" -type f -iname '*\.deb' -exec \ > - cp -f --no-preserve=owner -t > "${rootfs}"/var/cache/apt/archives/ '{}' + > + cp -n --no-preserve=owner -t > "${rootfs}"/var/cache/apt/archives/ '{}' + ' > } > > deb_dl_dir_export() { > export pc="${DEBDIR}/${DISTRO}/" > export rootfs="${1}" > + export T > mkdir -p "${pc}" > flock "${pc}".lock -c ' \ > - find "${rootfs}"/var/cache/apt/archives/ -type f -iname > '*\.deb' |\ > + find "${rootfs}"/var/cache/apt/archives/ \ > + -cnewer "${T}"/deb_dl_dir_import.stamp \ > + -type f -iname '*\.deb' |\ > while read p; do > repo_contains_package "${REPO_ISAR_DIR}"/"${DISTRO}" > "${p}" && \ continue > - sudo cp -f "${p}" "${pc}" > + sudo cp -n "${p}" "${pc}" > done > sudo chown -R $(id -u):$(id -g) "${pc}" > ' > + rm -f "${T}"/deb_dl_dir_import.stamp > }