From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:a2e:9a01:: with SMTP id o1mr15046108lji.247.1580756362534; Mon, 03 Feb 2020 10:59:22 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:6809:: with SMTP id c9ls1825340lja.1.gmail; Mon, 03 Feb 2020 10:59:21 -0800 (PST) X-Google-Smtp-Source: APXvYqx3rM7Gx5N+3MKjA0FCkNZvgzz+zDbSdPWQ/E8XUQLWaGOayqw3S8jooNeeWgUd9RvWk3qf X-Received: by 2002:a2e:2283:: with SMTP id i125mr15205632lji.244.1580756361619; Mon, 03 Feb 2020 10:59:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1580756361; cv=none; d=google.com; s=arc-20160816; b=lk80oB7tZ6vZz/d0CLrqSYaqFdCx1t6lNiYVGoHUNtSVnvl1usmZjzZZHdn+vq4SX1 k8MnHbsUkhNJ3Z4DsneIV+39gTjEqxqZrmZxTop97lK3jPF5vx1nIon1vEBNIVIEtgrb 605Ytw+pYmKJhira+HNffRIPiOjaijaT4qhfS1FXGj7o9eIB0PlbGsNaWm1C7b1crxvL NeXNXfh0zmcp77szj5WTYShOjI0XdBDewQ5Q4sgf6U0oEmoKRypfpt/xKTI1m4TvLOwU +u9SsTUb2DkmnIYDntu/QJgZH2wDdkHJX65Eev6HNABj3+ukMQLZKjhFq7T+EktbFc80 DAww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=v0LCAx0YGOEe5XbewBujwLUBwkSIE2u2fP+cF0KYYow=; b=UbuuVNVnu+Tw72iXzWTWJR1dSvxEyWfK1H/q/smk6cyHsP6wJFaHApv0euZw9VCdmK sOmBb2MrZmUrZAQLCQJaWkHuwAhjcKC9ZBr77zCMwUfP9el8E2dOD4GLORt+xaKNsmRF 9m6/gTSHj5ZWQOe6FUCNx32mFDCX7GeLcnwz1m+VA+2aMH9aCSReUtWmhn7CwGC8s2Q1 IRO2xVYHeKD2oeXzLQS1GRxFpK8bScJiAcLVduWKhs+EFFQBTfIDN+ijIb5fw65FlvUn X9xV/OlWhbXjhKg91yLS5RZdyVyq0EGqZ5Waq//qkDhWVOfLJzO1vw8i17kB2AD5CvWx FSzQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id b29si1007680lfo.2.2020.02.03.10.59.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Feb 2020 10:59:21 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 013IxKLO020532 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 3 Feb 2020 19:59:20 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.193]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 013IxKhl010787; Mon, 3 Feb 2020 19:59:20 +0100 Date: Mon, 3 Feb 2020 19:59:19 +0100 From: Henning Schild To: Jan Kiszka Cc: , "Su, Bao Cheng (RC-CN DI FA R&D SW)" , Vijai Kumar K Subject: Re: [PATCHv4 22/26] meta: deb-dl-dir: do not cache debs from isar-apt Message-ID: <20200203195919.02560450@md1za8fc.ad001.siemens.net> In-Reply-To: <20200203192701.2700782a@md1za8fc.ad001.siemens.net> References: <20200131143000.14873-1-henning.schild@siemens.com> <20200131143000.14873-23-henning.schild@siemens.com> <16977d1f-fb4d-481b-9c55-53ed064df333@siemens.com> <20200203192701.2700782a@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: bk7CKXPtKbiI Am Mon, 3 Feb 2020 19:27:01 +0100 schrieb "[ext] Henning Schild" : > Am Mon, 3 Feb 2020 18:20:11 +0100 > schrieb Jan Kiszka : > > > On 31.01.20 15:29, [ext] Henning Schild wrote: > > > From: Henning Schild > > > > > > Packages from isar-apt are not downloaded from the outside and > > > should not be cached. > > > > > > Signed-off-by: Henning Schild > > > --- > > > meta/classes/deb-dl-dir.bbclass | 14 ++++++++++---- > > > 1 file changed, 10 insertions(+), 4 deletions(-) > > > > > > diff --git a/meta/classes/deb-dl-dir.bbclass > > > b/meta/classes/deb-dl-dir.bbclass index ab4b1f09..f9699603 100644 > > > --- a/meta/classes/deb-dl-dir.bbclass > > > +++ b/meta/classes/deb-dl-dir.bbclass > > > @@ -3,8 +3,10 @@ > > > # > > > # SPDX-License-Identifier: MIT > > > > > > +inherit repository > > > + > > > deb_dl_dir_import() { > > > - export pc="${DEBDIR}/${DISTRO}" > > > + export pc="${DEBDIR}/${DISTRO}/" > > > export rootfs="${1}" > > > [ ! -d "${pc}" ] && return 0 > > > flock -s "${pc}".lock -c ' \ > > > @@ -14,12 +16,16 @@ deb_dl_dir_import() { > > > } > > > > > > deb_dl_dir_export() { > > > - export pc="${DEBDIR}/${DISTRO}" > > > + export pc="${DEBDIR}/${DISTRO}/" > > > export rootfs="${1}" > > > mkdir -p "${pc}" > > > flock "${pc}".lock -c ' \ > > > - sudo find "${rootfs}"/var/cache/apt/archives/ -type f > > > -iname '*\.deb' \ > > > - -exec cp -f '{}' "${pc}" \; > > > + find "${rootfs}"/var/cache/apt/archives/ -type f -iname > > > '*\.deb' |\ > > > + while read p; do > > > + repo_contains_package "${REPO_ISAR_DIR}"/"${DISTRO}" > > > "${p}" && \ > > > > repo_contains_package may not be found inside the flock shell > > context, as Bao Cheng noticed out. He suggests the pattern > > That is possible indeed. I remember that i struggled re-using the > function since i did not want to code it twice. > > But to be honest i do not understand the problem with the given > description. Bao Cheng please go into more detail. This is a hot code-path and if it is _very_ broken that would be very visible. Just tried a build and am looking at the download cache and isar-apt find tmp/deploy/isar-apt/ -iname *hello* tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello/libhello-dbgsym_0.1_amd64.deb tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello/libhello-dev_0.1_amd64.deb tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello/libhello_0.1_amd64.deb .... tmp/deploy/isar-apt/apt/debian-buster/pool/main/h/hello-isar/hello-isar_0.3_amd64.deb Ok so we have the hello stuff in isar-apt, the own package as well as the rebuild upstream (also own). And nothing in the download cache on hello -> expected for the image find downloads/deb/ -iname *hello* Meaning the filtering worked in my case. And the caching of all required outside stuff works as well, since the offline rebuild works. Caching and filtering are the two main aspects of that code. Please make sure to give feedback for sure, i do not want such a unclear description slowing down the merge even more. Henning > Maybe the filtering indeed does not work, i will try that on a simple > test now. And it might be a good idea to make sure that code gets a > "set -e". > > Henning > > > (flock 8 > > ... > > ) 8>${LOCKFILE} > > > > Bao Check, maybe you can describe how you noticed. > > > > Jan > > > > > + continue > > > + sudo cp -f "${p}" "${pc}" > > > + done > > > sudo chown -R $(id -u):$(id -g) "${pc}" > > > ' > > > } > > > > > >