From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:a5d:4d12:: with SMTP id z18mr21676670wrt.139.1580808804073; Tue, 04 Feb 2020 01:33:24 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a7b:c0d9:: with SMTP id s25ls2314453wmh.0.gmail; Tue, 04 Feb 2020 01:33:23 -0800 (PST) X-Google-Smtp-Source: APXvYqxW9BMNHuD5Rf1EoKDJyFQZn0mT9i+A+pjNb+VeqaFXX7iRP9nRsl9ZebcFX/N2qYSDyh3A X-Received: by 2002:a1c:7c18:: with SMTP id x24mr5057835wmc.185.1580808803181; Tue, 04 Feb 2020 01:33:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1580808803; cv=none; d=google.com; s=arc-20160816; b=pTfmJkA2RWwcCp5VfLXAhCYEIOlhS5DLzMs96SBtSjRUt4MTVmLEVjRmJrvo+/JGwA 10TaLirH2xDL6TNx+45IQH+1LU5rmV5uuEf1BmbxFIToiViSDnO3KfgNKJl3B4Lpv1QE mbp3y8N014wx0Old5U8zCs0DhGWunSyOt5a3Qo9FV/k/UtZBYqhQzzaXPxekZQL17ecN 0RikbDfELZW5awWm/dkU80iS/HjnH6VTrmoHSMtadvA6qoNH6DLL6Kybw1XVBG7I5rwt ilgC8y5S5LljShWz423RZw1FpAuS5BuCLcxgn7psN5/ty4Iw8IdfEri8J7TMtTS7Ih1T ZgrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=eVi0fp08aFT4fo9g0ifjInNxITAf1ngPlHmt2iq90Zs=; b=zRJZPYACBa15kqZcgSxm6CpEEhIBbNCY/OPZKwPXkkqLT450eUbpn/KQACVdsuFT5d ke3y/MXwZw5pEzd1fR21ubD1A2iOWgnHXFVnVAuEdrNMT/hhXEQolDRYl8vmOOH93/bj WlaxhGL0FB9GjCyYi+x9KgmR2+xsKrazQFKGcKC84NKA9uMx/6SoqxDxuyPQuulaxy0O NOA8K4Gh3rNOSCLvqbrH4gje4VfJ3zl6/mbevSrt0e60VPAGzBKCsNNM5347JsPZNbLQ 1CebH/pPNPHf1yQjaZOJdX8HH+2KNp+fPcIwh8neKGidVLvBFUR/du+m3p4zD8V27zvU /5og== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id i18si869497wrn.0.2020.02.04.01.33.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Feb 2020 01:33:23 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 0149XMnE012912 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 4 Feb 2020 10:33:22 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.193]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 0149XM59006329; Tue, 4 Feb 2020 10:33:22 +0100 Date: Tue, 4 Feb 2020 10:33:21 +0100 From: Henning Schild To: "Su, Bao Cheng (RC-CN DI FA R&D SW)" Cc: "Kiszka, Jan (CT RDA IOT SES-DE)" , "isar-users@googlegroups.com" , Vijai Kumar K Subject: Re: [PATCHv4 22/26] meta: deb-dl-dir: do not cache debs from isar-apt Message-ID: <20200204103321.538ba97e@md1za8fc.ad001.siemens.net> In-Reply-To: <2ad5750c-43f6-4b19-aeee-a9978fe39886@siemens.com> References: <20200131143000.14873-1-henning.schild@siemens.com> <20200131143000.14873-23-henning.schild@siemens.com> <16977d1f-fb4d-481b-9c55-53ed064df333@siemens.com> <20200203192701.2700782a@md1za8fc.ad001.siemens.net> <20200203195919.02560450@md1za8fc.ad001.siemens.net> <5a62e0f4-f8a9-4e08-8010-5a47f49bd2b6@siemens.com> <2ad5750c-43f6-4b19-aeee-a9978fe39886@siemens.com> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-TUID: 31rhQIpBhzV2 I can not reproduce this issue. Meaning we have different build environments or different isar-versions. Since i know you ported to q onto some unknown-to-me state of isar and i do not know your build environment, i would blame of the two. I suggest you try your build env on https://github.com/henning-schild-work/isar/tree/henning/staging4 If the error occurs is could be connected to the build env. I am using a docker based workflow with "kasproject/kas-isar:latest" where latest is pretty new. If the error does not occur i would blame your isar-version or your layer. And with blame i mean we have to look for differences there, nothing more. Henning=20 On Tue, 4 Feb 2020 05:10:05 +0000 "Su, Bao Cheng (RC-CN DI FA R&D SW)" wrote: > Still exists. >=20 > Under `build/tmp/work/`: >=20 > $ grep "command not found" . -r --include=3D"log.do_*" > ./isar-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.329:/bin/= bash: > line 5: repo_contains_package: command not found > ./isar-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.329:/bin/= bash: > line 5: repo_contains_package: command not found > ./isar-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.329:/bin/= bash: > line 5: repo_contains_package: command not found > ./isar-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.329:/bin/= bash: > line 5: repo_contains_package: command not found >=20 > $ grep "command not found" . -r --include=3D"log.do_*" | wc -l > 1596 >=20 > This is only a half-way build, for a full build, there are more > occurrences. >=20 > With best regards > ________________________________ > =E5=8F=91=E4=BB=B6=E4=BA=BA=EF=BC=9A Su, Bao Cheng (RC-CN DI FA R&D SW) > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4=EF=BC=9A 2020=E5=B9=B42=E6=9C=884=E6= =97=A5=E6=98=9F=E6=9C=9F=E4=BA=8C 10:11 > =E6=94=B6=E4=BB=B6=E4=BA=BA=EF=BC=9A Schild, Henning (CT RDA IOT SES-DE);= Kiszka, Jan (CT RDA IOT > SES-DE) =E6=8A=84=E9=80=81=EF=BC=9A isar-users@googlegroups.com; Vijai Ku= mar K > =E4=B8=BB=E9=A2=98=EF=BC=9A Re: [PATCHv4 22/26] meta: deb-dl-dir: do not = cache debs from > isar-apt >=20 > after building, grep "command not found" or "repo_contains_package" > in the log.do_* file. >=20 > I met this problem on an early version of v4 round patch, not sure if > still exists on the latest version. will run a test build against the > latest patch to check. >=20 > Sorry for mailing via phone, due to coronavirus, not convenient for > me to use laptop emails. >=20 > With best regards > ________________________________ > =E5=8F=91=E4=BB=B6=E4=BA=BA=EF=BC=9A Schild, Henning (CT RDA IOT SES-DE) > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4=EF=BC=9A 2020=E5=B9=B42=E6=9C=884=E6= =97=A5=E6=98=9F=E6=9C=9F=E4=BA=8C 02:59 > =E6=94=B6=E4=BB=B6=E4=BA=BA=EF=BC=9A Kiszka, Jan (CT RDA IOT SES-DE) > =E6=8A=84=E9=80=81=EF=BC=9A isar-users@googlegroups.com; Su, Bao Cheng (R= C-CN DI FA R&D > SW); Vijai Kumar K =E4=B8=BB=E9=A2=98=EF=BC=9A Re: [PATCHv4 22/26] meta: = deb-dl-dir: do > not cache debs from isar-apt >=20 > Am Mon, 3 Feb 2020 19:27:01 +0100 > schrieb "[ext] Henning Schild" : >=20 > > Am Mon, 3 Feb 2020 18:20:11 +0100 > > schrieb Jan Kiszka : > > =20 > > > On 31.01.20 15:29, [ext] Henning Schild wrote: =20 > > > > From: Henning Schild > > > > > > > > Packages from isar-apt are not downloaded from the outside and > > > > should not be cached. > > > > > > > > Signed-off-by: Henning Schild > > > > --- > > > > meta/classes/deb-dl-dir.bbclass | 14 ++++++++++---- > > > > 1 file changed, 10 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/meta/classes/deb-dl-dir.bbclass > > > > b/meta/classes/deb-dl-dir.bbclass index ab4b1f09..f9699603 > > > > 100644 --- a/meta/classes/deb-dl-dir.bbclass > > > > +++ b/meta/classes/deb-dl-dir.bbclass > > > > @@ -3,8 +3,10 @@ > > > > # > > > > # SPDX-License-Identifier: MIT > > > > > > > > +inherit repository > > > > + > > > > deb_dl_dir_import() { > > > > - export pc=3D"${DEBDIR}/${DISTRO}" > > > > + export pc=3D"${DEBDIR}/${DISTRO}/" > > > > export rootfs=3D"${1}" > > > > [ ! -d "${pc}" ] && return 0 > > > > flock -s "${pc}".lock -c ' \ > > > > @@ -14,12 +16,16 @@ deb_dl_dir_import() { > > > > } > > > > > > > > deb_dl_dir_export() { > > > > - export pc=3D"${DEBDIR}/${DISTRO}" > > > > + export pc=3D"${DEBDIR}/${DISTRO}/" > > > > export rootfs=3D"${1}" > > > > mkdir -p "${pc}" > > > > flock "${pc}".lock -c ' \ > > > > - sudo find "${rootfs}"/var/cache/apt/archives/ -type f > > > > -iname '*\.deb' \ > > > > - -exec cp -f '{}' "${pc}" \; > > > > + find "${rootfs}"/var/cache/apt/archives/ -type f -iname > > > > '*\.deb' |\ > > > > + while read p; do > > > > + repo_contains_package > > > > "${REPO_ISAR_DIR}"/"${DISTRO}" "${p}" && \ =20 > > > > > > repo_contains_package may not be found inside the flock shell > > > context, as Bao Cheng noticed out. He suggests the pattern =20 > > > > That is possible indeed. I remember that i struggled re-using the > > function since i did not want to code it twice. > > > > But to be honest i do not understand the problem with the given > > description. Bao Cheng please go into more detail. =20 >=20 > This is a hot code-path and if it is _very_ broken that would be very > visible. >=20 > Just tried a build and am looking at the download cache and isar-apt >=20 > find tmp/deploy/isar-apt/ -iname *hello* > tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello > tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello/libhello-db= gsym_0.1_amd64.deb > tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello/libhello-de= v_0.1_amd64.deb > tmp/deploy/isar-apt/apt/debian-buster/pool/main/libh/libhello/libhello_0.= 1_amd64.deb > .... > tmp/deploy/isar-apt/apt/debian-buster/pool/main/h/hello-isar/hello-isar_0= .3_amd64.deb >=20 > Ok so we have the hello stuff in isar-apt, the own package as well as > the rebuild upstream (also own). >=20 > And nothing in the download cache on hello -> expected for the image > find downloads/deb/ -iname *hello* > >=20 > Meaning the filtering worked in my case. >=20 > And the caching of all required outside stuff works as well, since the > offline rebuild works. >=20 > Caching and filtering are the two main aspects of that code. >=20 > Please make sure to give feedback for sure, i do not want such a > unclear description slowing down the merge even more. >=20 > Henning >=20 > > Maybe the filtering indeed does not work, i will try that on a > > simple test now. And it might be a good idea to make sure that code > > gets a "set -e". > > > > Henning > > =20 > > > (flock 8 > > > ... > > > ) 8>${LOCKFILE} > > > > > > Bao Check, maybe you can describe how you noticed. > > > > > > Jan > > > =20 > > > > + continue > > > > + sudo cp -f "${p}" "${pc}" > > > > + done > > > > sudo chown -R $(id -u):$(id -g) "${pc}" > > > > ' > > > > } > > > > =20 > > > =20 > > =20 >=20