From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6771003121820762112 X-Received: by 2002:a05:600c:118a:: with SMTP id i10mr3063478wmf.142.1578649875929; Fri, 10 Jan 2020 01:51:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a7b:c3d8:: with SMTP id t24ls1872263wmj.5.canary-gmail; Fri, 10 Jan 2020 01:51:15 -0800 (PST) X-Google-Smtp-Source: APXvYqyTBHEnuOe4qGqC2MoypgAuOz8Nhs1CqnrdLVooeJPR45baCJPr9S44M+nlXZW8fIPwXNip X-Received: by 2002:a7b:c114:: with SMTP id w20mr3189844wmi.151.1578649875426; Fri, 10 Jan 2020 01:51:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1578649875; cv=none; d=google.com; s=arc-20160816; b=Wf3o3ca++3xJBt57uQhw5JntlJBkJcd5Ve0fjmqaQdbihRD0h0X99iolxSXTMf+FJj QAGn+Q3tIAbsEXquZKrEnkj7sA5CYVvX0Fuc0WX9WMA58oVoLj8/fOBkDxcH+WaLgGvc zKMT7dQwDJfemsLqqZepZG5dMQqKZBPRRpNcmuJoaW/j/Rahz4gprcqu3xsHIjQ8tuf8 YS/FxAXerix0+0buJljLEnFQecPZq40S8SPsw0iS2vmskGNKmFi8rToGPPGBqPqbzWHo ABh5DUXd4CTdidSc4/d72gdc0IcScs34h5fByjT7tdFXJ8IFTaUca90QzwvrdmPm2Vn/ iq5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=V8/o/53X1kiIv8voWTDtNtP6V1a6uL/o6tp4wDMtPFI=; b=rU83CJNL7irfFDhTcHlFcy9PmosoqekiDhpxMvfvU4JfCImYP/VFbXzyMB7X+b98jw arrFK9fyozH7WbW7zoFPGm84gniN1QOudUOV10Jo7NEu4eRpj7nY8n2McuNrD0eHTXcU jkvMtR6a0b++g76xhlvZ85zt8ye/h1R/wmmRpbD6YIjuw6ZWS2VBAkso3rYlWg1H6c0/ IEIKS5TyMAqd4S2gz8KGTlMS1ii5CCS2q/0Ib0m4WOWthzudM2zraWLzQBmGEYPtqznI 9tnSJA6oR7wSEeBF3xtV1c+sw8wIviFLdFMUtLELrBaoagLH7TISzpCLLyLXkYU0INCw pS2A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id b9si75405wrw.2.2020.01.10.01.51.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Jan 2020 01:51:15 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 00A9pEf8023157 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 Jan 2020 10:51:14 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.125]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 00A9pEbF027070; Fri, 10 Jan 2020 10:51:14 +0100 Date: Fri, 10 Jan 2020 10:51:11 +0100 From: Henning Schild To: vijai kumar Cc: isar-users Subject: Re: [PATCHv2 02/25] dpkg-base: add download caching of apt:// downloads Message-ID: <20200110105111.3d935274@md1za8fc.ad001.siemens.net> In-Reply-To: <590c1f63-c923-47c1-8c48-9ed3edc766a0@googlegroups.com> References: <20191216115011.17664-1-henning.schild@siemens.com> <20191216115011.17664-3-henning.schild@siemens.com> <590c1f63-c923-47c1-8c48-9ed3edc766a0@googlegroups.com> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: hT0ugQKlz+qU Am Thu, 9 Jan 2020 03:27:25 -0800 schrieb vijai kumar : > On Monday, December 16, 2019 at 5:20:15 PM UTC+5:30, Henning Schild > wrote: > > > > From: Henning Schild > > > > > Store the results of "apt-get source" in the DL_DIR. This means we > > really only need to fetch once on incremental rebuilds or if > > multiple recipes use the same SRC_URI. > > We can also later collect the results and put them in a repository. > > > > Signed-off-by: Henning Schild > > > --- > > meta/classes/dpkg-base.bbclass | 20 ++++++++++++++++++-- > > meta/conf/bitbake.conf | 1 + > > 2 files changed, 19 insertions(+), 2 deletions(-) > > > > diff --git a/meta/classes/dpkg-base.bbclass > > b/meta/classes/dpkg-base.bbclass > > index f38168c..7a97d91 100644 > > --- a/meta/classes/dpkg-base.bbclass > > +++ b/meta/classes/dpkg-base.bbclass > > @@ -63,14 +63,30 @@ do_apt_fetch() { > > -o Dir::Etc::SourceParts="-" \ > > -o APT::Get::List-Cleanup="0" > > > > - sudo -E chroot --userspec=$( id -u ):$( id -g ) > > ${BUILDCHROOT_DIR} \ > > - sh -c 'cd ${PP} && apt-get -y --only-source source > > ${SRC_APT}' > > + for uri in "${SRC_APT}"; do > > + sudo -E chroot --userspec=$( id -u ):$( id -g ) > > ${BUILDCHROOT_DIR} \ > > + sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd > > /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only > > --only-source source "$2"' my_script "${DISTRO}" "${uri}" > > + sudo -E chroot --userspec=$( id -u ):$( id -g ) > > ${BUILDCHROOT_DIR} \ > > + sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd > > ${PP} && apt-get -y --only-source source "$2"' my_script > > "${DISTRO}" "${uri}" > > Hi Henning, > > Thank you for the base-apt rework. > BTW. What does my_script do here? It fills $0 of the embedded shell script with a "program name". Henning > Thanks, > Vijai Kumar K > > + done > > + > > dpkg_undo_mounts > > } > > > > addtask apt_fetch after do_unpack before do_patch > > do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > > > > +addtask cleanall_apt before do_cleanall > > +do_cleanall_apt[nostamp] = "1" > > +do_cleanall_apt() { > > + if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then > > + exit > > + fi > > + for uri in "${SRC_APT}"; do > > + rm -rf "${DEBSRCDIR}"/"${DISTRO}"/"$uri" > > + done > > +} > > + > > def get_package_srcdir(d): > > s = os.path.abspath(d.getVar("S", True)) > > workdir = os.path.abspath(d.getVar("WORKDIR", True)) > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > > index 0eb3b54..ab608da 100644 > > --- a/meta/conf/bitbake.conf > > +++ b/meta/conf/bitbake.conf > > @@ -26,6 +26,7 @@ DEPLOY_DIR = "${TMPDIR}/deploy" > > FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}" > > FILESEXTRAPATHS ?= "__default:" > > GITDIR = "${DL_DIR}/git" > > +DEBSRCDIR = "${DL_DIR}/deb-src" > > P = "${PN}-${PV}" > > PF = "${PN}-${PV}-${PR}" > > PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', > > False),d)[0] or 'defaultpkgname'}" > > -- > > 2.24.1 > > > > >