Re: [PATCH 02/17] dpkg-base: add download caching of apt:// downloads

[Henning Schild <henning.schild@siemens.com>]

On Tue, 29 Oct 2019 05:57:49 -0700
<chombourger@gmail.com> wrote:

> On Monday, October 28, 2019 at 6:01:35 PM UTC+1, Henning Schild wrote:
> >
> > From: Henning Schild <hennin...@siemens.com <javascript:>> 
> >
> > Store the results of "apt-get source" in the DL_DIR. This means we 
> > really only need to fetch once on incemental rebuilds or if
> > multiple recipes use the same SRC_URI. 
> > We can also later collect the results and put them in a repository. 
> >
> > Signed-off-by: Henning Schild <hennin...@siemens.com <javascript:>> 
> > --- 
> >  meta/classes/dpkg-base.bbclass | 20 ++++++++++++++++++-- 
> >  meta/conf/bitbake.conf         |  1 + 
> >  2 files changed, 19 insertions(+), 2 deletions(-) 
> >
> > diff --git a/meta/classes/dpkg-base.bbclass 
> > b/meta/classes/dpkg-base.bbclass 
> > index 3f1e99f..b39df5c 100644 
> > --- a/meta/classes/dpkg-base.bbclass 
> > +++ b/meta/classes/dpkg-base.bbclass 
> > @@ -37,14 +37,30 @@ do_apt_fetch() { 
> >          -o Dir::Etc::SourceParts="-" \ 
> >          -o APT::Get::List-Cleanup="0" 
> >   
> > -    sudo -E chroot --userspec=$( id -u ):$( id -g )
> > ${BUILDCHROOT_DIR} \ 
> > -        sh -c 'cd ${PP} && apt-get -y --only-source source
> > ${SRC_APT}' 
> > +    for uri in "${SRC_APT}"; do 
> > +        sudo -E chroot --userspec=$( id -u ):$( id -g ) 
> > ${BUILDCHROOT_DIR} \ 
> > +            sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd 
> > /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only
> > --only-source source "$2"' my_script "${DISTRO}" "${uri}" 
> > +        sudo -E chroot --userspec=$( id -u ):$( id -g ) 
> > ${BUILDCHROOT_DIR} \ 
> > +            sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd
> > ${PP} && apt-get -y --only-source source "$2"' my_script
> > "${DISTRO}" "${uri}" 
> > +    done 
> > + 
> >      dpkg_undo_mounts 
> >  } 
> >   
> >  addtask apt_fetch after do_unpack before do_patch 
> >  do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" 
> >   
> > +addtask cleanall_apt before do_cleanall 
> > +do_cleanall_apt[nostamp] = "1" 
> > +do_cleanall_apt() { 
> > +    if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then 
> > +        exit 
> >  
> 
> shouldn't this be "return 0" instead?
> (or are we really wanting to exit the bitbake process here?

I am pretty sure i tested it and bitbake survived. But i will
double-check or maybe just return 0 anyways.

Henning

> 
> > +    fi 
> > +    for uri in "${SRC_APT}"; do 
> > +        rm -rf "${DEBSRCDIR}"/"${DISTRO}"/"$uri" 
> > +    done 
> > +} 
> > + 
> >  def get_package_srcdir(d): 
> >      s = os.path.abspath(d.getVar("S", True)) 
> >      workdir = os.path.abspath(d.getVar("WORKDIR", True)) 
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf 
> > index bca1114..4c0a809 100644 
> > --- a/meta/conf/bitbake.conf 
> > +++ b/meta/conf/bitbake.conf 
> > @@ -26,6 +26,7 @@ DEPLOY_DIR = "${TMPDIR}/deploy" 
> >  FILESPATH = 
> > "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}" 
> >
> >  FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}" 
> >  GITDIR = "${DL_DIR}/git" 
> > +DEBSRCDIR = "${DL_DIR}/deb-src" 
> >  P = "${PN}-${PV}" 
> >  PF = "${PN}-${PV}-${PR}" 
> >  PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE',
> > False),d)[0] or 'defaultpkgname'}" 
> > -- 
> > 2.23.0 
> >
> >  
>