Re: [PATCHv5 25/29] meta: deb-dl-dir: only export newly downloaded files

[Henning Schild <henning.schild@siemens.com>]

Hi,

i think i have a working replacement for this one now. In order to
prepare a new q i still lack the whitespace fixes or the review on what
needed fixing.

Henning

On Wed, 18 Mar 2020 08:59:43 +0100
"[ext] Henning Schild" <henning.schild@siemens.com> wrote:

> On Tue, 10 Mar 2020 22:56:27 +0100
> Henning Schild <henning.schild@siemens.com> wrote:
> 
> > From: Henning Schild <henning.schild@siemens.com>
> > 
> > Remember the time we imported the previously downloaded files, and
> > only copy out new ones when exporting again. That saved pointless
> > copies and time.
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/deb-dl-dir.bbclass | 13 ++++++++++---
> >  1 file changed, 10 insertions(+), 3 deletions(-)
> > 
> > diff --git a/meta/classes/deb-dl-dir.bbclass
> > b/meta/classes/deb-dl-dir.bbclass index 011dd717..55c4a839 100644
> > --- a/meta/classes/deb-dl-dir.bbclass
> > +++ b/meta/classes/deb-dl-dir.bbclass
> > @@ -8,24 +8,31 @@ inherit repository
> >  deb_dl_dir_import() {
> >      export pc="${DEBDIR}/${DISTRO}/"
> >      export rootfs="${1}"
> > +    rm -f "${T}"/deb_dl_dir_import.stamp
> > +    touch "${T}"/deb_dl_dir_import.stamp  
> 
> Creating the stamp outside the flock is a race. We could create a new
> stamp and get stuck on the lock while a reader would skip new packages
> that have not been imported yet.
> 
> Henning
> 
> >      [ ! -d "${pc}" ] && return 0
> > +    [ ! -d "${rootfs}"/var/cache/apt/archives/ ] && return 0
> >      flock -s "${pc}".lock -c ' \
> >          sudo find "${pc}" -type f -iname '*\.deb' -exec \
> > -            cp -f --no-preserve=owner -t
> > "${rootfs}"/var/cache/apt/archives/ '{}' +
> > +            cp -n --no-preserve=owner -t
> > "${rootfs}"/var/cache/apt/archives/ '{}' + '
> >  }
> >  
> >  deb_dl_dir_export() {
> >      export pc="${DEBDIR}/${DISTRO}/"
> >      export rootfs="${1}"
> > +    export T
> >      mkdir -p "${pc}"
> >      flock "${pc}".lock -c ' \
> > -        find "${rootfs}"/var/cache/apt/archives/ -type f -iname
> > '*\.deb' |\
> > +        find "${rootfs}"/var/cache/apt/archives/ \
> > +	    -cnewer "${T}"/deb_dl_dir_import.stamp \
> > +	    -type f -iname '*\.deb' |\
> >          while read p; do
> >               repo_contains_package "${REPO_ISAR_DIR}"/"${DISTRO}"
> > "${p}" && \ continue
> > -             sudo cp -f "${p}" "${pc}"
> > +             sudo cp -n "${p}" "${pc}"
> >          done
> >          sudo chown -R $(id -u):$(id -g) "${pc}"
> >      '
> > +    rm -f "${T}"/deb_dl_dir_import.stamp
> >  }  
>