Re: [PATCH] deb-dl-dir: Fallback to copying when import-export

[Henning Schild <henning.schild@siemens.com>]

Am Tue, 31 Jan 2023 12:20:12 +0100
schrieb Uladzimir Bely <ubely@ilbers.de>:

> By default, hardlinks are used when doing DL_DIR import and export.
> This fails if DL_DIR's filesystem differs from TMPDIR's one.
> 
> Fallback to file copying (with lower performance and huge disk usage)
> in this case.
> 
> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> ---
>  meta/classes/deb-dl-dir.bbclass | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/deb-dl-dir.bbclass
> b/meta/classes/deb-dl-dir.bbclass index 7db25251..d572e6fb 100644
> --- a/meta/classes/deb-dl-dir.bbclass
> +++ b/meta/classes/deb-dl-dir.bbclass
> @@ -80,13 +80,20 @@ deb_dl_dir_import() {
>      export pc="${DEBDIR}/${2}"
>      export rootfs="${1}"
>      sudo mkdir -p "${rootfs}"/var/cache/apt/archives/
> +
> +    # Use hardlinks with fallback to `cp` if different filesystems
> +    export cmd="ln -Pf -t"
> +    if [ "$(stat -c %m ${pc})" != "$(stat -c %m ${rootfs})" ]; then

Instead of dealing only with the known problem and expecting it to
come, maybe try an actual hardlink and "catch the exception".

cmd = ln
cmd testfile || cmd = cp
rm -f testfile

Who knows which other reasons we might find where hardlinks do not
work. And who knows how that stat could get confused by bind-mounts or
funny filesystems ... where hardlinks might work after all, even if the
stat would guess not.

So instead of guessing i am all for actually trying.

Henning

> +        export cmd="cp -n --no-preserve=owner"
> +    fi
> +
>      [ ! -d "${pc}" ] && return 0
>      flock -s "${pc}".lock -c '
>          set -e
>          printenv | grep -q BB_VERBOSE_LOGS && set -x
>  
>          sudo find "${pc}" -type f -iname "*\.deb" -exec \
> -            ln -Pf -t "${rootfs}"/var/cache/apt/archives/ {} +
> +            ${cmd} "${rootfs}"/var/cache/apt/archives/ {} +
>      '
>  }
>  
> @@ -94,6 +101,13 @@ deb_dl_dir_export() {
>      export pc="${DEBDIR}/${2}"
>      export rootfs="${1}"
>      mkdir -p "${pc}"
> +
> +    # Use hardlinks with fallback to `cp` if different filesystems
> +    export cmd="ln -Pf"
> +    if [ "$(stat -c %m ${pc})" != "$(stat -c %m ${rootfs})" ]; then
> +        export cmd="cp -n"
> +    fi
> +
>      flock "${pc}".lock -c '
>          set -e
>          printenv | grep -q BB_VERBOSE_LOGS && set -x
> @@ -109,7 +123,7 @@ deb_dl_dir_export() {
>              if [ -n "$package" ]; then
>                  cmp --silent "$package" "$p" && continue
>              fi
> -            sudo ln -Pf "${p}" "${pc}"
> +            sudo ${cmd} "${p}" "${pc}"
>          done
>          sudo chown -R $(id -u):$(id -g) "${pc}"
>      '