[PATCH v2] dpkg: Account for empty local apt cache after package build

[PATCH v2] dpkg: Account for empty local apt cache after package build

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

A build dependency-free package combined with sstate caching and cleared
deb cache can trigger

| cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t /home/builder/example-raw/rootfs/var/cache/apt/archives/
| --------------------------------------------------------------------------------------------------------------------
|
| cp: cannot stat '/var/cache/apt/archives/*.deb': No such file or directory
|
| E: Command 'cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t /home/builder/example-raw/rootfs/var/cache/apt/archives/' failed to run.

Account for that.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

Changes in v2:
 - avoid returning error via negativ [ -n ] result

 meta/classes/dpkg.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index d89b0fd9..d61e9377 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -109,7 +109,7 @@ dpkg_runbuild() {
         --chroot-setup-commands="mkdir -p ${deb_dir}" \
         --chroot-setup-commands="ln -sf ${ext_deb_dir}/*.deb -t ${deb_dir}/" \
         --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \
-        --finished-build-commands="cp -Ln --no-preserve=owner ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
+        --finished-build-commands="[ -z "$(find ${deb_dir} -maxdepth 1 -name '*.deb' -print -quit)" ] || cp -Ln --no-preserve=owner ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
         --finished-build-commands="cp /var/log/dpkg.log ${ext_root}/dpkg_partial.log" \
         --debbuildopts="--source-option=-I" \
         --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
-- 
2.35.3

Re: [PATCH v2] dpkg: Account for empty local apt cache after package build

[MOESSBAUER, Felix]

On Thu, 2023-10-19 at 06:39 +0200, 'Jan Kiszka' via isar-users wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> A build dependency-free package combined with sstate caching and
> cleared
> deb cache can trigger
> 
> > cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t
> > /home/builder/example-raw/rootfs/var/cache/apt/archives/
> > -------------------------------------------------------------------
> > -------------------------------------------------
> > 
> > cp: cannot stat '/var/cache/apt/archives/*.deb': No such file or
> > directory
> > 
> > E: Command 'cp -Ln --no-preserve=owner
> > /var/cache/apt/archives/*.deb -t /home/builder/example-
> > raw/rootfs/var/cache/apt/archives/' failed to run.
> 
> Account for that.
> 

Yes, this is needed and blocks us updating ISAR.

Acked-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Felix

> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> Changes in v2:
>  - avoid returning error via negativ [ -n ] result
> 
>  meta/classes/dpkg.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
> index d89b0fd9..d61e9377 100644
> --- a/meta/classes/dpkg.bbclass
> +++ b/meta/classes/dpkg.bbclass
> @@ -109,7 +109,7 @@ dpkg_runbuild() {
>          --chroot-setup-commands="mkdir -p ${deb_dir}" \
>          --chroot-setup-commands="ln -sf ${ext_deb_dir}/*.deb -t
> ${deb_dir}/" \
>          --finished-build-commands="rm -f ${deb_dir}/sbuild-build-
> depends-main-dummy_*.deb" \
> -        --finished-build-commands="cp -Ln --no-preserve=owner
> ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
> +        --finished-build-commands="[ -z "$(find ${deb_dir} -maxdepth
> 1 -name '*.deb' -print -quit)" ] || cp -Ln --no-preserve=owner
> ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
>          --finished-build-commands="cp /var/log/dpkg.log
> ${ext_root}/dpkg_partial.log" \
>          --debbuildopts="--source-option=-I" \
>          --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
> -- 
> 2.35.3
> 

Re: [PATCH v2] dpkg: Account for empty local apt cache after package build

[Uladzimir Bely]

On Thu, 2023-10-19 at 06:39 +0200, 'Jan Kiszka' via isar-users wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> A build dependency-free package combined with sstate caching and
> cleared
> deb cache can trigger
> 
> > cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t
> > /home/builder/example-raw/rootfs/var/cache/apt/archives/
> > -------------------------------------------------------------------
> > -------------------------------------------------
> > 
> > cp: cannot stat '/var/cache/apt/archives/*.deb': No such file or
> > directory
> > 
> > E: Command 'cp -Ln --no-preserve=owner
> > /var/cache/apt/archives/*.deb -t /home/builder/example-
> > raw/rootfs/var/cache/apt/archives/' failed to run.
> 
> Account for that.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> Changes in v2:
>  - avoid returning error via negativ [ -n ] result
> 
>  meta/classes/dpkg.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 

Applied to next, thanks.

Re: [PATCH v2] dpkg: Account for empty local apt cache after package build

[Uladzimir Bely]

On Thu, 2023-10-19 at 06:39 +0200, 'Jan Kiszka' via isar-users wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> A build dependency-free package combined with sstate caching and
> cleared
> deb cache can trigger
> 
> > cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t
> > /home/builder/example-raw/rootfs/var/cache/apt/archives/
> > -------------------------------------------------------------------
> > -------------------------------------------------
> > 
> > cp: cannot stat '/var/cache/apt/archives/*.deb': No such file or
> > directory
> > 
> > E: Command 'cp -Ln --no-preserve=owner
> > /var/cache/apt/archives/*.deb -t /home/builder/example-
> > raw/rootfs/var/cache/apt/archives/' failed to run.
> 
> Account for that.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> Changes in v2:
>  - avoid returning error via negativ [ -n ] result
> 
>  meta/classes/dpkg.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
> index d89b0fd9..d61e9377 100644
> --- a/meta/classes/dpkg.bbclass
> +++ b/meta/classes/dpkg.bbclass
> @@ -109,7 +109,7 @@ dpkg_runbuild() {
>          --chroot-setup-commands="mkdir -p ${deb_dir}" \
>          --chroot-setup-commands="ln -sf ${ext_deb_dir}/*.deb -t
> ${deb_dir}/" \
>          --finished-build-commands="rm -f ${deb_dir}/sbuild-build-
> depends-main-dummy_*.deb" \
> -        --finished-build-commands="cp -Ln --no-preserve=owner
> ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
> +        --finished-build-commands="[ -z "$(find ${deb_dir} -maxdepth
> 1 -name '*.deb' -print -quit)" ] || cp -Ln --no-preserve=owner
> ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
>          --finished-build-commands="cp /var/log/dpkg.log
> ${ext_root}/dpkg_partial.log" \
>          --debbuildopts="--source-option=-I" \
>          --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
> -- 
> 2.35.3
> 

When experimenting with base-apt, I found that this patch causes build
issues on second (cached) build.

Debug shows that new logic prevents exporting packages downloaded by
sbuild from internal layer to external directory.

It's reproducible for me when I use kas-container.

```
isar $./kas/kas-container menu #Just save default config here
isar $ ./kas/kas-container shell
builder@6026019cd1fb:/build$ bitbake -v hello
...
I: Finished running 'rm -f /var/cache/apt/archives/sbuild-build-
depends-main-dummy_*.deb'.
[ -z  ] || cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t
/home/builder/hello/rootfs/var/cache/apt/archives/
I: Finished running '[ -z  ] || cp -Ln --no-preserve=owner
/var/cache/apt/archives/*.deb -t
/home/builder/hello/rootfs/var/cache/apt/archives/'.
...
```

The problem is in empty condition `[ -z  ]`.

I added some "--finished-build-commands" before existing lines for
debugging:

```
        --finished-build-commands="echo =-=-=" \
        --finished-build-commands="find ${deb_dir} -maxdepth 1 -name
'*.deb' -print -quit" \
        --finished-build-commands="echo $(find ${deb_dir} -maxdepth 1 -
name '*.deb' -print -quit)" \

```

And that's what I see:

```
+----------------------------------------------------------------------
--------+
| Finished Timed Build Commands                                       
|
+----------------------------------------------------------------------
--------+
echo =-=-=
----------
=-=-=
I: Finished running 'echo =-=-='.
find /var/cache/apt/archives -maxdepth 1 -name '*.deb' -print -quit
-------------------------------------------------------------------
/var/cache/apt/archives/libgcc-s1_12.2.0-14_amd64.deb
I: Finished running 'find /var/cache/apt/archives -maxdepth 1 -name
'*.deb' -print -quit'.
echo 
-----
I: Finished running 'echo '.
rm -f /var/cache/apt/archives/sbuild-build-depends-main-dummy_*.deb
-------------------------------------------------------------------
I: Finished running 'rm -f /var/cache/apt/archives/sbuild-build-
depends-main-dummy_*.deb'.
[ -z  ] || cp -Ln --no-preserve=owner /var/cache/apt/archives/*.deb -t
/home/builder/hello/rootfs/var/cache/apt/archives/
-----------------------------------------------------------------------
-------------------------------------------------
I: Finished running '[ -z  ] || cp -Ln --no-preserve=owner
/var/cache/apt/archives/*.deb -t
/home/builder/hello/rootfs/var/cache/apt/archives/'.
```

E.g., "find" itself works (libgcc-s1_12.2.0-14_amd64.deb is found), but
in form of $(find  ) output becomes empty for some reasons, and no
packages are exported.

The issue is not reproducable in simple bullseye chroot (that's why it
was not caught by CI), but it's reproducable under kas container. Even
if I enter kas-container shell and then manually clone isar inside it
in order to try common (`. isar-init-build-env`) way of building isar.