[PATCH] Fix cache-deb-src functionality for non-amd64 builds

[PATCH] Fix cache-deb-src functionality for non-amd64 builds

[Anton Mikanovich]

Commit f0da749f introduced `apt-cache -o Dir=${rootfs} dumpavail`
method to get info about packages installed in rootfs, instead of
parsing dpkg.log previuosly used.

But it does not work in case of foreing architecture used. The list
comes empty and no source packages get downloaded.

Instead, chroot into the rootfs and run this inside. The difference:

Works for the same arch:
  apt-cache -o Dir=/build/tmp/work/debian-bookworm-amd64/isar-image-base-qemuamd64/1.0-r0/rootfs dumpavail | wc -l
  1190397

Doesn't work for the foreign arch:
  apt-cache -o Dir=/build/tmp/work/debian-bookworm-arm64/isar-image-base-qemuarm64/1.0-r0/rootfs/ dumpavail | wc -l
  0

After the patch:
  sudo chroot /build/tmp/work/debian-bookworm-arm64/isar-image-base-qemuarm64/1.0-r0/rootfs/ apt-cache dumpavail | wc -l
  1170789

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/deb-dl-dir.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 3f560da4..1f3876db 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -68,7 +68,9 @@ debsrc_download() {
     trap "rm -f ${avail} ${wanted}" EXIT
 
     # List all packages known to apt
-    apt-cache -o Dir=${rootfs} dumpavail | debsrc_source_version_filter > ${avail}
+    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
+        apt-cache dumpavail \
+        | debsrc_source_version_filter > ${avail}
 
     # Use apt-ftparchive to scan all .deb files found in the download directory
     # and get the <source> <version> pairs that we wish to download
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250717105159.1989307-1-amikan%40ilbers.de.

Re: [PATCH] Fix cache-deb-src functionality for non-amd64 builds

['cedric.hombourger@siemens.com' via isar-users]

On Thu, 2025-07-17 at 13:51 +0300, Anton Mikanovich wrote:
> Commit f0da749f introduced `apt-cache -o Dir=${rootfs} dumpavail`
> method to get info about packages installed in rootfs, instead of
> parsing dpkg.log previuosly used.

previously

> 
> But it does not work in case of foreing architecture used. The list

foreign

> comes empty and no source packages get downloaded.
> 
> Instead, chroot into the rootfs and run this inside. The difference:

Yes but with a significant performance hit (QEMU)
I can recheck our options. This is a reminder that I had earlier
proposed
https://lists.isar-build.org/isar-users/20250515150727.1764989-1-cedric.hombourger@siemens.com/T/#t
(probably does not address this specific issue though)

> 
> Works for the same arch:
>   apt-cache -o Dir=/build/tmp/work/debian-bookworm-amd64/isar-image-
> base-qemuamd64/1.0-r0/rootfs dumpavail | wc -l
>   1190397
> 
> Doesn't work for the foreign arch:
>   apt-cache -o Dir=/build/tmp/work/debian-bookworm-arm64/isar-image-
> base-qemuarm64/1.0-r0/rootfs/ dumpavail | wc -l
>   0
> 
> After the patch:
>   sudo chroot /build/tmp/work/debian-bookworm-arm64/isar-image-base-
> qemuarm64/1.0-r0/rootfs/ apt-cache dumpavail | wc -l
>   1170789
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  meta/classes/deb-dl-dir.bbclass | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-
> dir.bbclass
> index 3f560da4..1f3876db 100644
> --- a/meta/classes/deb-dl-dir.bbclass
> +++ b/meta/classes/deb-dl-dir.bbclass
> @@ -68,7 +68,9 @@ debsrc_download() {
>      trap "rm -f ${avail} ${wanted}" EXIT
>  
>      # List all packages known to apt
> -    apt-cache -o Dir=${rootfs} dumpavail |
> debsrc_source_version_filter > ${avail}
> +    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
> +        apt-cache dumpavail \
> +        | debsrc_source_version_filter > ${avail}
>  
>      # Use apt-ftparchive to scan all .deb files found in the
> download directory
>      # and get the <source> <version> pairs that we wish to download
> -- 
> 2.34.1
> 

-- 
Cedric Hombourger
Siemens AG
www.siemens.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/df56c2824af6fa857374c4f2631983aae5f9585c.camel%40siemens.com.

Re: [PATCH] Fix cache-deb-src functionality for non-amd64 builds

['cedric.hombourger@siemens.com' via isar-users]

On Tue, 2025-07-22 at 07:27 +0200, Cedric Hombourger wrote:
> On Thu, 2025-07-17 at 13:51 +0300, Anton Mikanovich wrote:
> > Commit f0da749f introduced `apt-cache -o Dir=${rootfs} dumpavail`
> > method to get info about packages installed in rootfs, instead of
> > parsing dpkg.log previuosly used.
> 
> previously
> 
> > 
> > But it does not work in case of foreing architecture used. The list
> 
> foreign
> 
> > comes empty and no source packages get downloaded.
> > 
> > Instead, chroot into the rootfs and run this inside. The
> > difference:
> 
> Yes but with a significant performance hit (QEMU)
> I can recheck our options. This is a reminder that I had earlier
> proposed
> https://lists.isar-build.org/isar-users/20250515150727.1764989-1-cedric.hombourger@siemens.com/T/#t
> (probably does not address this specific issue though)

That issue did ring a bell and I was able to find the fix I had
suggested earlier "specify target architecture for apt-cache lookups"


https://lists.isar-build.org/isar-users/20250417201747.90299-1-cedric.hombourger@siemens.com/

> 
> > 
> > Works for the same arch:
> >   apt-cache -o Dir=/build/tmp/work/debian-bookworm-amd64/isar-
> > image-
> > base-qemuamd64/1.0-r0/rootfs dumpavail | wc -l
> >   1190397
> > 
> > Doesn't work for the foreign arch:
> >   apt-cache -o Dir=/build/tmp/work/debian-bookworm-arm64/isar-
> > image-
> > base-qemuarm64/1.0-r0/rootfs/ dumpavail | wc -l
> >   0
> > 
> > After the patch:
> >   sudo chroot /build/tmp/work/debian-bookworm-arm64/isar-image-
> > base-
> > qemuarm64/1.0-r0/rootfs/ apt-cache dumpavail | wc -l
> >   1170789
> > 
> > Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> > ---
> >  meta/classes/deb-dl-dir.bbclass | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-
> > dir.bbclass
> > index 3f560da4..1f3876db 100644
> > --- a/meta/classes/deb-dl-dir.bbclass
> > +++ b/meta/classes/deb-dl-dir.bbclass
> > @@ -68,7 +68,9 @@ debsrc_download() {
> >      trap "rm -f ${avail} ${wanted}" EXIT
> >  
> >      # List all packages known to apt
> > -    apt-cache -o Dir=${rootfs} dumpavail |
> > debsrc_source_version_filter > ${avail}
> > +    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
> > +        apt-cache dumpavail \
> > +        | debsrc_source_version_filter > ${avail}
> >  
> >      # Use apt-ftparchive to scan all .deb files found in the
> > download directory
> >      # and get the <source> <version> pairs that we wish to
> > download
> > -- 
> > 2.34.1
> > 
> 

-- 
Cedric Hombourger
Siemens AG
www.siemens.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/1fe9d1d61cf2db06b18823d331fa5d13a4c21bf5.camel%40siemens.com.

Re: [PATCH] Fix cache-deb-src functionality for non-amd64 builds

['MOESSBAUER, Felix' via isar-users]

On Tue, 2025-07-22 at 06:35 +0000, 'cedric.hombourger@siemens.com' via
isar-users wrote:
> On Tue, 2025-07-22 at 07:27 +0200, Cedric Hombourger wrote:
> > On Thu, 2025-07-17 at 13:51 +0300, Anton Mikanovich wrote:
> > > Commit f0da749f introduced `apt-cache -o Dir=${rootfs} dumpavail`
> > > method to get info about packages installed in rootfs, instead of
> > > parsing dpkg.log previuosly used.
> > 
> > previously
> > 
> > > 
> > > But it does not work in case of foreing architecture used. The
> > > list
> > 
> > foreign
> > 
> > > comes empty and no source packages get downloaded.
> > > 
> > > Instead, chroot into the rootfs and run this inside. The
> > > difference:
> > 
> > Yes but with a significant performance hit (QEMU)
> > I can recheck our options. This is a reminder that I had earlier
> > proposed
> > https://lists.isar-build.org/isar-users/20250515150727.1764989-1-cedric.hombourger@siemens.com/T/#t
> > (probably does not address this specific issue though)
> 
> That issue did ring a bell and I was able to find the fix I had
> suggested earlier "specify target architecture for apt-cache lookups"
> 
> 
> https://lists.isar-build.org/isar-users/20250417201747.90299-1-cedric.hombourger@siemens.com/

If that works here as well, it would be WAY better than falling back to
a changeroot.

Felix

> 
> 
> -- 
> Cedric Hombourger
> Siemens AG
> www.siemens.com

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/3223fea8d6e6cad1748a18a391b2aa2a06849023.camel%40siemens.com.

Re: [PATCH] Fix cache-deb-src functionality for non-amd64 builds

[Anton Mikanovich]

22/07/2025 09:35, cedric.hombourger@siemens.com wrote:
> That issue did ring a bell and I was able to find the fix I had
> suggested earlier "specify target architecture for apt-cache lookups"
>
>
> https://lists.isar-build.org/isar-users/20250417201747.90299-1-cedric.hombourger@siemens.com/
>
Thanks. We will check the original issue with this patch, but probably
PACKAGE_ARCH should be used instead, to properly download sources for 
:compat
packages.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/8ec6cf49-cbc8-4f21-9248-a0dd829ac67b%40ilbers.de.