Re: [PATCH 1/1] image: move deb-src-caching functionality after image creation

[Uladzimir Bely <ubely@ilbers.de>]

On Thu, 2025-03-20 at 07:18 +0000, Arjunan, Srinu wrote:
> 
> 
> -----Original Message-----
> From: Uladzimir Bely <ubely@ilbers.de> 
> Sent: 20 March 2025 11:02
> To: Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>;
> isar-users@googlegroups.com
> Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger,
> Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; Niedermayr,
> Benedikt (FT RPD CED OES-DE) <benedikt.niedermayr@siemens.com>;
> Adithya.Balakumar@toshiba-tsip.com
> Subject: Re: [PATCH 1/1] image: move deb-src-caching functionality
> after image creation
> 
> On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote:
> > 
> > 
> > -----Original Message-----
> > From: Arjunan, Srinu (FT FDS CES LX PBU 2)
> > <srinuvasan.a@siemens.com>
> > Sent: 10 March 2025 17:22
> > To: isar-users@googlegroups.com
> > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Hombourger, 
> > Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com>; 
> > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE) 
> > <benedikt.niedermayr@siemens.com>;
> > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES LX
> > PBU
> > 2) <srinuvasan.a@siemens.com>
> > Subject: [PATCH 1/1] image: move deb-src-caching functionality
> > after 
> > image creation
> > 
> > From: srinuvasan <srinuvasan.a@siemens.com>
> > 
> > When we do deb-src caching, we are missing to cache imager related 
> > packages, because the deb-src-caching functionality runs in the
> > stage 
> > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs after
> > the 
> > rootfs creation, hence imager related packages are missing in
> > rootfs's 
> > var/cache/apt/archives.
> > 
> > To solve this issue run the deb-src-caching after image creation,
> > now 
> > we can able to download deb-src for all the deb packages.
> > 
> > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > ---
> >  meta/classes/image.bbclass  | 8 +++++++-
> > meta/classes/rootfs.bbclass 
> > > 3 ++-
> >  2 files changed, 9 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/image.bbclass
> > b/meta/classes/image.bbclass 
> > index 56eca202..66b4d313 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?=
> > "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
> >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
> >  
> > -ROOTFS_POSTPROCESS_COMMAND:prepend =
> > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 
> > 'cache_deb_src', '', d)} "
> > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES',
> > 'cache-
> > deb-src', '1', '0', d)}"
> > +
> > +python () {
> > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')):
> > +        bb.build.addtask('do_cache_deb_src', 'do_deploy',
> > 'do_image',
> > +d) }
> > +
> >  
> >  inherit rootfs
> >  inherit sdk
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass 
> > index 2348e269..c9e832dc 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask 
> > rootfs_install before do_rootfs_postprocess after do_unpack
> >  
> > -cache_deb_src() {
> > +do_cache_deb_src[network] = "${TASK_USE_SUDO}"
> > +do_cache_deb_src() {
> >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
> >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
> >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf 
> > "${ROOTFSDIR}"/etc/resolv.conf.isar
> > --
> > 2.39.5
> > 
> > Hi All,
> > 
> >           Can you please update the status of this patch?, or
> > anything 
> > missing from my end?
> > 
> 
> Hello.
> 
> Sorry for delays related to "installer" currently breaking full CI on
> "next" branch.
> 
> Tested recently your patch, it also makes CI fail at:
> 
> CrossTest.test_cross_debsrc:
> 
> ```
> [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task
> (mc:qemuarm64-bookworm:/workspace/build/isar_ub_devel_fast/214/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-19
> 21:30:44,921 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
> 
> NoCrossTest.test_nocross_debsrc:
> 
> ```
> [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: Logfile of
> failure stored in:
> /workspace/build/isar_ub_devel/126/build/tmp/work/debian-bookworm-
> armhf/isar-image-ci-qemuarm/1.0-r0/temp/log.do_cache_deb_src.218291
> [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task
> (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta-
> test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) failed
> with exit code '1'
> [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG:
> Executing shell function do_cache_deb_src [stdlog] 2025-03-20
> 01:19:35,823 avocado.test INFO | | Reading package lists...
> [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You must
> put some 'deb-src' URIs in your sources.list ```
> 
> 
> Hi Uladzimir,
> 
>                       Thank you for the reply, I am not able to
> reproduce the build issue what you reported , I triggered the build
> for isar-image-ci target and it's getting passed.
> 
> This patch depend on Cedric patch [1]
> [1]: https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y
> 
> I validated my patch along with the [1], it seems working fine.
> 
> Or Can you please provide the steps to reproduce the issue?
> 

I tried to find easy way to reproduce (e.g., one target case), but
didn't find it - build was OK. But it fails in CI - most probably, due
to specific sequence of tests, that share sstate cache and download
directory between each other.

Similar behavior is reproduced in CI also with other patches, like
"deb-dl-dir: remove excessive calls to dpkg-deb in debsrc_download".
Building some target alone is OK, but under CI it fails.

> > Many thanks,
> > Srinu
> > 
> 
> --
> Best regards,
> Uladzimir.

-- 
Best regards,
Uladzimir.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/362f19d61f23ad0f1c218476fe9f939b2b70a68f.camel%40ilbers.de.