On Thu, 2025-03-20 at 18:02 +0000, cedric.hombourger@siemens.com wrote: > On Thu, 2025-03-20 at 20:56 +0300, Uladzimir Bely wrote: > > On Thu, 2025-03-20 at 07:18 +0000, Arjunan, Srinu wrote: > > > > > > > > > -----Original Message----- > > > From: Uladzimir Bely > > > Sent: 20 March 2025 11:02 > > > To: Arjunan, Srinu (FT FDS CES LX PBU 2) > > > ; > > > isar-users@googlegroups.com > > > Cc: Kiszka, Jan (FT RPD CED) ; > > > Hombourger, > > > Cedric (FT FDS CES LX) ; > > > Niedermayr, > > > Benedikt (FT RPD CED OES-DE) ; > > > Adithya.Balakumar@toshiba-tsip.com > > > Subject: Re: [PATCH 1/1] image: move deb-src-caching > > > functionality > > > after image creation > > > > > > On Tue, 2025-03-18 at 16:45 +0000, Arjunan, Srinu wrote: > > > > > > > > > > > > -----Original Message----- > > > > From: Arjunan, Srinu (FT FDS CES LX PBU 2) > > > > > > > > Sent: 10 March 2025 17:22 > > > > To: isar-users@googlegroups.com > > > > Cc: Kiszka, Jan (FT RPD CED) ; > > > > Hombourger, > > > > Cedric (FT FDS CES LX) ; > > > > ubely@ilbers.de; Niedermayr, Benedikt (FT RPD CED OES-DE) > > > > ; > > > > Adithya.Balakumar@toshiba-tsip.com; Arjunan, Srinu (FT FDS CES > > > > LX > > > > PBU > > > > 2) > > > > Subject: [PATCH 1/1] image: move deb-src-caching functionality > > > > after > > > > image creation > > > > > > > > From: srinuvasan > > > > > > > > When we do deb-src caching, we are missing to cache imager > > > > related > > > > packages, because the deb-src-caching functionality runs in the > > > > stage > > > > of ROOTFS_POSTPROCESS_COMAMND, but image creation task runs > > > > after > > > > the > > > > rootfs creation, hence imager related packages are missing in > > > > rootfs's > > > > var/cache/apt/archives. > > > > > > > > To solve this issue run the deb-src-caching after image > > > > creation, > > > > now > > > > we can able to download deb-src for all the deb packages. > > > > > > > > Signed-off-by: srinuvasan > > > > --- > > > >  meta/classes/image.bbclass  | 8 +++++++- > > > > meta/classes/rootfs.bbclass > > > > > 3 ++- > > > >  2 files changed, 9 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/meta/classes/image.bbclass > > > > b/meta/classes/image.bbclass > > > > index 56eca202..66b4d313 100644 > > > > --- a/meta/classes/image.bbclass > > > > +++ b/meta/classes/image.bbclass > > > > @@ -72,7 +72,13 @@ ROOTFS_MANIFEST_DEPLOY_DIR ?= > > > > "${DEPLOY_DIR_IMAGE}" > > > >  ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" > > > >  ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}" > > > > > > > > -ROOTFS_POSTPROCESS_COMMAND:prepend = > > > > "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', > > > > 'cache_deb_src', '', d)} " > > > > +CACHE_DEB_SRC = "${@bb.utils.contains('BASE_REPO_FEATURES', > > > > 'cache- > > > > deb-src', '1', '0', d)}" > > > > + > > > > +python () { > > > > +    if bb.utils.to_boolean(d.getVar('CACHE_DEB_SRC')): > > > > +        bb.build.addtask('do_cache_deb_src', 'do_deploy', > > > > 'do_image', > > > > +d) } > > > > + > > > > > > > >  inherit rootfs > > > >  inherit sdk > > > > diff --git a/meta/classes/rootfs.bbclass > > > > b/meta/classes/rootfs.bbclass > > > > index 2348e269..c9e832dc 100644 > > > > --- a/meta/classes/rootfs.bbclass > > > > +++ b/meta/classes/rootfs.bbclass > > > > @@ -272,7 +272,8 @@ python do_rootfs_install() {  }  addtask > > > > rootfs_install before do_rootfs_postprocess after do_unpack > > > > > > > > -cache_deb_src() { > > > > +do_cache_deb_src[network] = "${TASK_USE_SUDO}" > > > > +do_cache_deb_src() { > > > >      if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] || > > > >         [ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then > > > >          sudo mv "${ROOTFSDIR}"/etc/resolv.conf > > > > "${ROOTFSDIR}"/etc/resolv.conf.isar > > > > -- > > > > 2.39.5 > > > > > > > > Hi All, > > > > > > > >           Can you please update the status of this patch?, or > > > > anything > > > > missing from my end? > > > > > > > > > > Hello. > > > > > > Sorry for delays related to "installer" currently breaking full > > > CI > > > on > > > "next" branch. > > > > > > Tested recently your patch, it also makes CI fail at: > > > > > > CrossTest.test_cross_debsrc: > > > > > > ``` > > > [stdlog] 2025-03-19 21:29:50,542 avocado.app ERROR| ERROR: Task > > > (mc:qemuarm64- > > > bookworm:/workspace/build/isar_ub_devel_fast/214/meta- > > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) > > > failed > > > with exit code '1' > > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | DEBUG: > > > Executing shell function do_cache_deb_src [stdlog] 2025-03-19 > > > 21:30:44,921 avocado.test INFO | | Reading package lists... > > > [stdlog] 2025-03-19 21:30:44,921 avocado.test INFO | | E: You > > > must > > > put some 'deb-src' URIs in your sources.list ``` > > > > > > NoCrossTest.test_nocross_debsrc: > > > > > > ``` > > > [stdlog] 2025-03-20 01:09:35,759 avocado.app ERROR| ERROR: > > > Logfile > > > of > > > failure stored in: > > > /workspace/build/isar_ub_devel/126/build/tmp/work/debian- > > > bookworm- > > > armhf/isar-image-ci-qemuarm/1.0- > > > r0/temp/log.do_cache_deb_src.218291 > > > [stdlog] 2025-03-20 01:09:35,832 avocado.app ERROR| ERROR: Task > > > (mc:qemuarm-bookworm:/workspace/build/isar_ub_devel/126/meta- > > > test/recipes-core/images/isar-image-ci.bb:do_cache_deb_src) > > > failed > > > with exit code '1' > > > [stdlog] 2025-03-20 01:19:35,822 avocado.test INFO | | DEBUG: > > > Executing shell function do_cache_deb_src [stdlog] 2025-03-20 > > > 01:19:35,823 avocado.test INFO | | Reading package lists... > > > [stdlog] 2025-03-20 01:19:35,823 avocado.test INFO | | E: You > > > must > > > put some 'deb-src' URIs in your sources.list ``` > > > > > > > > > Hi Uladzimir, > > > > > >                       Thank you for the reply, I am not able to > > > reproduce the build issue what you reported , I triggered the > > > build > > > for isar-image-ci target and it's getting passed. > > > > > > This patch depend on Cedric patch [1] > > > [1]: > > > https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y > > > > > > I validated my patch along with the [1], it seems working fine. > > > > > > Or Can you please provide the steps to reproduce the issue? > > > > > > > I tried to find easy way to reproduce (e.g., one target case), but > > didn't find it - build was OK. But it fails in CI - most probably, > > due > > to specific sequence of tests, that share sstate cache and download > > directory between each other. > > > > Similar behavior is reproduced in CI also with other patches, like > > "deb-dl-dir: remove excessive calls to dpkg-deb in > > debsrc_download". > > Building some target alone is OK, but under CI it fails. > > Ah damn it! > > I had checked with ./scripts/ci_build.sh -T fast. I may have to try > with the full CI (without -T fast) > > Is the error exactly the same when you tried with my patch (and only > that one)? No, you patch I've mentioned failed in "cache-deb-src" task, but in NoCrossTest.test_nocross_rpi_debsrc, and with a different error: [stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | DEBUG: Executing shell function cache_deb_src [stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | [stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | W: Unable to read directory /workspace/build/isar_ub_devel/119/build/tmp/work/raspios-bookworm- armhf/isar-image-base-rpi-arm-v7l/1.0- r0/rootfs/var/cache/apt/archives/partial [stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | Reading package lists... [stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | E: Can not find version '2023.3+deb12u1' of package 'debian-archive-keyring' [stdlog] 2025-03-07 12:48:01,379 avocado.test INFO | | E: Unable to find a source package for debian-archive-keyring > > Would you be able to send a full CI log? > Attached archive with both logs: isar_ub_devel_119: full CI logs with "deb-dl-dir: remove excessive calls to dpkg-deb in debsrc_download" applied. isar_ub_devel_fast_214: fast CI logs with "image: move deb-src- caching functionality after image creation" applied. Some additional info. For the latest patch, I tried to debug it: - added `BASE_REPO_FEATURES = "cache-deb-src"` to local.conf (that is the line CrossTest.test_cross_debsrc test adds; the line was removed in the followed tests). - chrooted to "isar-image-ci-qemuarm64" rootfs, where error happend. - executed "apt-get -y --download-only --only-source source gcc- 12=12.2.0-14 download-src" - the same that failed in CI. Result: ``` # apt-get -y --download-only --only-source source gcc-12=12.2.0-14 download-src Reading package lists... Done E: You must put some 'deb-src' URIs in your sources.list ``` At the same time, sources.list looked fine: ``` # cat /etc/apt/sources.list.d/bootstrap.list deb     http://deb.debian.org/debian bookworm main contrib non-free- firmware non-free deb-src http://deb.debian.org/debian bookworm main contrib non-free- firmware non-free deb     http://deb.debian.org/debian-security bookworm-security main contrib non-free-firmware non-free deb-src http://deb.debian.org/debian-security bookworm-security main contrib non-free-firmware non-free deb     http://deb.debian.org/debian bookworm-updates main contrib non- free-firmware non-free deb-src http://deb.debian.org/debian bookworm-updates main contrib non- free-firmware non-free ``` After executing "apt-get update" in this chroot, error gone. It looks like despite proper sources list, internal apt cache was not complete and didn't know anything about "deb-src" records. Why - I still don't clearly understand. > > > > > > Many thanks, > > > > Srinu > > > > > > > > > > -- > > > Best regards, > > > Uladzimir. > > > > -- > Cedric Hombourger > Siemens AG > http://www.siemens.com/ -- Best regards, Uladzimir. -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/def800632029c0f3f6903a77df4f7a12042badf2.camel%40ilbers.de.