RE: [PATCH v2 1/3] mmdebstrap: add "skip=cleanup/reproducible" option to mmdebstrap

["'Arjunan, Srinu' via isar-users" <isar-users@googlegroups.com>]

-----Original Message-----
From: Hombourger, Cedric (FT FDS CES LX) <cedric.hombourger@siemens.com> 
Sent: 09 March 2025 00:46
To: isar-users@googlegroups.com; Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Arjunan, Srinu (FT FDS CES LX PBU 2) <srinuvasan.a@siemens.com>
Cc: ubely@ilbers.de; Adithya.Balakumar@toshiba-tsip.com
Subject: Re: [PATCH v2 1/3] mmdebstrap: add "skip=cleanup/reproducible" option to mmdebstrap

On Sat, 2025-03-08 at 18:25 +0000, Arjunan, Srinu (FT FDS CES LX PBU 2)
wrote:
> 
> 
> -----Original Message-----
> From: Hombourger, Cedric (FT FDS CES LX) 
> <cedric.hombourger@siemens.com>
> Sent: 08 March 2025 19:15
> To: isar-users@googlegroups.com; Kiszka, Jan (FT RPD CED) 
> <jan.kiszka@siemens.com>; Arjunan, Srinu (FT FDS CES LX PBU 2) 
> <srinuvasan.a@siemens.com>
> Cc: ubely@ilbers.de; Adithya.Balakumar@toshiba-tsip.com
> Subject: Re: [PATCH v2 1/3] mmdebstrap: add 
> "skip=cleanup/reproducible" option to mmdebstrap
> 
> On Fri, 2025-03-07 at 22:59 +0100, Jan Kiszka wrote:
> > On 07.03.25 13:25, srinuvasan.a@siemens.com wrote:
> > > From: srinuvasan <srinuvasan.a@siemens.com>
> > > 
> > > Without this option, mmdebstrap deletes the var/log/dpkg.log and 
> > > its siblings by default, but this is needed to download deb-src 
> > > packages for base-apt creation.
> 
> I don't see where deb-src needs dpkg.log. Can you shed some light?
> Both the old code and the code I recently changed parse the .deb files 
> that were downloaded to determine which source packages should be 
> downloaded
> 
> Ref:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou
> ps.google.com%2Fg%2Fisar-users%2Fc%2FOwrSBmCPe0Y&data=05%7C02%7Csrinuv
> asan.a%40siemens.com%7C9492cda997b94dc4e81708dd5e759d85%7C38ae3bcd9579
> 4fd4addab42e1495d55a%7C1%7C0%7C638770581452433346%7CUnknown%7CTWFpbGZs
> b3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
> oiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=b49oZTatP9h6CxEys%2BPv%2
> F%2F%2F5v2BmE%2FX1%2FdhDPs%2F1qOY%3D&reserved=0
> 
> > > Basically, we are referring to the package's status under 
> > > /var/log/dpkg.log file and performing the debsrc_download 
> > > operation.
> > > 
> > > Without these changes, bootstrap related deb-src files are not 
> > > downloaded under the /downloads/deb-src folder, due to this the 
> > > repo sanity check functions failed during base-apt caching stage.
> 
> What are these repo sanity checks?
> Where do we see the failures?
> 
> From my base-apt workdir:
> 
> pwd && grep -ir error
> <...>/build/base-apt/tmp/work/debian-bookworm-any/base-apt/1.0-
> r0/temp
> run.repo.7287:          echo "ERROR: $*"
> 
> We probably need to be more specific if we are mentioning a failure.
> Without this, people on this list will not understand what problem(s) 
> you are fixing
> 
> 
> Hi cedric, jan
> 
> I will explain the issue in the details below:
> 
>                 To reproduce the issue ( Ref:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Filbers%2Fisar%2Fblob%2Fmaster%2Fdoc%2Fuser_manual.md%23creati
> on-of-local-apt-repo-caching-upstream-debian-packages&data=05%7C02%7Cs
> rinuvasan.a%40siemens.com%7C9492cda997b94dc4e81708dd5e759d85%7C38ae3bc
> d95794fd4addab42e1495d55a%7C1%7C0%7C638770581452457810%7CUnknown%7CTWF
> pbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsI
> kFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=3XId9a7IuBK9xyzYWnu
> dmeG0aKe3awlNSGSmwId7D%2BU%3D&reserved=0
>  ):
> 
>                 Trigger the warmup build to download deb and deb-src 
> packages ( Enable the BASE_REPO_FEATURES = "cache-deb-src" in
> conf/local.conf)
> 
>                 e.x: bitbake mc:qemuamd64-bookworm:isar-image-base
> 
>                 Once all the deb and deb-src downloaded for the 
> particular BSP under /download folder
> 
>                 Trigger the base-apt target to cache all the deb and 
> deb-src (Enable ISAR_USE_CACHED_BASE_REPO = "1" in conf/local.conf)
> 
>                 e.x: bitbake base-apt
> 
>                 You will see the failures:
>                 
>                 Logs: 
>                 
>                 NOTE: Resolving any missing task queue dependencies 
> Sstate summary: Wanted 0 Local 0 Mirrors 0 Missed 0 Current 0 (0% 
> match, 0% complete)#############################################            
> |
> ETA:  0:00:00
> Initialising tasks: 100%
> |####################################################################
> ################################################| Time: 0:00:00
> NOTE: No setscene tasks
> NOTE: Executing Tasks
> ERROR: base-apt-1.0-r0 do_cache: One or more sources are missing in 
> repo. bookworm adduser 3.134 pool/main/a/adduser/adduser_3.134_all.deb
> bookworm bmap-tools 3.6-2 pool/main/b/bmap-tools/bmap-tools_3.6-
> 2_all.deb
> bookworm debconf 1.5.82 pool/main/d/debconf/debconf_1.5.82_all.deb
> bookworm debian-archive-keyring 2023.3+deb12u1 pool/main/d/debian- 
> archive-keyring/debian-archive-keyring_2023.3+deb12u1_all.deb
> bookworm audit 1:3.0.9-1 pool/main/a/audit/libaudit-common_3.0.9-
> 1_all.deb
> bookworm libfile-find-rule-perl 0.34-3 pool/main/libf/libfile-find- 
> rule-perl/libfile-find-rule-perl_0.34-3_all.deb
> bookworm libnumber-compare-perl 0.03-3 pool/main/libn/libnumber- 
> compare-perl/libnumber-compare-perl_0.03-3_all.deb
> bookworm pam 1.5.2-6+deb12u1 pool/main/p/pam/libpam-runtime_1.5.2-
> 6+deb12u1_all.deb

.dsc for pam packages found in our downstream project

Yes cedric, validated from my end, it is available in downstream project because we have your changes in isar-patches (https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y)

But still we can reproduce the issue in vanilla ISAR without your patch, I started working on these changes before you proposed these changes (https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y)

Basically, your patches remove the is_not_part_of_current_build() function
And refactor the debsrc_download functionality.

 My patch becomes obsoleted if your (https://groups.google.com/g/isar-users/c/OwrSBmCPe0Y) changes are going to merge in ISAR.

Please provide your thoughts?

> bookworm libsemanage 3.4-1 pool/main/libs/libsemanage/libsemanage-
> common_3.4-1_all.deb
> bookworm libtext-glob-perl 0.11-3 pool/main/libt/libtext-glob- 
> perl/libtext-glob-perl_0.11-3_all.deb
> bookworm perl 5.36.0-7+deb12u1 pool/main/p/perl/perl-modules- 
> 5.36_5.36.0-7+deb12u1_all.deb bookworm six 1.16.0-4 
> pool/main/s/six/python3-six_1.16.0-4_all.deb
> bookworm tzdata 2024b-0+deb12u1 pool/main/t/tzdata/tzdata_2024b-
> 0+deb12u1_all.deb
> bookworm usrmerge 37~deb12u1
> pool/main/u/usrmerge/usrmerge_37~deb12u1_all.deb
> bookworm adduser 3.134 pool/main/a/adduser/adduser_3.134_all.deb
> 
> Just i captured few packages, but many of the deb packages don't have 
> their source files
> 
> In the base-apt recipes we are calling the repo_sanity_test function 
> finally to make sure all the deb's having the corresponding deb-src 
> packages should be present in the base-apt.

I checked again log.do_cache in our downstream project and I did not see any of the mentioned errors.

In downstream project we are not at all calling the repo_sanity_test function due to enabled the cache-deb-src in ROOTFS_POSTPROCESS_COMMAND rather than enabled in conf/local.conf

The below condition fails:
if [ "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'yes', 'no', d)}" = "yes" ]

that is the reason we are not calling this statement at all " reprepro -s -b "${dir}" --dbdir "${dbdir}" sourcemissing "

because we are not enabled the cache-deb-src via BASE_REPO_FEATURES


$ grep -v 'Export' ./tmp/work/debian-bookworm-any/base-apt/1.0-
r0/temp/log.do_cache
DEBUG: Executing python function do_cache
DEBUG: Using absolute /build/../repo/gnupg/205D601E.pub
DEBUG: Executing shell function repo
DEBUG: Shell function repo finished
DEBUG: Python function do_cache finished

That build was however done with the patch I have submitted earlier to this list. It also includes our own layer. I will do a plain Isar build (with my patch) to check if the issue is reproducible.

I took a few error cases from your example above and verified that the .dsc files are present in the generated base-apt build.

It should be noted that my patch removes "is_not_part_of_current_build()"

parsing dpkg.log was IMO not a great idea as I don't think its format is officially documented (I could be wrong though)

> 
>                 
> Yes, as you said, we are parsing deb files (from
> /var/cache/apt/archives) that were downloaded to determine which 
> source packages should be downloaded.
> but in the function "is_not_part_of_current_build "${package}" we are 
> skipping the download-src if parsing deb package (status installed) is 
> not available in the below files ```
>             "${IMAGE_ROOTFS}"/var/log/dpkg.log \
>             "${SCHROOT_HOST_DIR}"/var/log/dpkg.log \
>             "${SCHROOT_TARGET_DIR}"/var/log/dpkg.log \
>             "${SCHROOT_HOST_DIR}"/tmp/dpkg_common.log \
>             "${SCHROOT_TARGET_DIR}"/tmp/dpkg_common.log
> ```
> 
> As I mentioned above, the adduser package was installed in the stage 
> of bootstrap, but that package's status was not captured in the 
> bootstrap's /var/log/dpkg.log file.
> because mmdebstrap deletes those files by default, hence we didn't 
> capture those bootstrap package status.
> 
> As we know bootstrap rootfs is used for sbuildchroot creation, but 
> here whatever packages are installed during sbuild-chroot creation, 
> ${SCHROOT_TARGET_DIR}"/var/log/dpkg.log this file only contains the 
> sbuildchroot related package status, but our expectation is it should 
> have bootstrap + sbuildchroot package status available in 
> /var/log/dpkg.log file in sbuildchroot.
> 
> My first patch solves the above issue
> with the introduction of "skip=cleanup/reproducible", the bootstrap 
> rootfs generated with /var/log/dpkg.log files, which contain all the 
> bootstrap package status along with that if you use that boostrap for 
> sbuildchroot creation , the sbuildchroot rootfs contains the existing 
> bootstrap plus additionally installed sbuildchroot related packages 
> status is available in sbuldchroot /var/log/dpkg.log file
> 
> 
> My second patch resolve the below issue:
> 
> During image creation (IMAGE_FSTYPES = "wic") few packages were 
> installed in the session (e.g: gdisk, bmap-tools) but those packages 
> list available only in the sbuild environment, not outside, carry 
> those packages status to tmp/dpkg_common.log in the rootfs to get the 
> src packages for imager_install related deb packages, presently the 
> cache-deb-src function calls in the ROOTFS_POSTPROCESS_COMMAND but we 
> need to wait till imager generation That's why we pushed that 
> deb-src-caching after image creation.
> 
> My third path resolve the image rootfs related deb-src missing issue:
> 
> By default we are removing the /var/log/dpkg.log file in the 
> rootfs_postprocess_clean_log_files function, but this is needed when 
> We enable the BASE_REPO_FEATURES = "cache-deb-src", hence making this 
> function as a configurable one rather than enabled by default.
> to download IMAGE_ROOTFS related deb-src packages
> 
> I hope I explain the issue, presently am checking the reproducible 
> part (for jan queries), will update that part very soon.
> 
> Many thanks,
> Srinu
> 
> > > 
> > > Signed-off-by: srinuvasan <srinuvasan.a@siemens.com>
> > > ---
> > >  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 6 ++++++
> > >  1 file changed, 6 insertions(+)
> > > 
> > > diff --git a/meta/recipes-core/isar-mmdebstrap/isar-
> > > mmdebstrap.inc
> > > b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> > > index 931f6f13..7380a658 100644
> > > --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> > > +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> > > @@ -168,6 +168,11 @@ do_bootstrap() {
> > >          arch_param="$arch_param,${COMPAT_DISTRO_ARCH}"
> > >      fi
> > >  
> > > +    cleanup=
> > > +    if [ "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-
> > > src', 'yes', 'no', d)}" = "yes" ]; then
> > > +        cleanup="--skip=cleanup/reproducible"
> > > +    fi
> > > +
> > >      # Cleanup mounts if fails
> > >      trap 'exit 1' INT HUP QUIT TERM ALRM USR1
> > >      trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat
> > > "${WORKDIR}/mmtmpdir") \
> > > @@ -183,6 +188,7 @@ do_bootstrap() {
> > >  
> > >      sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args 
> > > \
> > >                     $arch_param \
> > > +                   $cleanup \
> > >                     --mode=unshare \
> > >                     ${MMHOOKS} \
> > >                     --setup-hook='mkdir -p 
> > > "$1/var/cache/apt/archives/"' \
> > 
> > What has changed in this version? I'm not seeing my concerns 
> > addressed, my questions answered.
> 
> +1
> 
> Please make sure new revisions of your patch series include a 
> changelog (use --cover-letter with git format-patch)
> 
> Sure
> 
> > 
> > Jan
> > 
> 
> 

--
Cedric Hombourger
Siemens AG
www.siemens.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/PUZPR06MB5747ED47D64E89DE5D9A70B388D72%40PUZPR06MB5747.apcprd06.prod.outlook.com.