Re: [PATCH] image.bbclass: fix non-reproducible file time-stamps inside rootfs image

[Henning Schild <henning.schild@siemens.com>]

Am Wed, 4 Jan 2023 13:48:10 +0000
schrieb <Venkata.Pyla@toshiba-tsip.com>:

> >-----Original Message-----
> >From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> >Behalf Of Henning Schild
> >Sent: 04 January 2023 15:00
> >To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting) <Venkata.Pyla@toshiba-  
> >tsip.com>  
> >Cc: isar-users@googlegroups.com; amikan@ilbers.de;
> >jan.kiszka@siemens.com; hayashi kazuhiro(林 和宏 □ＳＷＣ◯ＡＣＴ)
> ><kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(ＴＳＩＰ TMIEC ODG
> >Porting) <dinesh.kumar@toshiba-tsip.com>; Baurzhan Ismagulov
> ><ibr@radix50.net> Subject: Re: [PATCH] image.bbclass: fix
> >non-reproducible file time-stamps inside rootfs image
> >
> >Am Wed, 4 Jan 2023 07:54:44 +0000
> >schrieb <Venkata.Pyla@toshiba-tsip.com>:
> >  
> >> >-----Original Message-----
> >> >From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> >> >Behalf Of Henning Schild
> >> >Sent: 04 January 2023 00:36
> >> >To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting)
> >> ><Venkata.Pyla@toshiba- tsip.com>  
> >> >Cc: isar-users@googlegroups.com; amikan@ilbers.de;
> >> >jan.kiszka@siemens.com; hayashi kazuhiro(林 和宏 □ＳＷＣ◯ＡＣＴ)
> >> ><kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(ＴＳＩＰ TMIEC ODG
> >> >Porting) <dinesh.kumar@toshiba-tsip.com> Subject: Re: [PATCH]
> >> >image.bbclass: fix non-reproducible file time-stamps inside rootfs
> >> >image
> >> >
> >> >Am Tue, 3 Jan 2023 14:10:14 +0000
> >> >schrieb <Venkata.Pyla@toshiba-tsip.com>:
> >> >  
> >> >> >-----Original Message-----
> >> >> >From: isar-users@googlegroups.com
> >> >> ><isar-users@googlegroups.com> On Behalf Of Henning Schild
> >> >> >Sent: 02 January 2023 22:14
> >> >> >To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting)
> >> >> ><Venkata.Pyla@toshiba- tsip.com>
> >> >> >Cc: isar-users@googlegroups.com; amikan@ilbers.de;
> >> >> >jan.kiszka@siemens.com; hayashi kazuhiro(林 和宏
> >> >> >□ＳＷＣ◯ＡＣＴ) <kazuhiro3.hayashi@toshiba.co.jp>; dinesh
> >> >> >kumar(ＴＳＩＰ TMIEC ODG Porting)
> >> >> ><dinesh.kumar@toshiba-tsip.com> Subject: Re: [PATCH]
> >> >> >image.bbclass: fix non-reproducible file time-stamps inside
> >> >> >rootfs image
> >> >> >
> >> >> >Am Mon,  2 Jan 2023 20:28:28 +0530 schrieb
> >> >> >venkata.pyla@toshiba-tsip.com:
> >> >> >  
> >> >> >> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >> >> >>
> >> >> >> As part of reproducible-build work, the rootfs images
> >> >> >> generated on same source should be identical between two
> >> >> >> builds.
> >> >> >>
> >> >> >> In this commit it tries to solve one of the non-reproducible
> >> >> >> problem i.e. the rootfs file time-stamps generated during
> >> >> >> build time are not reproducible, it uses one of the solution
> >> >> >> provided in the debian live-build image project (refer [1]),
> >> >> >> it fixes by finding all the files/folders that are
> >> >> >> gernerated newly and set the time-stamp provided by
> >> >> >> `SOURCE_DATE_EPOCH` environment variable.
> >> >> >>
> >> >> >> [1]
> >> >> >> https://salsa.debian.org/live-team/live-build/-/merge_requests/2
> >> >> >> 18
> >> >> >>
> >> >> >> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >> >> >> ---
> >> >> >>  meta/classes/image.bbclass | 9 +++++++++
> >> >> >>  1 file changed, 9 insertions(+)
> >> >> >>
> >> >> >> diff --git a/meta/classes/image.bbclass
> >> >> >> b/meta/classes/image.bbclass index 813e1f3..f592a12 100644
> >> >> >> --- a/meta/classes/image.bbclass
> >> >> >> +++ b/meta/classes/image.bbclass
> >> >> >> @@ -430,6 +430,15 @@ do_rootfs_finalize() {
> >> >> >>              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
> >> >> >>
> >> >> >>          rm -f "${ROOTFSDIR}/etc/apt/sources-list"
> >> >> >> +
> >> >> >> +        # Set same time-stamps to the newly generated
> >> >> >> file/folders in the
> >> >> >> +        # rootfs image for the purpose of reproducible
> >> >> >> builds.
> >> >> >> +        test ! -z "${SOURCE_DATE_EPOCH}" && \
> >> >> >> +            find ${ROOTFSDIR} -newermt \
> >> >> >> +                "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d
> >> >> >> %H:%M:%S')" \
> >> >> >> +                -printf "%y %p\n" \
> >> >> >> +                -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH}
> >> >> >> ';'
> >> >> >> +  
> >> >> >
> >> >> >This looks like i have seen it before. For me that is _way_ too
> >> >> >generic and something that is not a package touches files all
> >> >> >over the place. If some package now wants to intentionally
> >> >> >bring a file that is from a far away future?
> >> >> >
> >> >> >Which files are we talking about here? It can basically only be
> >> >> >metadata and other little places where we violate our
> >> >> >"everything comes from a package" rule.  
> >> >>
> >> >> files/folder/symbolic-link that are modified or generated during
> >> >> build time like /etc/os-release /etc/hostname .
> >> >> .
> >> >> .
> >> >> /var/lib/dpkg/info/*
> >> >> /var/cache/*
> >> >> ---
> >> >>
> >> >> I have printed all the files that modified during build by
> >> >> executing below command find ${ROOTFSDIR} -newermt "$(date
> >> >> -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" -printf "%t %y  
> >%p\n"  
> >> >>
> >> >> modified_files_times.txt
> >> >>
> >> >> attached modified_files_times.txt for your reference, all these
> >> >> files/folders/symbolic-link are generated or modified during
> >> >> build time.  
> >> >
> >> >So i am guessing it is about anything that comes out of
> >> >postprocess functions and maintainer scripts like postinst ?  
> >>
> >> Yes, those are the files that are modified by the packages and have
> >> different timestamp on each build. I think it is okay to set
> >> reproducible times-tamp for such files, as they are not coming from
> >> the packages but are modified at build time.  
> >
> >Sure they are OK. But if i read your patch correctly you will
> >potentially adjust times on ALL files, not just some for which it
> >might be OK.
> >
> >And it uses a variable which does not have a default, should we not
> >set that to something? Or help users to choose a good value for it.  
> 
> Setting default value, I didn't think of it because reproducible
> builds may not be required for regular builds, however when someone
> wants it can be enable by setting the value to it. So as you
> mentioned it is good to mention somewhere to help users to choose
> good value for it, I am thinking to add it in doc/user_manual.md

Yes, user manual or maybe local.conf.sample but commented out. But some
value that works and makes sense needs to be given to people. Otherwise
i would set

SOURCE_DATE_EPOCH="just a guess 42"

which might not work

Henning

> >
> >Without this variable and a clue on a value to pick, this is all
> >dead code.
> >  
> >> >  
> >> >> >
> >> >> >Has this ever been tested against a complex layer, has any of
> >> >> >the repro work ever looked at something bigger than the very
> >> >> >artificial isar base image?  
> >> >> Similar implementations were used in the projects like
> >> >> Debian/live-build[1] and poky[2]
> >> >>
> >> >> [1]
> >> >> https://salsa.debian.org/live-team/live-build/-/blob/master/scripts
> >> >> /bu
> >> >> ild/efi-image#L57
> >> >> [2]
> >> >> https://github.com/yoctoproject/poky/blob/master/meta/classes-recip
> >> >> e/i
> >> >> mage.bbclass#L673  
> >> >
> >> >I am talking about isar layers not other projects.
> >> >  
> >> >> >
> >> >> >I think a better start would be to bbwarn and only much later
> >> >> >move to "-exec touch".
> >> >> >
> >> >> >We recently added CI tests for reproducible building. Would be
> >> >> >nice to the two patches. p1 writes a test-case that goes red,
> >> >> >p2 (this one) makes it go green  
> >> >> The patch (p1) already been sent [1], for writing test-case to
> >> >> verify reproducible build problems, the test-case is getting
> >> >> failed (shows non-reproducible time-stamp problem in
> >> >> build/diffoscope-output.txt) when executed without this patch,
> >> >> and after applying this patch it will still show fail (because
> >> >> there are other reproducible problems to fix) but the
> >> >> non-reproducible timestamp problem in rootfs shows fixed.  
> >> >
> >> >So the code is there but not used because it goes failed anyhow?
> >> >In that case i would suggest to rewrite it in a way that it does
> >> >not fail and can run in CI. It could for example ignore "known
> >> >issues" and be enabled, where then this very series would remove
> >> >a few "known issues" in p2 (p1 would be to introduce all
> >> >currently known issues and enable that thing in CI) and p3 would
> >> >be what we look at here  
> >>
> >> You mean run the test and just ignore the results?  
> >
> >Ignore some, the ones we know as violators that we still need to
> >work on. But do not ignore anything coming up as new problems,
> >maybe/likely in layers.
> >
> >Again, has this ever been tested on anything but the very boring
> >isar core? isar- cip-core could be interesting as well, especially
> >the image cip-core-image- security
> >  
> I have executed reproducible build tests in isar-cip-core image
> manually (not with the test-suite) and reported the problem as well
> here [1] [1]
> https://gitlab.com/cip-project/cip-core/isar-cip-core/-/issues/31
> 
> >> >
> >> >Not sure i understood but it sounds like we have a test that we
> >> >only run manually, and i want it to run in CI. In fact a script
> >> >to check for repro issues should be so generic that any layer can
> >> >use it.  
> >>
> >> Eventually the repro-build test-case should go to CI, I will plan
> >> to do it.
> >>
> >> Can I know where the isar CI pipe lines are running? I couldn't
> >> see in isar-github may be running some other place?  
> >
> >Unfortunately the two places that run it are not public and would
> >require you to receive an account. You could ask Baurzhan if you can
> >get one, or you just run it yourself manually while working on the
> >patches. If it is enabled and will ever fail, you will likely get
> >reports from CI sent to you.  
> 
> Got it, then I will test them manually as usual.
> 
> >
> >Henning
> >  
> >> >
> >> >So your vision would be that isar core brings that script and i.e.
> >> >jailhouse-images can run it on all its images. And a few calls to
> >> >the script will also be in testsuite/ and be called in pipelines
> >> >regularly.
> >> >
> >> >Henning
> >> >  
> >> >> [1]
> >> >> https://github.com/ilbers/isar/commit/9698445eade76fab81e2dc16d4cb5
> >> >> 8b2
> >> >> 4e953cb9
> >> >>  
> >> >> >
> >> >> >Henning
> >> >> >  
> >> >> >>  EOSUDO
> >> >> >>  }
> >> >> >>  addtask rootfs_finalize before do_rootfs after
> >> >> >> do_rootfs_postprocess  
> >> >> >
> >> >> >--
> >> >> >You received this message because you are subscribed to the
> >> >> >Google Groups "isar-users" group.
> >> >> >To unsubscribe from this group and stop receiving emails from
> >> >> >it, send an email to isar-users+unsubscribe@googlegroups.com.
> >> >> >To view this discussion on the web visit
> >> >> >https://groups.google.com/d/msgid/isar-
> >> >> >users/20230102174418.686715cf%40md1za8fc.ad001.siemens.net.  
> >> >
> >> >--
> >> >You received this message because you are subscribed to the Google
> >> >Groups "isar-users" group.
> >> >To unsubscribe from this group and stop receiving emails from it,
> >> >send an email to isar-users+unsubscribe@googlegroups.com.
> >> >To view this discussion on the web visit
> >> >https://groups.google.com/d/msgid/isar-
> >> >users/20230103200543.07e987ba%40md1za8fc.ad001.siemens.net.  
> >
> >--
> >You received this message because you are subscribed to the Google
> >Groups "isar-users" group.
> >To unsubscribe from this group and stop receiving emails from it,
> >send an email to isar-users+unsubscribe@googlegroups.com.
> >To view this discussion on the web visit
> >https://groups.google.com/d/msgid/isar-
> >users/20230104102934.042477a8%40md1za8fc.ad001.siemens.net.  
>