>-----Original Message----- >From: isar-users@googlegroups.com On Behalf >Of Henning Schild >Sent: 02 January 2023 22:14 >To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting) tsip.com> >Cc: isar-users@googlegroups.com; amikan@ilbers.de; jan.kiszka@siemens.com; >hayashi kazuhiro(林 和宏 □ＳＷＣ◯ＡＣＴ) ; >dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) >Subject: Re: [PATCH] image.bbclass: fix non-reproducible file time-stamps inside >rootfs image > >Am Mon, 2 Jan 2023 20:28:28 +0530 >schrieb venkata.pyla@toshiba-tsip.com: > >> From: venkata pyla >> >> As part of reproducible-build work, the rootfs images generated on >> same source should be identical between two builds. >> >> In this commit it tries to solve one of the non-reproducible problem >> i.e. the rootfs file time-stamps generated during build time are not >> reproducible, it uses one of the solution provided in the debian >> live-build image project (refer [1]), it fixes by finding all the >> files/folders that are gernerated newly and set the time-stamp >> provided by `SOURCE_DATE_EPOCH` environment variable. >> >> [1] https://salsa.debian.org/live-team/live-build/-/merge_requests/218 >> >> Signed-off-by: venkata pyla >> --- >> meta/classes/image.bbclass | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass >> index 813e1f3..f592a12 100644 >> --- a/meta/classes/image.bbclass >> +++ b/meta/classes/image.bbclass >> @@ -430,6 +430,15 @@ do_rootfs_finalize() { >> "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" >> >> rm -f "${ROOTFSDIR}/etc/apt/sources-list" >> + >> + # Set same time-stamps to the newly generated file/folders >> in the >> + # rootfs image for the purpose of reproducible builds. >> + test ! -z "${SOURCE_DATE_EPOCH}" && \ >> + find ${ROOTFSDIR} -newermt \ >> + "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d >> %H:%M:%S')" \ >> + -printf "%y %p\n" \ >> + -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' >> + > >This looks like i have seen it before. For me that is _way_ too generic and >something that is not a package touches files all over the place. >If some package now wants to intentionally bring a file that is from a far away >future? > >Which files are we talking about here? It can basically only be metadata and >other little places where we violate our "everything comes from a package" rule. files/folder/symbolic-link that are modified or generated during build time like /etc/os-release /etc/hostname . . . /var/lib/dpkg/info/* /var/cache/* --- I have printed all the files that modified during build by executing below command find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" -printf "%t %y %p\n" > modified_files_times.txt attached modified_files_times.txt for your reference, all these files/folders/symbolic-link are generated or modified during build time. > >Has this ever been tested against a complex layer, has any of the repro work >ever looked at something bigger than the very artificial isar base image? Similar implementations were used in the projects like Debian/live-build[1] and poky[2] [1] https://salsa.debian.org/live-team/live-build/-/blob/master/scripts/build/efi-image#L57 [2] https://github.com/yoctoproject/poky/blob/master/meta/classes-recipe/image.bbclass#L673 > >I think a better start would be to bbwarn and only much later move to "-exec >touch". > >We recently added CI tests for reproducible building. Would be nice to the two >patches. p1 writes a test-case that goes red, p2 (this one) makes it go green The patch (p1) already been sent [1], for writing test-case to verify reproducible build problems, the test-case is getting failed (shows non-reproducible time-stamp problem in build/diffoscope-output.txt) when executed without this patch, and after applying this patch it will still show fail (because there are other reproducible problems to fix) but the non-reproducible timestamp problem in rootfs shows fixed. [1] https://github.com/ilbers/isar/commit/9698445eade76fab81e2dc16d4cb58b24e953cb9 > >Henning > >> EOSUDO >> } >> addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess > >-- >You received this message because you are subscribed to the Google Groups >"isar-users" group. >To unsubscribe from this group and stop receiving emails from it, send an email >to isar-users+unsubscribe@googlegroups.com. >To view this discussion on the web visit >https://groups.google.com/d/msgid/isar- >users/20230102174418.686715cf%40md1za8fc.ad001.siemens.net.