Re: [PATCH v2 1/1] image.bbclass: fix non-reproducible file time-stamps inside rootfs

[Henning Schild <henning.schild@siemens.com>]

Am Thu, 5 Jan 2023 13:52:51 +0000
schrieb <Venkata.Pyla@toshiba-tsip.com>:

> >-----Original Message-----
> >From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> >Behalf Of Henning Schild
> >Sent: 05 January 2023 13:49
> >To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting) <Venkata.Pyla@toshiba-  
> >tsip.com>  
> >Cc: isar-users@googlegroups.com; amikan@ilbers.de;
> >jan.kiszka@siemens.com; hayashi kazuhiro(林 和宏 □ＳＷＣ◯ＡＣＴ)
> ><kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(ＴＳＩＰ TMIEC ODG
> >Porting) <dinesh.kumar@toshiba-tsip.com> Subject: Re: [PATCH v2 1/1]
> >image.bbclass: fix non-reproducible file time-stamps inside rootfs
> >
> >Am Thu,  5 Jan 2023 11:48:57 +0530
> >schrieb venkata.pyla@toshiba-tsip.com:
> >  
> >> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >>
> >> As part of reproducible-build work, the rootfs images generated on
> >> same source should be identical between two builds.
> >>
> >> In this commit it tries to solve one of the non-reproducible
> >> problem i.e. the rootfs file time-stamps generated during build
> >> time are not reproducible, it uses one of the solution provided in
> >> the debian live-build image project (refer [1]), it fixes by
> >> finding all the files/folders that are gernerated newly and set
> >> the time-stamp provided by `SOURCE_DATE_EPOCH` environment
> >> variable.
> >>
> >> [1]
> >> https://salsa.debian.org/live-team/live-build/-/merge_requests/218
> >>
> >> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >> ---
> >>  meta-isar/conf/local.conf.sample | 10 ++++++++++
> >>  meta/classes/image.bbclass       |  9 +++++++++
> >>  2 files changed, 19 insertions(+)
> >>
> >> diff --git a/meta-isar/conf/local.conf.sample
> >> b/meta-isar/conf/local.conf.sample index 57d0620..3c4a473 100644
> >> --- a/meta-isar/conf/local.conf.sample
> >> +++ b/meta-isar/conf/local.conf.sample
> >> @@ -255,3 +255,13 @@ USER_isar[flags] += "clear-text-password"
> >>  #CCACHE_TOP_DIR ?= "${TMPDIR}/ccache"
> >>  # Enable ccache debug mode
> >>  #CCACHE_DEBUG = "1"
> >> +
> >> +# Uncommnet and add value to it to build images reproducibly # #
> >> The +value for `SOURCE_DATE_EPOCH` should be latest source change
> >> time in +# seconds since the Epoch.
> >> +# Git repository users can use value from 'git log -1
> >> --pretty=%ct' +# Non git repository users can use value from 'stat
> >> -c%Y ChangeLog' +# To know more details about this variable and
> >> how to set the value refer below +#
> >> https://reproducible-builds.org/docs/source-date-epoch/
> >> +#SOURCE_DATE_EPOCH =  
> >
> >${@bb.process.run(git log ...)}
> >
> >would be nice here. So once uncommented it will keep moving as people
> >commit.
> >  
> >> diff --git a/meta/classes/image.bbclass
> >> b/meta/classes/image.bbclass index 813e1f3..38a9adf 100644
> >> --- a/meta/classes/image.bbclass
> >> +++ b/meta/classes/image.bbclass
> >> @@ -430,6 +430,15 @@ do_rootfs_finalize() {
> >>              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
> >>
> >>          rm -f "${ROOTFSDIR}/etc/apt/sources-list"
> >> +
> >> +        # Set same time-stamps to the newly generated file/folders
> >> in the
> >> +        # rootfs image for the purpose of reproducible builds.
> >> +        test ! -z "${SOURCE_DATE_EPOCH}" && \
> >> +            find ${ROOTFSDIR} -newermt \
> >> +                "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d
> >> %H:%M:%S')" \
> >> +                -printf "%y %p\n" \
> >> +                -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' >
> >> ${DEPLOY_DIR_IMAGE}/files.modified_timestamps +  EOSUDO  
> >
> >I would suggest to at least display a bbwarn if "wc -l" of that file
> >exceeds some number ... say 50. I guess if SOURCE_DATE_EPOCH was too
> >old, say 01.01.1990 the whole filesystem would be touched which
> >might indicate a problem.  
> 'SOURCE_DATE_EPOCH' is not set by default and regular builds will not
> have any problem for sure, but when someone uses it then I think they
> know what they are doing and can see the results.

We eventually want to enable it by default i guess and so we can
already think about what not so experienced users could stumble over
and how to help them.

> I will also add bbwarn about this files time modification and print
> all files that are modified.

Thanks.

Henning

> >
> >Not sure what a good number would be. We could also check for
> >certain files to _not_ be in there for sure.
> >
> >I might give that patch a try and see for myself what a too old
> >value would do. But right now i will keep going with the expectation
> >that it would "touch all files without big warning" and the thing
> >might still boot but the broken metadata could cause any kind of
> >problems in applications that can get confused by that big change.
> >
> >Henning
> >  
> >>  }
> >>  addtask rootfs_finalize before do_rootfs after
> >> do_rootfs_postprocess  
> >
> >--
> >You received this message because you are subscribed to the Google
> >Groups "isar-users" group.
> >To unsubscribe from this group and stop receiving emails from it,
> >send an email to isar-users+unsubscribe@googlegroups.com.
> >To view this discussion on the web visit
> >https://groups.google.com/d/msgid/isar-
> >users/20230105091904.530199bb%40md1za8fc.ad001.siemens.net.  
>