Re: [PATCH v2 1/1] image.bbclass: fix non-reproducible file time-stamps inside rootfs

[Henning Schild <henning.schild@siemens.com>]

Am Thu,  5 Jan 2023 11:48:57 +0530
schrieb venkata.pyla@toshiba-tsip.com:

> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 
> As part of reproducible-build work, the rootfs images generated on
> same source should be identical between two builds.
> 
> In this commit it tries to solve one of the non-reproducible problem
> i.e. the rootfs file time-stamps generated during build time are not
> reproducible, it uses one of the solution provided in the debian
> live-build image project (refer [1]), it fixes by finding all the
> files/folders that are gernerated newly and set the time-stamp
> provided by `SOURCE_DATE_EPOCH` environment variable.
> 
> [1] https://salsa.debian.org/live-team/live-build/-/merge_requests/218
> 
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
>  meta-isar/conf/local.conf.sample | 10 ++++++++++
>  meta/classes/image.bbclass       |  9 +++++++++
>  2 files changed, 19 insertions(+)
> 
> diff --git a/meta-isar/conf/local.conf.sample
> b/meta-isar/conf/local.conf.sample index 57d0620..3c4a473 100644
> --- a/meta-isar/conf/local.conf.sample
> +++ b/meta-isar/conf/local.conf.sample
> @@ -255,3 +255,13 @@ USER_isar[flags] += "clear-text-password"
>  #CCACHE_TOP_DIR ?= "${TMPDIR}/ccache"
>  # Enable ccache debug mode
>  #CCACHE_DEBUG = "1"
> +
> +# Uncommnet and add value to it to build images reproducibly
> +#
> +# The value for `SOURCE_DATE_EPOCH` should be latest source change
> time in +# seconds since the Epoch.
> +# Git repository users can use value from 'git log -1 --pretty=%ct'
> +# Non git repository users can use value from 'stat -c%Y ChangeLog'
> +# To know more details about this variable and how to set the value
> refer below +# https://reproducible-builds.org/docs/source-date-epoch/
> +#SOURCE_DATE_EPOCH =

${@bb.process.run(git log ...)}

would be nice here. So once uncommented it will keep moving as people
commit.

> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 813e1f3..38a9adf 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -430,6 +430,15 @@ do_rootfs_finalize() {
>              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
>  
>          rm -f "${ROOTFSDIR}/etc/apt/sources-list"
> +
> +        # Set same time-stamps to the newly generated file/folders
> in the
> +        # rootfs image for the purpose of reproducible builds.
> +        test ! -z "${SOURCE_DATE_EPOCH}" && \
> +            find ${ROOTFSDIR} -newermt \
> +                "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d
> %H:%M:%S')" \
> +                -printf "%y %p\n" \
> +                -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' >
> ${DEPLOY_DIR_IMAGE}/files.modified_timestamps +
>  EOSUDO

I would suggest to at least display a bbwarn if "wc -l" of that file
exceeds some number ... say 50. I guess if SOURCE_DATE_EPOCH was too
old, say 01.01.1990 the whole filesystem would be touched which might
indicate a problem.

Not sure what a good number would be. We could also check for certain
files to _not_ be in there for sure.

I might give that patch a try and see for myself what a too old value
would do. But right now i will keep going with the expectation that it
would "touch all files without big warning" and the thing might still
boot but the broken metadata could cause any kind of problems in
applications that can get confused by that big change.

Henning

>  }
>  addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess