Re: [PATCH] image.bbclass: set file timestamps inside the rootfs and initramfs image

[Henning Schild <henning.schild@siemens.com>]

Am Mon,  7 Nov 2022 13:55:03 +0530
schrieb venkata.pyla@toshiba-tsip.com:

> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 
> As part of reproducible-build work, one of the problem chosen to solve
> is the file time stamps inside rootfs and initramfs are not identical
> between two builds.
> 
> With the help of reproducible-builds.org and their suggestions, the
> above problem can be fixed using 'SOURCE_DATE_EPOCH' variable [2].
> 
> In case of rootfs file time-stamps, set all the files and folders that
> are newer than 'SOURCE_DATE_EPOCH' and set it to same.
> In case of initramfs, regenerate the initramfs image with
> 'SOURCE_DATE_EPOCH' variable set as the mkinitramfs script is already
> taken care of creating reproducible initramfs image when the
> variable is set in the environment[3].
> 
> The SOURCE_DATE_EPOCH variable should be set to the last modification
> of the git repository as explained in the documentation[2].
> 
> e.g:
> SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
> 
> To know more about the reproducible builds and its goals please refer
> [1].
> 
> [1] https://reproducible-builds.org/
> [2] https://reproducible-builds.org/docs/source-date-epoch/
> [3]
> https://manpages.debian.org/bullseye/initramfs-tools-core/mkinitramfs.8.en.html#ENVIRONMENT
> 
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
>  meta/classes/image.bbclass | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ccff810..c1bb4fd 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -431,6 +431,21 @@ do_rootfs_finalize() {
>              "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
>  
>          rm -f "${ROOTFSDIR}/etc/apt/sources-list"
> +
> +        # Recreate initramfs inorder to set timestamps to
> SOURCE_DATE_EPOCH
> +        # inorder to make reproducible initramfs
> +        test ! -z "${SOURCE_DATE_EPOCH}" && \
> +           SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH} chroot
> "${ROOTFSDIR}" \
> +                  update-initramfs -u -v

This should be done where that initial update-initramfs can be found.
And not fix things up later. Not every image will have an initrd so
that is wrong in multiple ways.

> +
> +	# Set timestamp to files inside the rootfs image inorder to
> make
> +	# reproducible rootfs
> +	test ! -z "${SOURCE_DATE_EPOCH}" && \
> +           find ${ROOTFSDIR} -newermt \
> +               "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d
> %H:%M:%S')" \
> +               -printf "%y %p\n" \
> +               -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';'
> +

No! Which files do we care about? My guess would content of packages
built with isar. We should export the variable in the dpkg class then
the packages will be correct and we do not have to mess around with
that rootfs and fix problems that should have never been there.

I would like to ask for test cases. Ideally first a breaking test and
later a commit fixing the issue.

Note that deriving the time from git means one needs git. And when
using layers just that one Isar git is clearly not good enough. And we
will have to check how that works with sstate. I assume any new commit
would rebuild all custom packages. Here i see a conflict between
regular dev work and repro pedantics. We might need a way to turn that
stuff off.

regards,
Henning

>  EOSUDO
>  }
>  addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess