Re: [RFC PATCH] image.bbclass: create separate task for creating reproducible image

[Henning Schild <henning.schild@siemens.com>]

Please do not default cc me on reproducible image topics. I do not care
enough and will see it coming or see it on the list.

Henning

Am Mon, 27 Feb 2023 13:20:32 +0530
schrieb venkata.pyla@toshiba-tsip.com:

> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 
> Currently the reproducible fix for setting same file timestamps across
> builds is present in the function `do_rootfs_finalize` and this fix
> may be skipped for the tasks that are added after this task or
> `do_rootfs` for e.g. in one of the child project(CIP) for swupdate
> target it adds additional task `do_generate_image_uuid` that is added
> after `do_rootfs` this task is updating some contents in the rootfs
> which is skipping the reproducible fix was applied in
> `do_rootfs_finalize`.
> 
> For this reason a separate task `do_image_make_reproducible` is
> created which should call before the final target image is created
> [tar, wic, squashfs] so that it applies reproducible fixes just
> before image creation.
> 
> RFC:
> In this patch I tried below dependency[1] for the function to solve
> the above problem, it works now but I am not sure whether it guarantee
> always this task `do_image_make_reproducible` will run just before
> `do_image_[tar,wic,squashfs]` tasks or does it need more changes to
> make it guarantee, any suggestions?
> 
> [1] addtask image_make_reprodcible after do_rootfs before do_image
> 
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
>  meta/classes/image.bbclass | 33 ++++++++++++++++++++-------------
>  1 file changed, 20 insertions(+), 13 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ef7d5a2..c3a3d45 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -446,19 +446,6 @@ do_rootfs_finalize() {
>          fi
>  EOSUDO
>  
> -    # Set same time-stamps to the newly generated file/folders in the
> -    # rootfs image for the purpose of reproducible builds.
> -    if [ -n "${SOURCE_DATE_EPOCH}" ]; then
> -        fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps"
> -        if sudo find ${ROOTFSDIR} -newermt "$(date
> -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \
> -            -printf "%y %p\n" -exec touch '{}' -h
> -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then
> -            if [ -e "$fn" ]; then
> -                bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of
> $(cat "$fn" | wc -l) files for image reproducibly." \
> -                       "List of files modified can be found in:
> .${DEPLOY_DIR_IMAGE}/files.modified_timestamps"
> -            fi
> -        fi
> -    fi
> -
>  }
>  do_rootfs_finalize[network] = "${TASK_USE_SUDO}"
>  addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess
> @@ -502,3 +489,23 @@ do_rootfs_quality_check() {
>  do_rootfs_quality_check[network] = "${TASK_USE_SUDO}"
>  
>  addtask rootfs_quality_check after do_rootfs_finalize before
> do_rootfs +
> +# Run this task just before final image creation of different image
> types +#     i.e do_image_tar, do_image_squashfs, do_image_wic etc.
> +#     to avoid modification of image contents that leads to
> non-reproducible +#     image
> +do_image_make_reproducible() {
> +    # Set same time-stamps to the newly generated file/folders in the
> +    # rootfs image for the purpose of reproducible builds.
> +    if [ -n "${SOURCE_DATE_EPOCH}" ]; then
> +        fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps"
> +        if sudo find ${ROOTFSDIR} -newermt "$(date
> -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \
> +            -printf "%y %p\n" -exec touch '{}' -h
> -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then
> +            if [ -e "$fn" ]; then
> +                bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of
> $(cat "$fn" | wc -l) files for image reproducibly." \
> +                       "List of files modified can be found in:
> .${DEPLOY_DIR_IMAGE}/files.modified_timestamps"
> +            fi
> +        fi
> +    fi
> +}
> +addtask image_make_reprodcible after do_rootfs before do_image