From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7204739790011367424 X-Received: by 2002:a05:6870:715:b0:172:2b6d:e85f with SMTP id ea21-20020a056870071500b001722b6de85fmr3221369oab.11.1677484202351; Sun, 26 Feb 2023 23:50:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aca:2209:0:b0:383:b2c0:4e6a with SMTP id b9-20020aca2209000000b00383b2c04e6als2837106oic.2.-pod-prod-gmail; Sun, 26 Feb 2023 23:50:01 -0800 (PST) X-Google-Smtp-Source: AK7set9FQQXAcbQy3i2Z7jYRdpdq4ADw+B8IstfQJa4Eh10kW4fQXkmgM6btmrPM4zjEKDCTLO6B X-Received: by 2002:a54:4692:0:b0:37a:f32f:730c with SMTP id k18-20020a544692000000b0037af32f730cmr9551790oic.35.1677484201759; Sun, 26 Feb 2023 23:50:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677484201; cv=none; d=google.com; s=arc-20160816; b=Uhv9X4sYGgRakjBEvOc5nMI1qN7tl7ybcUPPexgcGNP28wHZ6oEjJo1zQ70JYXJB3m vxtkCxWpXEgIDoxpBwxJriqCXd2fzc6twDvbo1+nQXLaHglTqogCwrmLQFbn4Fb3exq4 V0SLTHvPAPV7Qcfu82PoD0uzNZDqqDxySP0ZScAHmAM5tDfgC7PcQ332PtDUAyn12dXq ebxRU9EN7EmxDgfM4b7/GXpLDMH3BWjCKv5GvVx+syXJ0xz3dKnnqt8TG1bubxxdzhCE yMe/2AhZVYtwsJszkAlssbF57vtvxLnE+H6469mtPUyscva5wlyDumRWbwryE3P29nvA bBUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=lCyY+xKdxx/tf0yG0iZgRg9aOe5uAhKo5F/N8f37fzQ=; b=NomiIVSquM3hlc1fx0wMi7KFTyGSlVV8LhBZ5b2n5vRPzSt8E2VlgxLwk4xg1Fx/c4 oXG0VIjIqolrxSg5em0vqvE8GRVqAafpDgIeu/wjeENioR/ofYUqZDCXPqVVE1WmhhSj B7cvNfkwbHnvsgb6DPwiPLA4/8WpjoKB1JYhAJFxYsD7tcwgxjo/I1QEmAzIeA3DRlzp NC4fmMVD4b3P+txeI9e9dq+YAygHkte1kTKuGYMxFYJidxIM3QyNNL9b9j2L+Rtu+yJT RQskZp5z3yRnyrQGemyBj/HW7VUqtviX7ItzYcpOeA8IpNQsZ4qGxk7AlImasPS945I/ zgmg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com Return-Path: Received: from mo-csw.securemx.jp (mo-csw1514.securemx.jp. [210.130.202.153]) by gmr-mx.google.com with ESMTPS id bh4-20020a056808180400b0037fa46467c0si329926oib.0.2023.02.26.23.50.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Feb 2023 23:50:01 -0800 (PST) Received-SPF: pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) client-ip=210.130.202.153; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.153 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com Received: by mo-csw.securemx.jp (mx-mo-csw1514) id 31R7nxGs007160; Mon, 27 Feb 2023 16:49:59 +0900 X-Iguazu-Qid: 34tro5DMQMnkZK47sU X-Iguazu-QSIG: v=2; s=0; t=1677484199; q=34tro5DMQMnkZK47sU; m=mXOrX9X4QS4jMEBvqA1xXiVrIMhSHwJ6Z7yd8FwSmy0= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1510) id 31R7nwiA017099 (version=TLSv1.2 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 27 Feb 2023 16:49:58 +0900 From: venkata.pyla@toshiba-tsip.com To: isar-users@googlegroups.com Cc: venkata pyla , felix.moessbauer@siemens.com, roberto.foglietta@gmail.com, jan.kiszka@siemens.com, henning.schild@siemens.com, kazuhiro3.hayashi@toshiba.co.jp, dinesh.kumar@toshiba-tsip.com Subject: [RFC PATCH] image.bbclass: create separate task for creating reproducible image Date: Mon, 27 Feb 2023 13:20:32 +0530 X-TSB-HOP2: ON Message-Id: <20230227075032.28099-1-venkata.pyla@toshiba-tsip.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 27 Feb 2023 07:49:56.0591 (UTC) FILETIME=[15C917F0:01D94A80] X-TUID: Fz4ptVpPQiCT From: venkata pyla Currently the reproducible fix for setting same file timestamps across builds is present in the function `do_rootfs_finalize` and this fix may be skipped for the tasks that are added after this task or `do_rootfs` for e.g. in one of the child project(CIP) for swupdate target it adds additional task `do_generate_image_uuid` that is added after `do_rootfs` this task is updating some contents in the rootfs which is skipping the reproducible fix was applied in `do_rootfs_finalize`. For this reason a separate task `do_image_make_reproducible` is created which should call before the final target image is created [tar, wic, squashfs] so that it applies reproducible fixes just before image creation. RFC: In this patch I tried below dependency[1] for the function to solve the above problem, it works now but I am not sure whether it guarantee always this task `do_image_make_reproducible` will run just before `do_image_[tar,wic,squashfs]` tasks or does it need more changes to make it guarantee, any suggestions? [1] addtask image_make_reprodcible after do_rootfs before do_image Signed-off-by: venkata pyla --- meta/classes/image.bbclass | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index ef7d5a2..c3a3d45 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -446,19 +446,6 @@ do_rootfs_finalize() { fi EOSUDO - # Set same time-stamps to the newly generated file/folders in the - # rootfs image for the purpose of reproducible builds. - if [ -n "${SOURCE_DATE_EPOCH}" ]; then - fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" - if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ - -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then - if [ -e "$fn" ]; then - bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(cat "$fn" | wc -l) files for image reproducibly." \ - "List of files modified can be found in: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" - fi - fi - fi - } do_rootfs_finalize[network] = "${TASK_USE_SUDO}" addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess @@ -502,3 +489,23 @@ do_rootfs_quality_check() { do_rootfs_quality_check[network] = "${TASK_USE_SUDO}" addtask rootfs_quality_check after do_rootfs_finalize before do_rootfs + +# Run this task just before final image creation of different image types +# i.e do_image_tar, do_image_squashfs, do_image_wic etc. +# to avoid modification of image contents that leads to non-reproducible +# image +do_image_make_reproducible() { + # Set same time-stamps to the newly generated file/folders in the + # rootfs image for the purpose of reproducible builds. + if [ -n "${SOURCE_DATE_EPOCH}" ]; then + fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ + -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then + if [ -e "$fn" ]; then + bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(cat "$fn" | wc -l) files for image reproducibly." \ + "List of files modified can be found in: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + fi + fi + fi +} +addtask image_make_reprodcible after do_rootfs before do_image -- 2.20.1