From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7188194988551831552 X-Received: by 2002:a17:902:b948:b0:194:58c7:ab70 with SMTP id h8-20020a170902b94800b0019458c7ab70mr30019pls.136.1673632065745; Fri, 13 Jan 2023 09:47:45 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:a50d:b0:226:42ea:28cc with SMTP id a13-20020a17090aa50d00b0022642ea28ccls5018522pjq.0.-pod-preprod-gmail; Fri, 13 Jan 2023 09:47:44 -0800 (PST) X-Google-Smtp-Source: AMrXdXviAniWCsUsLFlr60wEFkGrFPV/X2/SfxcJyi1B+rKF9/28K9krpNLc9fGCZniW0w2Tsk3m X-Received: by 2002:a17:902:d50e:b0:191:4378:ec06 with SMTP id b14-20020a170902d50e00b001914378ec06mr14963020plg.61.1673632064689; Fri, 13 Jan 2023 09:47:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673632064; cv=none; d=google.com; s=arc-20160816; b=jjELuJp/xVXhj7zbay/3lcK5hHrSS5epZmBcKDK95bF8kqwoYmlR7t305UXoiJMylG eieDb8tyxvri5P2WmR4YINUqBPUbtUSlZCpPqFkiok5/3+ips2nw0LGJymEN6KxO+K15 Pb6y/RVEG+BMJ87JTOblRVJZOnu3WyzA5mrP3L7FO2U6eCseK0INVHCEIQNMcHJHbR/F SGysm6BqDNX3tw0iEpiouprrdRAUL80WvN9NmdNsvG3DOMgHh5lE8wtdxz4KOschkLED JU8dbouLoE7dcX3x8t8D9KXa/xC5ZUKuph+6Gj+vTbBjRFTP7HGLeAgtLgUKt/OrsFNN 5Gew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=fvDd9ccOF6PdU+Gu9JGMDKWUUMLpFBh2nCm0zj00lLc=; b=ORlfktIr8iGCWVTEdBcAK6Ww/cvByx9sMk6Nd5wOmJ8zR/8ST9KaRgA3RkeLEC9AWm ntALnMtquylr4k4ffVHe/1UW5UTJ/rTNQYsPxT8UOfI+SLXfRMhGVHtRT5gEnJq6XadG aw4MPZsuuqt/Gdq/BJ7H6huZ22xYRaVDCDYtpiRo2Tqb/e4WoS2JSnhCdGyD5lBZOdSA z9a5K6xf9vUwK6S4lKTY79fRRFnqEG8z5HuLyNbDUxLQGf9/2yN4K1jAlw6KoXY1PoXg T2yJs8vV+2p7uwLNA10fQ8HxC20LVGq6Xg0bV3mHvMDaVhH6Lobw0j49kfJzfv+bkeo1 xTpA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 2001:4b7a:2000:18::164 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Return-Path: Received: from relay03.th.seeweb.it (relay03.th.seeweb.it. [2001:4b7a:2000:18::164]) by gmr-mx.google.com with ESMTPS id d17-20020a170902f15100b00188c5696675si2338076plb.6.2023.01.13.09.47.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Jan 2023 09:47:44 -0800 (PST) Received-SPF: pass (google.com: domain of roberto.foglietta@linuxteam.org designates 2001:4b7a:2000:18::164 as permitted sender) client-ip=2001:4b7a:2000:18::164; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of roberto.foglietta@linuxteam.org designates 2001:4b7a:2000:18::164 as permitted sender) smtp.mailfrom=roberto.foglietta@linuxteam.org Received: from localhost.localdomain (unknown [IPv6:2a02:8071:3187:7b80:cd05:a8a7:67a9:5f16]) by m-r1.th.seeweb.it (Postfix) with ESMTPA id DDB3E1F8A1; Fri, 13 Jan 2023 18:47:41 +0100 (CET) From: roberto.foglietta@linuxteam.org To: isar-users@googlegroups.com Cc: roberto.foglietta@gmail.com Subject: [PATCH v5] suggested changes for reproducibility patchset v5 Date: Fri, 13 Jan 2023 18:47:37 +0100 Message-Id: <20230113174737.281104-1-roberto.foglietta@linuxteam.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: Kx71B5wXe2nB From: "Roberto A. Foglietta" suggested changes for reproducibility patchset WARNING: eval-image-1.0-r0 do_rootfs_finalize: modified timestamp (1673628837) of 3 files for image reproducibly List of files modified could be found here: ./build/tmp/deploy/images/debx86/files.modified_timestamps v.2: rebased on current ilbers:next v.3: new script added: wic-extract-rootfs-partition.sh [image.wic] v.4: example with for epoch generation from git v.5: reverted the example and rework some few code Signed-off-by: Roberto A. Foglietta --- meta-isar/conf/local.conf.sample | 2 +- meta/classes/image-account-extension.bbclass | 6 +-- meta/classes/image.bbclass | 21 ++++---- meta/classes/initramfs.bbclass | 4 +- wic-extract-rootfs-partition.sh | 52 ++++++++++++++++++++ 5 files changed, 70 insertions(+), 15 deletions(-) create mode 100755 wic-extract-rootfs-partition.sh diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index 6208623..1d7e178 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -257,4 +257,4 @@ USER_isar[flags] += "clear-text-password" # Non git repository users can use value from 'stat -c%Y ChangeLog' # To know more details about this variable and how to set the value refer below # https://reproducible-builds.org/docs/source-date-epoch/ -#SOURCE_DATE_EPOCH = +#SOURCE_DATE_EPOCH = "" diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/image-account-extension.bbclass index bb173b1..1d49054 100644 --- a/meta/classes/image-account-extension.bbclass +++ b/meta/classes/image-account-extension.bbclass @@ -256,11 +256,11 @@ image_postprocess_accounts() { # chpasswd adds a random salt when running against a clear-text password. # For reproducible images, we manually generate the password and use the # SOURCE_DATE_EPOCH to generate the salt in a deterministic way. - if [ -z "${SOURCE_DATE_EPOCH}"]; then + if [ -z "${SOURCE_DATE_EPOCH}" ]; then chpasswd_args="" else - salt="$(echo "${SOURCE_DATE_EPOCH}" | sha256sum -z | cut -c 1-15)" - password="$(openssl passwd -6 -salt $salt "$password")" + salt="$(echo ${SOURCE_DATE_EPOCH} | sha256sum -z | cut -c 1-15)" + password="$(openssl passwd -6 -salt $salt $password)" fi fi printf '%s:%s' "$name" "$password" | sudo chroot '${ROOTFSDIR}' \ diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 063b9a3..944733b 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -310,8 +310,8 @@ python() { # invalidate the SSTATE entries for most packages, even if they don't use the # global SOURCE_DATE_EPOCH variable. rootfs_install_pkgs_install_prepend() { - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + if [ -n "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH fi } @@ -443,13 +443,16 @@ EOSUDO # Set same time-stamps to the newly generated file/folders in the # rootfs image for the purpose of reproducible builds. - test ! -z "${SOURCE_DATE_EPOCH}" && \ - sudo find ${ROOTFSDIR} -newermt \ - "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ - -printf "%y %p\n" \ - -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' > ${DEPLOY_DIR_IMAGE}/files.modified_timestamps && \ - bbwarn "$(grep ^f ${DEPLOY_DIR_IMAGE}/files.modified_timestamps) \nModified above file timestamps to build image reproducibly" - + if [ -n "${SOURCE_DATE_EPOCH}" ]; then + fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ + -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' >"$fn"; then + if [ -e "$fn" ]; then + bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of $(egrep ^f "$fn" | wc -l) files for image reproducibly\n " \ + "List of files modified could be found here: .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" + fi + fi + fi } addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess diff --git a/meta/classes/initramfs.bbclass b/meta/classes/initramfs.bbclass index db28334..1b98bc0 100644 --- a/meta/classes/initramfs.bbclass +++ b/meta/classes/initramfs.bbclass @@ -33,8 +33,8 @@ do_generate_initramfs() { rootfs_do_qemu # generate reproducible initrd if requested - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then - export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + if [ -n "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH fi sudo -E chroot "${INITRAMFS_ROOTFS}" \ diff --git a/wic-extract-rootfs-partition.sh b/wic-extract-rootfs-partition.sh new file mode 100755 index 0000000..48de0d3 --- /dev/null +++ b/wic-extract-rootfs-partition.sh @@ -0,0 +1,52 @@ +#!/bin/bash +# +# Copyright (c) Roberto A. Foglietta, 2023 +# +# Authors: +# Roberto A. Foglietta +# +# SPDX-License-Identifier: MIT +# +#set -ex + +if [ "$(whoami)" != "root" ]; then + echo + echo "WARNING: this script should run as root, sudo!" + sudo -E $0 "$@" + exit $? +fi + +if [ -e "$1" ]; then + fimg=$(readlink -e $1) +fi + +cd $(dirname $0) + +if [ ! -n "$1" -a ! -e "$fimg" ]; then + fimg=$(ls -1 build/tmp/deploy/images/*/*.wic) + n=( $fimg ) + if [ ${#n[@]} -gt 1 ]; then + echo + echo "WARNING: more than one image found, choose one:" + echo + echo "$fimg" + echo + exit 1 + fi +fi + +if [ ! -e "$fimg" ]; then + echo + echo "ERROR: no any image or block device ${1:+'$1' }found, abort!" + echo + exit 1 +fi + +wicf=$fimg +losetup -Pf $wicf +ldev=$(losetup -j $wicf | cut -d: -f1 | tail -n1) +echo loopdev:$ldev +dd if=${ldev}p2 bs=1M of=${wicf/.wic/.rootfs} status=progress +chown $(id -u).$(id -g) ${wicf/.wic/.rootfs} +du -ms ${wicf/.wic/.rootfs} +losetup -d $ldev -- 2.34.1