From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7189006716065808384 X-Received: by 2002:ac2:4c52:0:b0:4cc:87bf:d585 with SMTP id o18-20020ac24c52000000b004cc87bfd585mr1129187lfk.91.1673822038008; Sun, 15 Jan 2023 14:33:58 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:b8cd:0:b0:28b:86c7:a456 with SMTP id s13-20020a2eb8cd000000b0028b86c7a456ls111117ljp.11.-pod-prod-gmail; Sun, 15 Jan 2023 14:33:56 -0800 (PST) X-Google-Smtp-Source: AMrXdXv3OxJh1eoNlvsA7D1QGfJXHOkwA7VH5x6D/YmOqc7zbCVj4vN3pkrH3lr/WBoaMCaO+0vl X-Received: by 2002:a2e:bf14:0:b0:286:927:accd with SMTP id c20-20020a2ebf14000000b002860927accdmr8622510ljr.5.1673822036625; Sun, 15 Jan 2023 14:33:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673822036; cv=none; d=google.com; s=arc-20160816; b=Q6yZCITMknI472O+athbpNunkyFKLB7QGxrMutmFDVr1ptCr2wIm1pYpdfn3E/O0NA Aekn9hS1qmWULozyMYrCDK4y9XZpBLqMTC6y163S6R8yyZu9kYEgq0PIsCXhvt8DG8Ec 5HNyBeB6LGLc7o6iwkwauUOYgqK5/PLYx9lilh4XXsHufwypmXOf6jP4hbYi72JaN+Ql ysGsVApqiAW+m+1sPe/WTKNDOkS+hqrQ/ot6D1lrLZfwTgHjOeWUEiOTpcxP/VygOoYb I1sZ7r0F4RR+6SKIGwG+vZKBBquRYqdl8ofmRcriAnTb1JIFQNOCfbWAYKRG7KcIdWBe VLUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:mime-version:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:dkim-signature; bh=BqXzE/CcatUIDrl+6UEdwuWfnB3F12JMojLk+BEisLs=; b=h6zOzT4i1Xjg5gEjfTAw499LnshDtHX8jGtTk+x17EdVHZ4ntJndJnkM6ib5KnjzGZ eUrJ61VOL+B2T/0QJA/rvoGV+b30ykuoxp71iSreCwstQaJqZBJhCF8ahtq6ScF7xGuy fw0uNsydlc48pAabsRpDp6bF+XENP6EIc7Nk2t9GEoEV5aU7RclpdcklITGb03CtqGSN avv1O2LlHxJBIoHN9atwcgT25zKtR3d4DlKaBsgeqQB+GshrabKc7Sn+dzfLDtOptyAU FoWCMqxmJEB/28XpXZG46h+SkrzbgY1utr3J8rfFVQ1JHLBUot9R36Uc4B3NJDuIdU7a zE3w== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=FGBxE9pL; spf=pass (google.com: domain of fm-68982-20230115223355a9f0b3b3c6b7d242f6-pjck7y@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-68982-20230115223355a9f0b3b3c6b7d242f6-PjCK7Y@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net. [185.136.64.226]) by gmr-mx.google.com with ESMTPS id k11-20020a2eb74b000000b0028b7cc84addsi119669ljo.2.2023.01.15.14.33.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jan 2023 14:33:56 -0800 (PST) Received-SPF: pass (google.com: domain of fm-68982-20230115223355a9f0b3b3c6b7d242f6-pjck7y@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) client-ip=185.136.64.226; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=FGBxE9pL; spf=pass (google.com: domain of fm-68982-20230115223355a9f0b3b3c6b7d242f6-pjck7y@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-68982-20230115223355a9f0b3b3c6b7d242f6-PjCK7Y@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 20230115223355a9f0b3b3c6b7d242f6 for ; Sun, 15 Jan 2023 23:33:55 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=florian.bezdeka@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=BqXzE/CcatUIDrl+6UEdwuWfnB3F12JMojLk+BEisLs=; b=FGBxE9pLr3YOr9EdINkOO10OgJcOjlmvOBXGizU3OST5AwVBsg9q7Irx61oEYcEoRdmi+B qTYNkkAoU5Gu3JBhqk+DIwfOMXxSaSqKIRalmsXwjGWYLtgQdagfFZsB/0t6sMPH7UvBQrCJ 8MgkSIqVxzcafZ1kerrYgafDv/gEw=; Message-ID: <23947d5255641d5d868639240ad8b5455ea6ab31.camel@siemens.com> Subject: Re: [PATCH v7] suggested changes for reproducibility patchset v7 From: Florian Bezdeka To: roberto.foglietta@linuxteam.org, isar-users@googlegroups.com Cc: roberto.foglietta@gmail.com Date: Sun, 15 Jan 2023 23:33:55 +0100 In-Reply-To: <20230115221734.741365-1-roberto.foglietta@linuxteam.org> References: <20230115221734.741365-1-roberto.foglietta@linuxteam.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-68982:519-21489:flowmailer X-TUID: I3C9/t+CGA1o On Sun, 2023-01-15 at 23:17 +0100, roberto.foglietta@linuxteam.org wrote: > From: "Roberto A. Foglietta" >=20 > suggested changes for reproducibility patchset >=20 > WARNING: eval-image-1.0-r0 do_rootfs_finalize: modified timestamp (167362= 8837) of 3 files for image reproducibly > List of files modified could be found here: ./build/tmp/deploy/i= mages/debx86/files.modified_timestamps >=20 > v.2: rebased on current ilbers:next >=20 > v.3: new script added: wic-extract-rootfs-partition.sh [image.wic] >=20 > v.4: example with for epoch generation from git >=20 > v.5: reverted the example and rework some few code >=20 > v.6: the 1st part of the warning shows up each time the epoch is used > while the 2nd line appears only when some files has been touched > This allows the user to know the current situation aboat epoch. >=20 > v.7: forgot to commit before producing the patch v6 but sent! >=20 > Signed-off-by: Roberto A. Foglietta >=20 > produc ^^^^^^^^^ What? Please also note the comments made to v6. Overlooked v7 in my first round some minutes ago. >=20 > Signed-off-by: Roberto A. Foglietta Duplicate Signed-off-by. > --- > meta-isar/conf/local.conf.sample | 2 +- > meta/classes/image-account-extension.bbclass | 6 +-- > meta/classes/image.bbclass | 22 +++++---- > meta/classes/initramfs.bbclass | 4 +- > wic-extract-rootfs-partition.sh | 52 ++++++++++++++++++++ > 5 files changed, 71 insertions(+), 15 deletions(-) > create mode 100755 wic-extract-rootfs-partition.sh >=20 > diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf= .sample > index 6208623e..1d7e178a 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -257,4 +257,4 @@ USER_isar[flags] +=3D "clear-text-password" > # Non git repository users can use value from 'stat -c%Y ChangeLog' > # To know more details about this variable and how to set the value refe= r below > # https://reproducible-builds.org/docs/source-date-epoch/ > -#SOURCE_DATE_EPOCH =3D > +#SOURCE_DATE_EPOCH =3D "" > diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/= image-account-extension.bbclass > index bb173b14..1d49054c 100644 > --- a/meta/classes/image-account-extension.bbclass > +++ b/meta/classes/image-account-extension.bbclass > @@ -256,11 +256,11 @@ image_postprocess_accounts() { > # chpasswd adds a random salt when running against a cle= ar-text password. > # For reproducible images, we manually generate the pass= word and use the > # SOURCE_DATE_EPOCH to generate the salt in a determinis= tic way. > - if [ -z "${SOURCE_DATE_EPOCH}"]; then > + if [ -z "${SOURCE_DATE_EPOCH}" ]; then > chpasswd_args=3D"" > else > - salt=3D"$(echo "${SOURCE_DATE_EPOCH}" | sha256sum -z= | cut -c 1-15)" > - password=3D"$(openssl passwd -6 -salt $salt "$passwo= rd")" > + salt=3D"$(echo ${SOURCE_DATE_EPOCH} | sha256sum -z |= cut -c 1-15)" > + password=3D"$(openssl passwd -6 -salt $salt $passwor= d)" > fi > fi > printf '%s:%s' "$name" "$password" | sudo chroot '${ROOTFSDI= R}' \ > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 063b9a3b..191c3940 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -310,8 +310,8 @@ python() { > # invalidate the SSTATE entries for most packages, even if they do= n't use the > # global SOURCE_DATE_EPOCH variable. > rootfs_install_pkgs_install_prepend() { > - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then > - export SOURCE_DATE_EPOCH=3D"${SOURCE_DATE_EPOCH}" > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + export SOURCE_DATE_EPOCH > fi > } > =20 > @@ -443,13 +443,17 @@ EOSUDO > =20 > # Set same time-stamps to the newly generated file/folders in the > # rootfs image for the purpose of reproducible builds. > - test ! -z "${SOURCE_DATE_EPOCH}" && \ > - sudo find ${ROOTFSDIR} -newermt \ > - "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ > - -printf "%y %p\n" \ > - -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' > ${DEPLOY_D= IR_IMAGE}/files.modified_timestamps && \ > - bbwarn "$(grep ^f ${DEPLOY_DIR_IMAGE}/files.modified_timesta= mps) \nModified above file timestamps to build image reproducibly" > - > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + fn=3D"${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > + sudo find ${ROOTFSDIR} -newermt "$(date -d@"${SOURCE_DATE_EPOCH}= " '+%Y-%m-%d %H:%M:%S')" \ > + -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH= } ';' >"$fn" > + msg=3D"" > + ncfs=3D$(egrep ^f "$fn" | wc -l) > + if [ $ncfs -gt 0 ]; then > + msg=3D"\n List of files modified could be found here= : ."${DEPLOY_DIR_IMAGE}"/files.modified_timestamps" > + fi > + bbwarn "Modified timestamp ("${SOURCE_DATE_EPOCH}") of "$ncfs" f= iles for image reproducibly.$msg" > + fi > } > addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess > =20 > diff --git a/meta/classes/initramfs.bbclass b/meta/classes/initramfs.bbcl= ass > index db283347..1b98bc06 100644 > --- a/meta/classes/initramfs.bbclass > +++ b/meta/classes/initramfs.bbclass > @@ -33,8 +33,8 @@ do_generate_initramfs() { > rootfs_do_qemu > =20 > # generate reproducible initrd if requested > - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then > - export SOURCE_DATE_EPOCH=3D"${SOURCE_DATE_EPOCH}" > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + export SOURCE_DATE_EPOCH > fi > =20 > sudo -E chroot "${INITRAMFS_ROOTFS}" \ > diff --git a/wic-extract-rootfs-partition.sh b/wic-extract-rootfs-partiti= on.sh > new file mode 100755 > index 00000000..48de0d3a > --- /dev/null > +++ b/wic-extract-rootfs-partition.sh > @@ -0,0 +1,52 @@ > +#!/bin/bash > +# > +# Copyright (c) Roberto A. Foglietta, 2023 > +# > +# Authors: > +# Roberto A. Foglietta > +# > +# SPDX-License-Identifier: MIT > +# > +#set -ex > + > +if [ "$(whoami)" !=3D "root" ]; then > + echo > + echo "WARNING: this script should run as root, sudo!" > + sudo -E $0 "$@" > + exit $? > +fi > + > +if [ -e "$1" ]; then > + fimg=3D$(readlink -e $1) > +fi > + > +cd $(dirname $0) > + > +if [ ! -n "$1" -a ! -e "$fimg" ]; then > + fimg=3D$(ls -1 build/tmp/deploy/images/*/*.wic) > + n=3D( $fimg ) > + if [ ${#n[@]} -gt 1 ]; then > + echo > + echo "WARNING: more than one image found, choose one:" > + echo > + echo "$fimg" > + echo > + exit 1 > + fi > +fi > + > +if [ ! -e "$fimg" ]; then > + echo > + echo "ERROR: no any image or block device ${1:+'$1' }found, abort!" > + echo > + exit 1 > +fi > + > +wicf=3D$fimg > +losetup -Pf $wicf > +ldev=3D$(losetup -j $wicf | cut -d: -f1 | tail -n1) > +echo loopdev:$ldev > +dd if=3D${ldev}p2 bs=3D1M of=3D${wicf/.wic/.rootfs} status=3Dprogress > +chown $(id -u).$(id -g) ${wicf/.wic/.rootfs} > +du -ms ${wicf/.wic/.rootfs} > +losetup -d $ldev > --=20 > 2.34.1 >=20