From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7189000421229199360 X-Received: by 2002:a05:600c:46d0:b0:3da:1ed2:15b5 with SMTP id q16-20020a05600c46d000b003da1ed215b5mr588536wmo.35.1673821920889; Sun, 15 Jan 2023 14:32:00 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:3b0c:b0:3cf:72dc:df8 with SMTP id m12-20020a05600c3b0c00b003cf72dc0df8ls6692335wms.0.-pod-canary-gmail; Sun, 15 Jan 2023 14:31:59 -0800 (PST) X-Google-Smtp-Source: AMrXdXuLs/ogvS6FLO3HFdZXD3KBRz04DMZt+P9smlp7fmEmU4aupBXOpDv4Q3n7VmiPl498NGtM X-Received: by 2002:a05:600c:1e19:b0:3da:2ba4:b97 with SMTP id ay25-20020a05600c1e1900b003da2ba40b97mr5110325wmb.19.1673821919751; Sun, 15 Jan 2023 14:31:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673821919; cv=none; d=google.com; s=arc-20160816; b=AW5xta6ypD4OxilTXdsu/C7C+nXH9428d+U0FZpHTxmxwH2KfTO40Q09gMtO1MUSkD 53ZvuK+3bjNI+aE0/PnpTdZEuJcPl179eNZwFZvwS2sUzgS5Qh0yxfgoQoPkK11KRg7g v1bW1jk00XDDjKFMa2eJaER49Jrk/YZaH6r13r6mKrHSeAYReq9trtAmo16g9909Wl1s /R8lbi2ixShHqX8hQy7ORcDX1aABXPu4ylib5M5WADMzaKI44yhkM/+G54Wrm25qTp+0 8TBy+swFUBCCzFC79yVFQDQb+SNzrw3cFi7/OfkjYIDesoEUN1Eex+MMOUObfirLBIif ZudA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:mime-version:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:dkim-signature; bh=7asfl7hW+SLKqOuu2mrk/LTz4F2/r2RHoniOXyyhveU=; b=mnNe5S2lFimmCcdxAxJyA+h+MSMYSatHRqPfBEqYXxx/IY8kkRVZn1he3VW/X5e/3Q FEjwcgFifAV0f3RbXDn42atDwFFwHLZzwN6OBiQU4J1uJIRuoQrfsSN0XNDP417RP3/4 LU8Jw4o72sCMRt7XqzJZlpy26Qg2GLGAfFCjmsVqOwOysJ/FzWCsjO1vpw+oAUdJr+vg WPu6RITRi3uwWny2pVR8tlfOgC5CsJEljNX6EsRHPF9ioVFsGA2b8/CeL6UYh5nVhoKb 4ZMauF1jcHkpSZirOFiu7EzX758v+Nui/tj3jzD7jrTfvQ2bwvaDlUbDIjUH43ztLKRN JspQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=WWQVq2zX; spf=pass (google.com: domain of fm-68982-20230115223159951350ad30e40ffd7d-hhvrhz@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-68982-20230115223159951350ad30e40ffd7d-HHVrhz@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id e4-20020a05600c448400b003d9c774d43fsi437999wmo.2.2023.01.15.14.31.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jan 2023 14:31:59 -0800 (PST) Received-SPF: pass (google.com: domain of fm-68982-20230115223159951350ad30e40ffd7d-hhvrhz@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=WWQVq2zX; spf=pass (google.com: domain of fm-68982-20230115223159951350ad30e40ffd7d-hhvrhz@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-68982-20230115223159951350ad30e40ffd7d-HHVrhz@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20230115223159951350ad30e40ffd7d for ; Sun, 15 Jan 2023 23:31:59 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=florian.bezdeka@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=7asfl7hW+SLKqOuu2mrk/LTz4F2/r2RHoniOXyyhveU=; b=WWQVq2zXf82HUa3e6TWzNfhcqFwnN+0lTJ7efadjSuR8aFTlLllwtlQRnEkKt106LIoR6I EsBK4t3heojtfD+jMwV2kbg6rCKczhDSOyrSjXmO9SOkISXkB9zUDkpHpB/RgrTpuDGeUfwD m3kVKDveZOFH5vXnCLIMHs4HDiP+c=; Message-ID: <551831fb188675b0c6d1a879f1a8b251efce4e4a.camel@siemens.com> Subject: Re: [PATCH v6] suggested changes for reproducibility patchset v6 From: Florian Bezdeka To: roberto.foglietta@linuxteam.org, isar-users@googlegroups.com Cc: roberto.foglietta@gmail.com Date: Sun, 15 Jan 2023 23:31:58 +0100 In-Reply-To: <20230115215310.732295-1-roberto.foglietta@linuxteam.org> References: <20230115215310.732295-1-roberto.foglietta@linuxteam.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-68982:519-21489:flowmailer X-TUID: 2rRIBodq6pv+ On Sun, 2023-01-15 at 22:53 +0100, roberto.foglietta@linuxteam.org wrote: > From: "Roberto A. Foglietta" >=20 > suggested changes for reproducibility patchset >=20 > WARNING: eval-image-1.0-r0 do_rootfs_finalize: modified timestamp (167362= 8837) of 3 files for image reproducibly > List of files modified could be found here: ./build/tmp/deploy/i= mages/debx86/files.modified_timestamps >=20 Can't follow. Patches / Commits need proper description (=3D commit message). I guess you fixed a warning, but the warning itself (=3D list of modified files) was inside the mentioned file, so we have to guess what changed? > v.2: rebased on current ilbers:next >=20 > v.3: new script added: wic-extract-rootfs-partition.sh [image.wic] >=20 > v.4: example with for epoch generation from git >=20 > v.5: reverted the example and rework some few code >=20 > v.6: the 1st part of the warning shows up each time the epoch is used > while the 2nd line appears only when some files has been touched > This allows the user to know the current situation aboat epoch. Version information does not belong here. See below. >=20 > Signed-off-by: Roberto A. Foglietta > --- Comments like changes between versions of your patches should be mentioned here. Not inside the commit message. > meta-isar/conf/local.conf.sample | 2 +- > meta/classes/image-account-extension.bbclass | 6 +-- > meta/classes/image.bbclass | 20 ++++---- > meta/classes/initramfs.bbclass | 4 +- > wic-extract-rootfs-partition.sh | 52 ++++++++++++++++++++ > 5 files changed, 69 insertions(+), 15 deletions(-) > create mode 100755 wic-extract-rootfs-partition.sh >=20 > diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf= .sample > index 6208623e..1d7e178a 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -257,4 +257,4 @@ USER_isar[flags] +=3D "clear-text-password" > # Non git repository users can use value from 'stat -c%Y ChangeLog' > # To know more details about this variable and how to set the value refe= r below > # https://reproducible-builds.org/docs/source-date-epoch/ > -#SOURCE_DATE_EPOCH =3D > +#SOURCE_DATE_EPOCH =3D "" > diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/= image-account-extension.bbclass > index bb173b14..1d49054c 100644 > --- a/meta/classes/image-account-extension.bbclass > +++ b/meta/classes/image-account-extension.bbclass > @@ -256,11 +256,11 @@ image_postprocess_accounts() { > # chpasswd adds a random salt when running against a cle= ar-text password. > # For reproducible images, we manually generate the pass= word and use the > # SOURCE_DATE_EPOCH to generate the salt in a determinis= tic way. > - if [ -z "${SOURCE_DATE_EPOCH}"]; then > + if [ -z "${SOURCE_DATE_EPOCH}" ]; then > chpasswd_args=3D"" > else > - salt=3D"$(echo "${SOURCE_DATE_EPOCH}" | sha256sum -z= | cut -c 1-15)" > - password=3D"$(openssl passwd -6 -salt $salt "$passwo= rd")" > + salt=3D"$(echo ${SOURCE_DATE_EPOCH} | sha256sum -z |= cut -c 1-15)" > + password=3D"$(openssl passwd -6 -salt $salt $passwor= d)" > fi > fi > printf '%s:%s' "$name" "$password" | sudo chroot '${ROOTFSDI= R}' \ > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 063b9a3b..bf3dfea8 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -310,8 +310,8 @@ python() { > # invalidate the SSTATE entries for most packages, even if they do= n't use the > # global SOURCE_DATE_EPOCH variable. > rootfs_install_pkgs_install_prepend() { > - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then > - export SOURCE_DATE_EPOCH=3D"${SOURCE_DATE_EPOCH}" > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + export SOURCE_DATE_EPOCH > fi > } > =20 > @@ -443,13 +443,15 @@ EOSUDO > =20 > # Set same time-stamps to the newly generated file/folders in the > # rootfs image for the purpose of reproducible builds. > - test ! -z "${SOURCE_DATE_EPOCH}" && \ > - sudo find ${ROOTFSDIR} -newermt \ > - "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ > - -printf "%y %p\n" \ > - -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' > ${DEPLOY_D= IR_IMAGE}/files.modified_timestamps && \ > - bbwarn "$(grep ^f ${DEPLOY_DIR_IMAGE}/files.modified_timesta= mps) \nModified above file timestamps to build image reproducibly" > - > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + msg=3D"" > + fn=3D"${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > + if sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOC= H} '+%Y-%m-%d %H:%M:%S')" \ > + -printf "%y %p\n" -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH= } ';' >"$fn"; then > + msg=3D"\n List of files modified could be found here:= .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > + fi > + bbwarn "Modified timestamp (${SOURCE_DATE_EPOCH}) of $(egrep ^f = '$fn' | wc -l) files for image reproducibly.$msg" > + fi > } > addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess > =20 > diff --git a/meta/classes/initramfs.bbclass b/meta/classes/initramfs.bbcl= ass > index db283347..1b98bc06 100644 > --- a/meta/classes/initramfs.bbclass > +++ b/meta/classes/initramfs.bbclass > @@ -33,8 +33,8 @@ do_generate_initramfs() { > rootfs_do_qemu > =20 > # generate reproducible initrd if requested > - if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then > - export SOURCE_DATE_EPOCH=3D"${SOURCE_DATE_EPOCH}" > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + export SOURCE_DATE_EPOCH > fi > =20 > sudo -E chroot "${INITRAMFS_ROOTFS}" \ > diff --git a/wic-extract-rootfs-partition.sh b/wic-extract-rootfs-partiti= on.sh > new file mode 100755 > index 00000000..48de0d3a > --- /dev/null > +++ b/wic-extract-rootfs-partition.sh > @@ -0,0 +1,52 @@ > +#!/bin/bash > +# > +# Copyright (c) Roberto A. Foglietta, 2023 > +# > +# Authors: > +# Roberto A. Foglietta > +# > +# SPDX-License-Identifier: MIT > +# > +#set -ex > + > +if [ "$(whoami)" !=3D "root" ]; then > + echo > + echo "WARNING: this script should run as root, sudo!" > + sudo -E $0 "$@" > + exit $? > +fi > + > +if [ -e "$1" ]; then > + fimg=3D$(readlink -e $1) > +fi > + > +cd $(dirname $0) > + > +if [ ! -n "$1" -a ! -e "$fimg" ]; then > + fimg=3D$(ls -1 build/tmp/deploy/images/*/*.wic) > + n=3D( $fimg ) > + if [ ${#n[@]} -gt 1 ]; then > + echo > + echo "WARNING: more than one image found, choose one:" > + echo > + echo "$fimg" > + echo > + exit 1 > + fi > +fi > + > +if [ ! -e "$fimg" ]; then > + echo > + echo "ERROR: no any image or block device ${1:+'$1' }found, abort!" > + echo > + exit 1 > +fi > + > +wicf=3D$fimg > +losetup -Pf $wicf > +ldev=3D$(losetup -j $wicf | cut -d: -f1 | tail -n1) > +echo loopdev:$ldev > +dd if=3D${ldev}p2 bs=3D1M of=3D${wicf/.wic/.rootfs} status=3Dprogress > +chown $(id -u).$(id -g) ${wicf/.wic/.rootfs} > +du -ms ${wicf/.wic/.rootfs} > +losetup -d $ldev > --=20 > 2.34.1 >=20