From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 05 Jul 2024 14:46:40 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-qt1-f185.google.com (mail-qt1-f185.google.com [209.85.160.185]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 465CkcmA016328 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 5 Jul 2024 14:46:39 +0200 Received: by mail-qt1-f185.google.com with SMTP id d75a77b69052e-445b5b87c64sf4750811cf.0 for ; Fri, 05 Jul 2024 05:46:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1720183593; x=1720788393; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:sender:from :to:cc:subject:date:message-id:reply-to; bh=YlLSXePAtMl8XNyagEfAA8Au7ksLRwFx7ztMNc1oTD4=; b=s/01iFGis7qW40sTQviMdhQ32//pGUzaN6yMKUhD1v0PMBqtadyvF5HXzKDeZ30dMb KENia0ffURHcBjgW+InWBvprSG+3VQlzDyHs6QhKD7Gf05zG06NuDT1i8U3xeKcSTPgA P7AIkcW+8jJl8Eq6sz4CNfHcyx17uImhJbVHfgLDvly0tADDSA2n6LId1nRKkOstna3J cuaaP2ti2R/rEsDnTQGtvgOHidSf6DYx6ZCtGK4++DId+piB/dxbddj04Yr+12L0uA2u L3fJUNJMlLRwj+YqWclwwRqVbdhSOdqUD3KvGmJeYcByDWb5qOHDL4xqYZvgnd2nrDH/ +rjg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720183593; x=1720788393; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=YlLSXePAtMl8XNyagEfAA8Au7ksLRwFx7ztMNc1oTD4=; b=J7wITL/Q//I3Bm6Hj5dFC1f4sgPjpwQ6UCBcw5+wG0F56nWixxXoncb+bBJEuxTJ06 acu1tJeYhFp8nSQRLXM6cH4Em06IexDt8QkpGz/db6nnVK0w2mivMlKckjLiT+ourVMw KcLeAEWrAH/XYlvmKUPtGC/t++HLIM7imQwJzQRj0UJrVns7UIqDHbmQEcDhzBlhNMox nrQb1Mi+2+maKDY2FdhuK2e7s7ie+CLPWaHQmhBAcv39OEcrZaG3lMhtrVxbQEA6xQiJ QTY2h119SVXmXRqZTngIBeRUUiwKVKvAAYXLxt0y7Xb9aqv+b5QtN5tm+JtAIYTz9RUi 8Iqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720183593; x=1720788393; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-sender:mime-version :subject:references:in-reply-to:message-id:to:from:date:x-beenthere :x-gm-message-state:sender:from:to:cc:subject:date:message-id :reply-to; bh=YlLSXePAtMl8XNyagEfAA8Au7ksLRwFx7ztMNc1oTD4=; b=H5Tk91vomohvbDusDK/n8Y8/zJ9zlJysJouzB/2gTIYpcjMnsA3ocCv991wCsFRoZz f8eSWf8kbjHpMXOm6v+/6i5SMe2cDo8645Ep7QdpN9iKjVAM0bstaKTTKmuLfrrRntZn 6OV4mOoA9T+r1WzD1o4mzWlq2m12AmCkTFMZyhyJuwYqFp6uM2vIRwu5OH3VSDdWZbdB nAkTXRDbiMTsyxH7sKygROC1y5VbsGV56IRuACOJOkRtbEnAWjovULmCwsa68tfx0yZV 2w910NzGOIhk/NXQBOZuuijCWucGsULIQR6gJXakmd4oz43thmhlvHOrbF3FwE9BYhFZ kZIA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=1; AJvYcCXCTf9PtxgTTyhjxejG/u03HQIhcaE5gxgCKp0Yrbq6S7dEGW4ERgO4IUaCoGQbAUuwcG1U3jSgRt7IW9IfL/ZKcGQ= X-Gm-Message-State: AOJu0YzjnymnK2uIHLRl/RauhVZsfne6/ulcuULxDPiwR/p8Bng04Foh Rl8yhVpsErGbnj4LTvJ+CbvBdwBUI46shDBmjfdcLnAycg0/fGfp X-Google-Smtp-Source: AGHT+IFtQicxv2hP4R5TsZlJ5Q7ustOqWlCA0lkmEv+u9qj5osygaZvuwdCMLN8375wKweadstEUjw== X-Received: by 2002:a05:6214:529c:b0:6b2:b5b5:124e with SMTP id 6a1803df08f44-6b5ece8f375mr45518216d6.0.1720183592914; Fri, 05 Jul 2024 05:46:32 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ad4:5cca:0:b0:6b5:1dc9:d595 with SMTP id 6a1803df08f44-6b5ea9336cels27586526d6.2.-pod-prod-06-us; Fri, 05 Jul 2024 05:46:32 -0700 (PDT) X-Received: by 2002:ad4:5ca2:0:b0:6b5:80b2:59bc with SMTP id 6a1803df08f44-6b5ecf79c64mr3597946d6.3.1720183592495; Fri, 05 Jul 2024 05:46:32 -0700 (PDT) Received: by 2002:a05:620a:371a:b0:79c:e3c:55d5 with SMTP id af79cd13be357-79d99c8bbdams85a; Fri, 5 Jul 2024 05:13:37 -0700 (PDT) X-Received: by 2002:a05:6214:19cf:b0:6b5:ecc4:f706 with SMTP id 6a1803df08f44-6b5ecfb13abmr1751936d6.2.1720181614208; Fri, 05 Jul 2024 05:13:34 -0700 (PDT) Date: Fri, 5 Jul 2024 05:13:33 -0700 (PDT) From: Adithya Balakumar To: isar-users Message-Id: <3d9226f6-f046-445e-9edc-e107111c2437n@googlegroups.com> In-Reply-To: <03db06fb48cc803750b0bdc79682cd140c027681.camel@siemens.com> References: <20231207154114.20400-1-venkata.pyla@toshiba-tsip.com> <20231207154114.20400-4-venkata.pyla@toshiba-tsip.com> <03db06fb48cc803750b0bdc79682cd140c027681.camel@siemens.com> Subject: Re: [PATCH 3/5] wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible ext4 images MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_310802_1356549698.1720181613938" X-Original-Sender: adithya190298@gmail.com Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: ceTiI/+JcYdO ------=_Part_310802_1356549698.1720181613938 Content-Type: multipart/alternative; boundary="----=_Part_310803_2135673286.1720181613938" ------=_Part_310803_2135673286.1720181613938 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Felix, I saw your mail regarding your attempt to make ext4 filesystem images from= =20 IMAGE_CMD:ext4 reproducible.=20 If you don't mind, could you briefly explain what was the problem you faced= =20 in achieving this?=20 I am also trying to understand on how to achieve the same. Thanks and Regards, Adithya Balakumar On Tuesday, April 23, 2024 at 2:47:11=E2=80=AFPM UTC+5:30 MOESSBAUER, Felix= wrote: On Thu, 2023-12-07 at 21:11 +0530, venkat...@toshiba-tsip.com wrote:=20 > From: venkata pyla =20 >=20 > E2FSPROGS_FAKE_TIME: sets fixed times for the inodes in the file=20 > system.=20 > hash_seed: creates reproducible directory indexes in the file system.=20 >=20 > Reference commit in e2fsprogs:=20 > e1f7100643a46456be107b33098f6034b0835e6d=20 >=20 > Signed-off-by: venkata pyla =20 > ---=20 > scripts/lib/wic/partition.py | 11 +++++++++++=20 > 1 file changed, 11 insertions(+)=20 >=20 > diff --git a/scripts/lib/wic/partition.py=20 > b/scripts/lib/wic/partition.py=20 > index e50871b8..90b2c037 100644=20 > --- a/scripts/lib/wic/partition.py=20 > +++ b/scripts/lib/wic/partition.py=20 > @@ -280,6 +280,17 @@ class Partition():=20 > =20 > extraopts =3D self.mkfs_extraopts or "-F -i 8192"=20 > =20 > + if os.getenv('SOURCE_DATE_EPOCH'):=20 > + sde_time =3D int(os.getenv('SOURCE_DATE_EPOCH'))=20 > + pseudo =3D "export E2FSPROGS_FAKE_TIME=3D%s;%s" % (sde_time,= =20 > pseudo)=20 > +=20 > + # Set hash_seed to generate deterministic directory=20 > indexes=20 > + namespace =3D uuid.UUID("e7429877-e7b3-4a68-a5c9-=20 > 2f2fdf33d460")=20 > + if self.fsuuid:=20 > + namespace =3D uuid.UUID(self.fsuuid)=20 > + hash_seed =3D str(uuid.uuid5(namespace, str(sde_time)))=20 > + extraopts +=3D " -E hash_seed=3D%s" % hash_seed=20 > +=20 Hi, while reworking the SDE in ISAR, I stumbled upon this as well.=20 This patch only covers the .wic part, but we need a similar patch for=20 the IMAGE_CMD:ext4 as well. I tried to mimic the pattern there, but I=20 was not able to make the .ext4 build reproducible (the diff indicated=20 that the inodes are still shuffled around). This makes me wonder if=20 mke2fs.ext4 even supports producing a reproducible rootfs.=20 Have you been able to create a bit-by-bit identical .wic of an ext4=20 partition?=20 Best regards,=20 Felix=20 > label_str =3D ""=20 > if self.label:=20 > label_str =3D "-L %s" % self.label=20 --=20 Siemens AG, Technology=20 Linux Expert Center=20 --=20 You received this message because you are subscribed to the Google Groups "= isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-users+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/= isar-users/3d9226f6-f046-445e-9edc-e107111c2437n%40googlegroups.com. ------=_Part_310803_2135673286.1720181613938 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Felix,

I saw your mail regarding your attempt to ma= ke ext4 filesystem images from IMAGE_CMD:ext4 reproducible.=C2=A0
If you don't mind, could you briefly explain what was the problem you face= d in achieving this?=C2=A0
I am also trying to understand on how = to achieve the same.

Thanks and Regards,
Adithya Balakumar


On Tuesday, April 23, 2024 at 2:47:11=E2=80=AFPM UTC+5:30 MOESSBAUER, Fel= ix wrote:
On Thu, 2023-12-07= at 21:11 +0530, venkat...@toshiba-tsip.com wrote:
> From: venkata pyla <venkat...@t= oshiba-tsip.com>
>=20
> E2FSPROGS_FAKE_TIME: sets fixed times for the inodes in the file
> system.
> hash_seed: creates reproducible directory indexes in the file sy= stem.
>=20
> Reference commit in e2fsprogs:
> e1f7100643a46456be107b33098f6034b0835e6d
>=20
> Signed-off-by: venkata pyla <ve= nkat...@toshiba-tsip.com>
> ---
> =C2=A0scripts/lib/wic/partition.py | 11 +++++++++++
> =C2=A01 file changed, 11 insertions(+)
>=20
> diff --git a/scripts/lib/wic/partition.py
> b/scripts/lib/wic/partition.py
> index e50871b8..90b2c037 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -280,6 +280,17 @@ class Partition():
> =C2=A0
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 extraopts =3D s= elf.mkfs_extraopts or "-F -i 8192"
> =C2=A0
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if os.getenv('SOURCE= _DATE_EPOCH'):
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 sde_time =3D int(os.getenv('SOURCE_DATE_EPOCH'))
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 pseudo =3D "export E2FSPROGS_FAKE_TIME=3D%s;%s" % (sde_time,
> pseudo)
> +
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 # Set hash_seed to generate deterministic directory
> indexes
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 namespace =3D uuid.UUID("e7429877-e7b3-4a68-a5c9-
> 2f2fdf33d460")
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 if self.fsuuid:
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 namespace =3D uuid.UUID(self.fsuuid)
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 hash_seed =3D str(uuid.uuid5(namespace, str(sde_time)))
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 extraopts +=3D " -E hash_seed=3D%s" % hash_seed
> +

Hi, while reworking the SDE in ISAR, I stumbled upon this as well.
This patch only covers the .wic part, but we need a similar patch for
the IMAGE_CMD:ext4 as well. I tried to mimic the pattern there, but I
was not able to make the .ext4 build reproducible (the diff indicated
that the inodes are still shuffled around). This makes me wonder if
mke2fs.ext4 even supports producing a reproducible rootfs.

Have you been able to create a bit-by-bit identical .wic of an ext4
partition?

Best regards,
Felix

> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 label_str =3D "= "
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if self.label:
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 label_str =3D "-L %s" % self.label

--=20
Siemens AG, Technology
Linux Expert Center


--
You received this message because you are subscribed to the Google Groups &= quot;isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-use= rs+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg= id/isar-users/3d9226f6-f046-445e-9edc-e107111c2437n%40googlegroups.com.=
------=_Part_310803_2135673286.1720181613938-- ------=_Part_310802_1356549698.1720181613938--