RE: [PATCH 3/5] wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible ext4 images

[<Adithya.Balakumar@toshiba-tsip.com>]

[-- Attachment #1: Type: text/plain, Size: 5308 bytes --]

Hi Felix,

To answer your question, No, these patches alone are not enough to make ext4 filesystem images reproducible.
These patches were mainly tested with isar-cip-core security images which has /home ad /var mounted as ext4 filesystem images.

With these patches I noticed the /home ext4 partition is reproducible but /var required an extra fix (https://groups.google.com/g/isar-users/c/RsjRjzigLOE)

But, I see reproducibility problems when entire rootfs is deployed in an ext4 filesystem (in the case of wic and with IMAGE_CMD:ext4). I even raised this point in the ISAR ML (https://groups.google.com/g/isar-users/c/Ll7t4G41Lfo)

That’s when I saw your previous mail.

Thanks and Regards,
Adithya Balakumar

From: Adithya Balakumar <adithya190298@gmail.com>
Sent: Monday, July 8, 2024 4:28 PM
To: balakumar adithya(ＴＳＩＰ TEUR) <Adithya.Balakumar@toshiba-tsip.com>
Subject: Fwd: [PATCH 3/5] wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible ext4 images


---------- Forwarded message ---------
From: MOESSBAUER, Felix <felix.moessbauer@siemens.com<mailto:felix.moessbauer@siemens.com>>
Date: Mon, 8 Jul, 2024, 16:13
Subject: Re: [PATCH 3/5] wic: use E2FSPROGS_FAKE_TIME and hash_seed to generate reproducible ext4 images
To: isar-users@googlegroups.com<mailto:isar-users@googlegroups.com> <isar-users@googlegroups.com<mailto:isar-users@googlegroups.com>>, adithya190298@gmail.com<mailto:adithya190298@gmail.com> <adithya190298@gmail.com<mailto:adithya190298@gmail.com>>


On Fri, 2024-07-05 at 05:13 -0700, Adithya Balakumar wrote:
> Hi Felix,
>
> I saw your mail regarding your attempt to make ext4 filesystem images
> from IMAGE_CMD:ext4 reproducible.
> If you don't mind, could you briefly explain what was the problem you
> faced in achieving this?

Hi, the problem is stated below:

> the diff indicated that the inodes are still shuffled around). This
makes me wonder if mke2fs.ext4 even supports producing a reproducible
rootfs.

I just copied the pattern from wic, but for whatever reason the inodes
still were not deterministic.

Are you sure, that this patch is sufficient to make the ext4
reproducible?

Felix

> I am also trying to understand on how to achieve the same.
>
> Thanks and Regards,
> Adithya Balakumar
>
>
> On Tuesday, April 23, 2024 at 2:47:11 PM UTC+5:30 MOESSBAUER, Felix
> wrote:
> > On Thu, 2023-12-07 at 21:11 +0530, venkat...@toshiba-tsip.com<mailto:venkat...@toshiba-tsip.com>
> > wrote:
> > > From: venkata pyla <venkat...@toshiba-tsip.com<mailto:venkat...@toshiba-tsip.com>>
> > >
> > > E2FSPROGS_FAKE_TIME: sets fixed times for the inodes in the file
> > > system.
> > > hash_seed: creates reproducible directory indexes in the file
> > > system.
> > >
> > > Reference commit in e2fsprogs:
> > > e1f7100643a46456be107b33098f6034b0835e6d
> > >
> > > Signed-off-by: venkata pyla <venkat...@toshiba-tsip.com<mailto:venkat...@toshiba-tsip.com>>
> > > ---
> > >  scripts/lib/wic/partition.py | 11 +++++++++++
> > >  1 file changed, 11 insertions(+)
> > >
> > > diff --git a/scripts/lib/wic/partition.py
> > > b/scripts/lib/wic/partition.py
> > > index e50871b8..90b2c037 100644
> > > --- a/scripts/lib/wic/partition.py
> > > +++ b/scripts/lib/wic/partition.py
> > > @@ -280,6 +280,17 @@ class Partition():
> > >
> > >          extraopts = self.mkfs_extraopts or "-F -i 8192"
> > >
> > > +        if os.getenv('SOURCE_DATE_EPOCH'):
> > > +            sde_time = int(os.getenv('SOURCE_DATE_EPOCH'))
> > > +            pseudo = "export E2FSPROGS_FAKE_TIME=%s;%s" %
> > > (sde_time,
> > > pseudo)
> > > +
> > > +            # Set hash_seed to generate deterministic directory
> > > indexes
> > > +            namespace = uuid.UUID("e7429877-e7b3-4a68-a5c9-
> > > 2f2fdf33d460")
> > > +            if self.fsuuid:
> > > +                namespace = uuid.UUID(self.fsuuid)
> > > +            hash_seed = str(uuid.uuid5(namespace,
> > > str(sde_time)))
> > > +            extraopts += " -E hash_seed=%s" % hash_seed
> > > +
> >
> > Hi, while reworking the SDE in ISAR, I stumbled upon this as well.
> > This patch only covers the .wic part, but we need a similar patch
> > for
> > the IMAGE_CMD:ext4 as well. I tried to mimic the pattern there, but
> > I
> > was not able to make the .ext4 build reproducible (the diff
> > indicated
> > that the inodes are still shuffled around). This makes me wonder if
> > mke2fs.ext4 even supports producing a reproducible rootfs.
> >
> > Have you been able to create a bit-by-bit identical .wic of an ext4
> > partition?
> >
> > Best regards,
> > Felix
> >
> > >          label_str = ""
> > >          if self.label:
> > >              label_str = "-L %s" % self.label
> >
> > --
> > Siemens AG, Technology
> > Linux Expert Center
> >
> >

--
Siemens AG, Technology
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/TYCPR01MB96695360B889AA307AC0FBEBC4DA2%40TYCPR01MB9669.jpnprd01.prod.outlook.com.

[-- Attachment #2: Type: text/html, Size: 11466 bytes --]