[PATCH v1 0/1] Set hash-seed for empty ext partition

[PATCH v1 0/1] Set hash-seed for empty ext partition

[Adithya Balakumar]

ext filesystems require a hash_seed to generate deterministic directory 
indexes for reproducible builds. This is handled by wic in the case where 
the rootfs plugin is used but this not handled in the case when an empty 
ext4 partition is deployed.

This patch is also shared with openembedded-core and also accepted. 
Upstream commit: 0202fb594fb05098cb8d8b6088e63beb40b5906e

Adithya Balakumar (1):
  wic/partition.py: Set hash_seed for empty ext partition

 scripts/lib/wic/partition.py | 37 ++++++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 14 deletions(-)

-- 
2.39.2

[PATCH v1 1/1] wic/partition.py: Set hash_seed for empty ext partition

[Adithya Balakumar]

From: Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>

Although setting hash_seed is handled for the rootfs plugin case, but
this is missed when deploying an empty ext partition.

Signed-off-by: Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
---
 scripts/lib/wic/partition.py | 37 ++++++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 14 deletions(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 795707ec..bf2c34d5 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -284,19 +284,8 @@ class Partition():
 
         extraopts = self.mkfs_extraopts or "-F -i 8192"
 
-        if os.getenv('SOURCE_DATE_EPOCH'):
-            sde_time = int(os.getenv('SOURCE_DATE_EPOCH'))
-            if pseudo:
-                pseudo = "export E2FSPROGS_FAKE_TIME=%s;%s " % (sde_time, pseudo)
-            else:
-                pseudo = "export E2FSPROGS_FAKE_TIME=%s; " % sde_time
-
-            # Set hash_seed to generate deterministic directory indexes
-            namespace = uuid.UUID("e7429877-e7b3-4a68-a5c9-2f2fdf33d460")
-            if self.fsuuid:
-                namespace = uuid.UUID(self.fsuuid)
-            hash_seed = str(uuid.uuid5(namespace, str(sde_time)))
-            extraopts += " -E hash_seed=%s" % hash_seed
+        # use hash_seed to generate reproducible ext4 images
+        (extraopts, pseudo) = self.get_hash_seed_ext4(extraopts, pseudo)
 
         label_str = ""
         if self.label:
@@ -344,6 +333,23 @@ class Partition():
 
         self.check_for_Y2038_problem(rootfs, native_sysroot)
 
+    def get_hash_seed_ext4(self, extraopts, pseudo):
+        if os.getenv('SOURCE_DATE_EPOCH'):
+            sde_time = int(os.getenv('SOURCE_DATE_EPOCH'))
+            if pseudo:
+                pseudo = "export E2FSPROGS_FAKE_TIME=%s;%s " % (sde_time, pseudo)
+            else:
+                pseudo = "export E2FSPROGS_FAKE_TIME=%s; " % sde_time
+
+            # Set hash_seed to generate deterministic directory indexes
+            namespace = uuid.UUID("e7429877-e7b3-4a68-a5c9-2f2fdf33d460")
+            if self.fsuuid:
+                namespace = uuid.UUID(self.fsuuid)
+            hash_seed = str(uuid.uuid5(namespace, str(sde_time)))
+            extraopts += " -E hash_seed=%s" % hash_seed
+
+        return (extraopts, pseudo)
+
     def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -437,13 +443,16 @@ class Partition():
 
         extraopts = self.mkfs_extraopts or "-i 8192"
 
+        # use hash_seed to generate reproducible ext4 images
+        (extraopts, pseudo) = self.get_hash_seed_ext4(extraopts, None)
+
         label_str = ""
         if self.label:
             label_str = "-L %s" % self.label
 
         mkfs_cmd = "mkfs.%s -F %s %s -U %s %s" % \
             (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
-        exec_native_cmd(mkfs_cmd, native_sysroot)
+        exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
         self.check_for_Y2038_problem(rootfs, native_sysroot)
 
-- 
2.39.2

Re: [PATCH v1 0/1] Set hash-seed for empty ext partition

[Jan Kiszka]

On 25.06.24 08:50, Adithya Balakumar wrote:
> ext filesystems require a hash_seed to generate deterministic directory 
> indexes for reproducible builds. This is handled by wic in the case where 
> the rootfs plugin is used but this not handled in the case when an empty 
> ext4 partition is deployed.
> 
> This patch is also shared with openembedded-core and also accepted. 
> Upstream commit: 0202fb594fb05098cb8d8b6088e63beb40b5906e
> 
> Adithya Balakumar (1):
>   wic/partition.py: Set hash_seed for empty ext partition
> 
>  scripts/lib/wic/partition.py | 37 ++++++++++++++++++++++--------------
>  1 file changed, 23 insertions(+), 14 deletions(-)
> 

Shouldn't we sync to a wic revision >= the one containing this?

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: [PATCH v1 0/1] Set hash-seed for empty ext partition

[Anton Mikanovich]

25/06/2024 10:37, Jan Kiszka wrote:
> Shouldn't we sync to a wic revision >= the one containing this?
>
> Jan
>
Hello Jan,

This is our preferred process by default, but this commit is in oe-core 
master
branch only for now. We should probably wait for release or backporting.

We are currently using the latest Scarthgap LTS version 5.0 branch.

Re: [PATCH v1 0/1] Set hash-seed for empty ext partition

[Jan Kiszka]

On 25.06.24 15:09, Anton Mikanovich wrote:
> 25/06/2024 10:37, Jan Kiszka wrote:
>> Shouldn't we sync to a wic revision >= the one containing this?
>>
>> Jan
>>
> Hello Jan,
> 
> This is our preferred process by default, but this commit is in oe-core
> master
> branch only for now. We should probably wait for release or backporting.
> 
> We are currently using the latest Scarthgap LTS version 5.0 branch.
> 

Maybe propose the commit for 5.0-LTS branch and then sync that in? But
if you are fine with taking this backport, I will not hold you back. It
will be valuable for us, that is for sure.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

RE: [PATCH v1 0/1] Set hash-seed for empty ext partition

[Adithya.Balakumar]

Hi Anton and Jan,

Thanks for the feedback. Let me propose this patch to Scarthgap also. And once accepted in Scarthgap, I will share the patch with isar again.

Thanks,
Adithya 

-----Original Message-----
From: Jan Kiszka <jan.kiszka@siemens.com> 
Sent: Tuesday, June 25, 2024 8:00 PM
To: Anton Mikanovich <amikan@ilbers.de>; balakumar adithya(ＴＳＩＰ TEUR) <Adithya.Balakumar@toshiba-tsip.com>; isar-users@googlegroups.com
Cc: hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>
Subject: Re: [PATCH v1 0/1] Set hash-seed for empty ext partition

On 25.06.24 15:09, Anton Mikanovich wrote:
> 25/06/2024 10:37, Jan Kiszka wrote:
>> Shouldn't we sync to a wic revision >= the one containing this?
>>
>> Jan
>>
> Hello Jan,
> 
> This is our preferred process by default, but this commit is in 
> oe-core master branch only for now. We should probably wait for 
> release or backporting.
> 
> We are currently using the latest Scarthgap LTS version 5.0 branch.
> 

Maybe propose the commit for 5.0-LTS branch and then sync that in? But if you are fine with taking this backport, I will not hold you back. It will be valuable for us, that is for sure.

Jan

--
Siemens AG, Technology
Linux Expert Center