[RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

Hi ISAR developers,

this series is the summary of a nice journey through the file system
jungle regarding Y2038 problem. It all began with a warning which is
reported by kernels >= 5.4:

ext4 filesystem being mounted at (mountpoint) supports timestamps until
2038 (0x7fffffff)

I guess that most ISAR layers are using the Debian kernels, so that
warning was not recognized yet or at least not very often.

When reading this warning I was surprised. Shouldn't a modern file
system like ext4 be Y2038-safe? As it turned out it depends on the
inode size if an ext4 file system is safe or not. So why was the
inode size not sufficient in my case?

The inode size is chosen during file system generation and depends on
the size of the file system that is going to be created. For details
let's have a look at `man mke2fs`:

-T usage-type[,...]
    Specify how the filesystem is going to be used, so that mke2fs can
    choose optimal filesystem parameters for that use. The usage types
    that are supported are defined in the configuration file
    /etc/mke2fs.conf. The user may specify one or more usage types
    using a comma separated list.

    If this option is is not specified, mke2fs will pick a single
    default usage type based on the size of the filesystem to be
    created. If the filesystem size is less than 3 megabytes, mke2fs
    will use the filesystem type floppy. If the filesystem size is
    greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
    will use the filesystem type small.

The relevant parts from /etc/mke2fs.conf:
[fs_types]
...
        small = {
                blocksize = 1024
                inode_size = 128
                inode_ratio = 4096
        }
...

So whenever you create an ext4 file system with less than 512MB in
size you will end up with 128 byte inodes and your file system is
not Y2038-safe.

The ISAR part:
ext4 may often be used in combination with the expand-on-first-boot
recipe / feature. So whenever creating a small partition (e.g. inside
a wic file) and extending it later may result in a Y2038 affected ext4
file system.

That is exactly what happened to me and I would like to make sure that
all other ISAR users are aware of this situation.

Valid workarounds found so far:
 - Tell wic that an partition will grow:
   Add `--mkfs-extraopts "-T ext4"` to your wic partition definition
 - Set the inode size to 256 (for small ext4 partitions)
   Add `--mkfs-extraopts "-I 256"` to your wic partition definition

The upstream part:
None of the following patches has been sent to any upstream (OE)
mailing lists yet but hopefully that will happen soon. So far: Any
comments welcome!

Best regards,
Florian

Florian Bezdeka (2):
  wic-img: Forward warnings from wic to bitbake
  wic: Warn if an ext filesystem affected by the Y2038 problem is used

 meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
 scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+), 5 deletions(-)

-- 
2.29.2

[RFC PATCH 1/2] wic-img: Forward warnings from wic to bitbake

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

By now warnings generated by wic are visible in the wic log file, but
they are note visible in the bitbake output (stdout) so they can easily
be overlooked.

To forward the warnings from the logfile to bitbake the logfile is
being parsed once the wic image generation has been done.

The bitbake task (do_wic_image) is now a python function which first
calls the previous (unchanged) shell function (now called
generate_wic_image) which does the image generation and checks for wic
warnings afterwards.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 meta/classes/wic-img.bbclass | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index bbf5dd8..a11e493 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -129,7 +129,21 @@ do_rootfs_wicenv[prefuncs] = 'set_image_size'
 
 WIC_IMAGE_FILE ="${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.img"
 
-do_wic_image() {
+python check_for_wic_warnings() {
+    with open("{}/log.do_wic_image".format(d.getVar("T"))) as f:
+        for line in f.readlines():
+            if line.startswith("WARNING"):
+                bb.warn(line.strip())
+}
+
+do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
+python do_wic_image() {
+    bb.build.exec_func("generate_wic_image", d)
+    bb.build.exec_func("check_for_wic_warnings", d)
+}
+addtask wic_image before do_image after do_image_tools
+
+generate_wic_image() {
     buildchroot_do_mounts
     sudo -s <<'EOSUDO'
         ( flock 9
@@ -186,7 +200,3 @@ EOSUDO
     rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
     rm -rf ${IMAGE_ROOTFS}/../pseudo
 }
-
-do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-
-addtask wic_image before do_image after do_image_tools
-- 
2.29.2

[RFC PATCH 2/2] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

From: Florian Bezdeka <florian.bezdeka@siemens.com>

We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.

While ext2 and ext3 file systems are always affected by the time overflow,
let's warn the user if these file systems are still being used.

If ext4 is affected depends on the inode size chosen during file system
creation. At least 256 bytes are necessary to be save. As ext4 is
used very often (and partitions may be small first and extended later)
this might be an issue for many users.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 85eb15c..c165457 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -270,6 +270,8 @@ class Partition():
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
                              native_sysroot, pseudo):
         """
@@ -356,6 +358,8 @@ class Partition():
             (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot)
 
+        self.check_for_Y2038_problem(rootfs, native_sysroot)
+
     def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
                                       native_sysroot):
         """
@@ -417,3 +421,37 @@ class Partition():
 
         mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, path)
         exec_native_cmd(mkswap_cmd, native_sysroot)
+
+    def check_for_Y2038_problem(self, rootfs, native_sysroot):
+        """
+        Check if the filesystem is affected by the Y2038 problem
+        (Y2038 problem = 32 bit time_t overflow in January 2038)
+        """
+        def get_err_str(part):
+            err = "The {} filesystem {} has no Y2038 support."
+            if part.mountpoint:
+                args = [part.fstype, "mounted at %s" % part.mountpoint]
+            elif part.label:
+                args = [part.fstype, "labeled %s" % part.label]
+            elif part.part_name:
+                args = [part.fstype, "in partition %s" % part.part_name]
+            else:
+                args = [part.fstype, ""]
+            return err.format(*args)
+
+        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
+
+        # ext2 and ext3 are always affected by the Y2038 problem
+        if self.fstype in ["ext2", "ext3"]:
+            logger.warn(get_err_str(self))
+            return
+
+        # if ext4 is affected by the Y2038 problem depends on the inode size
+        # Remember: inode size depends on the file system size
+        for line in out.splitlines():
+            if line.startswith("Inode size:"):
+                size = int(line.split(":")[1].strip())
+                if size < 256:
+                    logger.warn("%s Inodes (of size %d) are too small." % \
+                                (get_err_str(self), size))
+                break
\ No newline at end of file
-- 
2.29.2

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Anton Mikanovich]

01.02.2021 21:58, florian.bezdeka@siemens.com wrote:
> From: Florian Bezdeka <florian.bezdeka@siemens.com>
>
> Hi ISAR developers,
>
> this series is the summary of a nice journey through the file system
> jungle regarding Y2038 problem. It all began with a warning which is
> reported by kernels >= 5.4:
>
> ext4 filesystem being mounted at (mountpoint) supports timestamps until
> 2038 (0x7fffffff)
>
> I guess that most ISAR layers are using the Debian kernels, so that
> warning was not recognized yet or at least not very often.
>
> When reading this warning I was surprised. Shouldn't a modern file
> system like ext4 be Y2038-safe? As it turned out it depends on the
> inode size if an ext4 file system is safe or not. So why was the
> inode size not sufficient in my case?
>
> The inode size is chosen during file system generation and depends on
> the size of the file system that is going to be created. For details
> let's have a look at `man mke2fs`:
>
> -T usage-type[,...]
>      Specify how the filesystem is going to be used, so that mke2fs can
>      choose optimal filesystem parameters for that use. The usage types
>      that are supported are defined in the configuration file
>      /etc/mke2fs.conf. The user may specify one or more usage types
>      using a comma separated list.
>
>      If this option is is not specified, mke2fs will pick a single
>      default usage type based on the size of the filesystem to be
>      created. If the filesystem size is less than 3 megabytes, mke2fs
>      will use the filesystem type floppy. If the filesystem size is
>      greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
>      will use the filesystem type small.
>
> The relevant parts from /etc/mke2fs.conf:
> [fs_types]
> ...
>          small = {
>                  blocksize = 1024
>                  inode_size = 128
>                  inode_ratio = 4096
>          }
> ...
>
> So whenever you create an ext4 file system with less than 512MB in
> size you will end up with 128 byte inodes and your file system is
> not Y2038-safe.
>
> The ISAR part:
> ext4 may often be used in combination with the expand-on-first-boot
> recipe / feature. So whenever creating a small partition (e.g. inside
> a wic file) and extending it later may result in a Y2038 affected ext4
> file system.
>
> That is exactly what happened to me and I would like to make sure that
> all other ISAR users are aware of this situation.
>
> Valid workarounds found so far:
>   - Tell wic that an partition will grow:
>     Add `--mkfs-extraopts "-T ext4"` to your wic partition definition
>   - Set the inode size to 256 (for small ext4 partitions)
>     Add `--mkfs-extraopts "-I 256"` to your wic partition definition
>
> The upstream part:
> None of the following patches has been sent to any upstream (OE)
> mailing lists yet but hopefully that will happen soon. So far: Any
> comments welcome!
>
> Best regards,
> Florian
>
> Florian Bezdeka (2):
>    wic-img: Forward warnings from wic to bitbake
>    wic: Warn if an ext filesystem affected by the Y2038 problem is used
>
>   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
>   scripts/lib/wic/partition.py | 38 ++++++++++++++++++++++++++++++++++++
>   2 files changed, 53 insertions(+), 5 deletions(-)
>
Applied to next, thanks.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Henning Schild]

Hi all,

i never got around to reviewing this. But did we just fork wic? These
patches need to go into wic and we later backport them once they are
accepted upstream.

Maybe they are already ... did not check.

When it comes to changing bitbake or wic, we should really not ... We
have forks of some files, like the wic plugins and bitbake config,
those are fine but should also stay very close to upstream.

The recently applied patch from Vijai also violates that. Since the
fork of the plugins was not updated with the wic bump and the repair
just takes a few bits of what we probably should take.

Henning

Am Thu, 11 Feb 2021 11:07:52 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:
> > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> >
> > Hi ISAR developers,
> >
> > this series is the summary of a nice journey through the file system
> > jungle regarding Y2038 problem. It all began with a warning which is
> > reported by kernels >= 5.4:
> >
> > ext4 filesystem being mounted at (mountpoint) supports timestamps
> > until 2038 (0x7fffffff)
> >
> > I guess that most ISAR layers are using the Debian kernels, so that
> > warning was not recognized yet or at least not very often.
> >
> > When reading this warning I was surprised. Shouldn't a modern file
> > system like ext4 be Y2038-safe? As it turned out it depends on the
> > inode size if an ext4 file system is safe or not. So why was the
> > inode size not sufficient in my case?
> >
> > The inode size is chosen during file system generation and depends
> > on the size of the file system that is going to be created. For
> > details let's have a look at `man mke2fs`:
> >
> > -T usage-type[,...]
> >      Specify how the filesystem is going to be used, so that mke2fs
> > can choose optimal filesystem parameters for that use. The usage
> > types that are supported are defined in the configuration file
> >      /etc/mke2fs.conf. The user may specify one or more usage types
> >      using a comma separated list.
> >
> >      If this option is is not specified, mke2fs will pick a single
> >      default usage type based on the size of the filesystem to be
> >      created. If the filesystem size is less than 3 megabytes,
> > mke2fs will use the filesystem type floppy. If the filesystem size
> > is greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
> >      will use the filesystem type small.
> >
> > The relevant parts from /etc/mke2fs.conf:
> > [fs_types]
> > ...
> >          small = {
> >                  blocksize = 1024
> >                  inode_size = 128
> >                  inode_ratio = 4096
> >          }
> > ...
> >
> > So whenever you create an ext4 file system with less than 512MB in
> > size you will end up with 128 byte inodes and your file system is
> > not Y2038-safe.
> >
> > The ISAR part:
> > ext4 may often be used in combination with the expand-on-first-boot
> > recipe / feature. So whenever creating a small partition (e.g.
> > inside a wic file) and extending it later may result in a Y2038
> > affected ext4 file system.
> >
> > That is exactly what happened to me and I would like to make sure
> > that all other ISAR users are aware of this situation.
> >
> > Valid workarounds found so far:
> >   - Tell wic that an partition will grow:
> >     Add `--mkfs-extraopts "-T ext4"` to your wic partition
> > definition
> >   - Set the inode size to 256 (for small ext4 partitions)
> >     Add `--mkfs-extraopts "-I 256"` to your wic partition definition
> >
> > The upstream part:
> > None of the following patches has been sent to any upstream (OE)
> > mailing lists yet but hopefully that will happen soon. So far: Any
> > comments welcome!
> >
> > Best regards,
> > Florian
> >
> > Florian Bezdeka (2):
> >    wic-img: Forward warnings from wic to bitbake
> >    wic: Warn if an ext filesystem affected by the Y2038 problem is
> > used
> >
> >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> >   scripts/lib/wic/partition.py | 38
> > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > insertions(+), 5 deletions(-) 
> Applied to next, thanks.
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Jan Kiszka]

On 11.02.21 09:23, Henning Schild wrote:
> Hi all,
> 
> i never got around to reviewing this. But did we just fork wic? These
> patches need to go into wic and we later backport them once they are
> accepted upstream.
> 
> Maybe they are already ... did not check.
> 
> When it comes to changing bitbake or wic, we should really not ... We
> have forks of some files, like the wic plugins and bitbake config,
> those are fine but should also stay very close to upstream.
> 
> The recently applied patch from Vijai also violates that. Since the
> fork of the plugins was not updated with the wic bump and the repair
> just takes a few bits of what we probably should take.
> 

If you are referring to
https://groups.google.com/d/msgid/isar-users/20201126091750.28048-1-Vijaikumar_Kanagarajan%40mentor.com:
That one was "only" patching an isar version, though I agree that we
should make sure to realign it with the original plugins if we are now
imbalanced.

This one here is more critical as it changed a formerly vanilla wic
file. That should be fixed quickly.

Florian, maybe you can propose a similar change to OE upstream? In the
meantime, is there a chance to move the changes out of partition.py, to
a file that is isar-specific?

Jan

> Henning
> 
> Am Thu, 11 Feb 2021 11:07:52 +0300
> schrieb Anton Mikanovich <amikan@ilbers.de>:
> 
>> 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:
>>> From: Florian Bezdeka <florian.bezdeka@siemens.com>
>>>
>>> Hi ISAR developers,
>>>
>>> this series is the summary of a nice journey through the file system
>>> jungle regarding Y2038 problem. It all began with a warning which is
>>> reported by kernels >= 5.4:
>>>
>>> ext4 filesystem being mounted at (mountpoint) supports timestamps
>>> until 2038 (0x7fffffff)
>>>
>>> I guess that most ISAR layers are using the Debian kernels, so that
>>> warning was not recognized yet or at least not very often.
>>>
>>> When reading this warning I was surprised. Shouldn't a modern file
>>> system like ext4 be Y2038-safe? As it turned out it depends on the
>>> inode size if an ext4 file system is safe or not. So why was the
>>> inode size not sufficient in my case?
>>>
>>> The inode size is chosen during file system generation and depends
>>> on the size of the file system that is going to be created. For
>>> details let's have a look at `man mke2fs`:
>>>
>>> -T usage-type[,...]
>>>      Specify how the filesystem is going to be used, so that mke2fs
>>> can choose optimal filesystem parameters for that use. The usage
>>> types that are supported are defined in the configuration file
>>>      /etc/mke2fs.conf. The user may specify one or more usage types
>>>      using a comma separated list.
>>>
>>>      If this option is is not specified, mke2fs will pick a single
>>>      default usage type based on the size of the filesystem to be
>>>      created. If the filesystem size is less than 3 megabytes,
>>> mke2fs will use the filesystem type floppy. If the filesystem size
>>> is greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
>>>      will use the filesystem type small.
>>>
>>> The relevant parts from /etc/mke2fs.conf:
>>> [fs_types]
>>> ...
>>>          small = {
>>>                  blocksize = 1024
>>>                  inode_size = 128
>>>                  inode_ratio = 4096
>>>          }
>>> ...
>>>
>>> So whenever you create an ext4 file system with less than 512MB in
>>> size you will end up with 128 byte inodes and your file system is
>>> not Y2038-safe.
>>>
>>> The ISAR part:
>>> ext4 may often be used in combination with the expand-on-first-boot
>>> recipe / feature. So whenever creating a small partition (e.g.
>>> inside a wic file) and extending it later may result in a Y2038
>>> affected ext4 file system.
>>>
>>> That is exactly what happened to me and I would like to make sure
>>> that all other ISAR users are aware of this situation.
>>>
>>> Valid workarounds found so far:
>>>   - Tell wic that an partition will grow:
>>>     Add `--mkfs-extraopts "-T ext4"` to your wic partition
>>> definition
>>>   - Set the inode size to 256 (for small ext4 partitions)
>>>     Add `--mkfs-extraopts "-I 256"` to your wic partition definition
>>>
>>> The upstream part:
>>> None of the following patches has been sent to any upstream (OE)
>>> mailing lists yet but hopefully that will happen soon. So far: Any
>>> comments welcome!
>>>
>>> Best regards,
>>> Florian
>>>
>>> Florian Bezdeka (2):
>>>    wic-img: Forward warnings from wic to bitbake
>>>    wic: Warn if an ext filesystem affected by the Y2038 problem is
>>> used
>>>
>>>   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
>>>   scripts/lib/wic/partition.py | 38
>>> ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
>>> insertions(+), 5 deletions(-) 
>> Applied to next, thanks.
>>
> 


-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[florian.bezdeka]

On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:
> On 11.02.21 09:23, Henning Schild wrote:
> > Hi all,
> > 
> > i never got around to reviewing this. But did we just fork wic? These
> > patches need to go into wic and we later backport them once they are
> > accepted upstream.
> > 
> > Maybe they are already ... did not check.
> > 
> > When it comes to changing bitbake or wic, we should really not ... We
> > have forks of some files, like the wic plugins and bitbake config,
> > those are fine but should also stay very close to upstream.
> > 
> > The recently applied patch from Vijai also violates that. Since the
> > fork of the plugins was not updated with the wic bump and the repair
> > just takes a few bits of what we probably should take.
> > 
> 
> If you are referring to
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&data=04%7C01%7Cflorian.bezdeka%40siemens.com%7Ca5e6b57fc2f34070817c08d8ce6d6dbd%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486316681424173%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xERzeBGXiuxTVEC2n4CSuGmlFB9O7h07Hm9ODn33llg%3D&reserved=0:
> That one was "only" patching an isar version, though I agree that we
> should make sure to realign it with the original plugins if we are now
> imbalanced.
> 
> This one here is more critical as it changed a formerly vanilla wic
> file. That should be fixed quickly.
> 
> Florian, maybe you can propose a similar change to OE upstream? In the
> meantime, is there a chance to move the changes out of partition.py, to
> a file that is isar-specific?
> 

I guess the "RFC" tag of this series has been overlooked. It was not
intended for merging (yet). Part one (forwarding wic warnings to
bitbake) is a pure ISAR change and could be taken as is (if no further
comments come up).

Sorry for the long description of the series, but if you read closely I
already mentioned that the second part should go to OE. I sent it out
for feedback collection only.

The upstreaming to OE will take some time due to internal
clarifications. I never contributed to OE before, so some kind of
approval process has to be followed first.

At first glance there was no easy way moving the warnings from wic to
ISAR. We would have to re-parse the wic template file again and check
all the partitions afterwards. wic has all the necessary information at
hand so I guess that's way easier.


> Jan
> 
> > Henning
> > 
> > Am Thu, 11 Feb 2021 11:07:52 +0300
> > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > 
> > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:
> > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > 
> > > > Hi ISAR developers,
> > > > 
> > > > this series is the summary of a nice journey through the file system
> > > > jungle regarding Y2038 problem. It all began with a warning which is
> > > > reported by kernels >= 5.4:
> > > > 
> > > > ext4 filesystem being mounted at (mountpoint) supports timestamps
> > > > until 2038 (0x7fffffff)
> > > > 
> > > > I guess that most ISAR layers are using the Debian kernels, so that
> > > > warning was not recognized yet or at least not very often.
> > > > 
> > > > When reading this warning I was surprised. Shouldn't a modern file
> > > > system like ext4 be Y2038-safe? As it turned out it depends on the
> > > > inode size if an ext4 file system is safe or not. So why was the
> > > > inode size not sufficient in my case?
> > > > 
> > > > The inode size is chosen during file system generation and depends
> > > > on the size of the file system that is going to be created. For
> > > > details let's have a look at `man mke2fs`:
> > > > 
> > > > -T usage-type[,...]
> > > >      Specify how the filesystem is going to be used, so that mke2fs
> > > > can choose optimal filesystem parameters for that use. The usage
> > > > types that are supported are defined in the configuration file
> > > >      /etc/mke2fs.conf. The user may specify one or more usage types
> > > >      using a comma separated list.
> > > > 
> > > >      If this option is is not specified, mke2fs will pick a single
> > > >      default usage type based on the size of the filesystem to be
> > > >      created. If the filesystem size is less than 3 megabytes,
> > > > mke2fs will use the filesystem type floppy. If the filesystem size
> > > > is greater than or equal to 3 but less than 512 megabytes, mke2fs(8)
> > > >      will use the filesystem type small.
> > > > 
> > > > The relevant parts from /etc/mke2fs.conf:
> > > > [fs_types]
> > > > ...
> > > >          small = {
> > > >                  blocksize = 1024
> > > >                  inode_size = 128
> > > >                  inode_ratio = 4096
> > > >          }
> > > > ...
> > > > 
> > > > So whenever you create an ext4 file system with less than 512MB in
> > > > size you will end up with 128 byte inodes and your file system is
> > > > not Y2038-safe.
> > > > 
> > > > The ISAR part:
> > > > ext4 may often be used in combination with the expand-on-first-boot
> > > > recipe / feature. So whenever creating a small partition (e.g.
> > > > inside a wic file) and extending it later may result in a Y2038
> > > > affected ext4 file system.
> > > > 
> > > > That is exactly what happened to me and I would like to make sure
> > > > that all other ISAR users are aware of this situation.
> > > > 
> > > > Valid workarounds found so far:
> > > >   - Tell wic that an partition will grow:
> > > >     Add `--mkfs-extraopts "-T ext4"` to your wic partition
> > > > definition
> > > >   - Set the inode size to 256 (for small ext4 partitions)
> > > >     Add `--mkfs-extraopts "-I 256"` to your wic partition definition
> > > > 
> > > > The upstream part:
> > > > None of the following patches has been sent to any upstream (OE)
> > > > mailing lists yet but hopefully that will happen soon. So far: Any
> > > > comments welcome!
> > > > 
> > > > Best regards,
> > > > Florian
> > > > 
> > > > Florian Bezdeka (2):
> > > >    wic-img: Forward warnings from wic to bitbake
> > > >    wic: Warn if an ext filesystem affected by the Y2038 problem is
> > > > used
> > > > 
> > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > >   scripts/lib/wic/partition.py | 38
> > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > insertions(+), 5 deletions(-) 
> > > Applied to next, thanks.
> > > 
> > 
> 
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Henning Schild]

Am Thu, 11 Feb 2021 10:57:31 +0100
schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
<florian.bezdeka@siemens.com>:

> On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:
> > On 11.02.21 09:23, Henning Schild wrote:  
> > > Hi all,
> > >
> > > i never got around to reviewing this. But did we just fork wic?
> > > These patches need to go into wic and we later backport them once
> > > they are accepted upstream.
> > >
> > > Maybe they are already ... did not check.
> > >
> > > When it comes to changing bitbake or wic, we should really not
> > > ... We have forks of some files, like the wic plugins and bitbake
> > > config, those are fine but should also stay very close to
> > > upstream.
> > >
> > > The recently applied patch from Vijai also violates that. Since
> > > the fork of the plugins was not updated with the wic bump and the
> > > repair just takes a few bits of what we probably should take.
> > >  
> >
> > If you are referring to
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Cd67820e7b5d841cf320f08d8ce7372f9%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486342521796327%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=YQM5jQg9YSx6f9FiuYaduPEccCnNspRle4ZH8ES0nH4%3D&amp;reserved=0:
> > That one was "only" patching an isar version, though I agree that we
> > should make sure to realign it with the original plugins if we are
> > now imbalanced.
> >
> > This one here is more critical as it changed a formerly vanilla wic
> > file. That should be fixed quickly.
> >
> > Florian, maybe you can propose a similar change to OE upstream? In
> > the meantime, is there a chance to move the changes out of
> > partition.py, to a file that is isar-specific?
> >  
> 
> I guess the "RFC" tag of this series has been overlooked. It was not
> intended for merging (yet). Part one (forwarding wic warnings to
> bitbake) is a pure ISAR change and could be taken as is (if no further
> comments come up).

I guess that calls for a revert. And for more attention on the
maintainers side.

> Sorry for the long description of the series, but if you read closely
> I already mentioned that the second part should go to OE. I sent it
> out for feedback collection only.
> 
> The upstreaming to OE will take some time due to internal
> clarifications. I never contributed to OE before, so some kind of
> approval process has to be followed first.
> 
> At first glance there was no easy way moving the warnings from wic to
> ISAR. We would have to re-parse the wic template file again and check
> all the partitions afterwards. wic has all the necessary information
> at hand so I guess that's way easier.

I guess it can be moved into a task after wic. Here one would need to
parse the partition table, which kind of sucks. "losetup" or "kpartx"
might help but will not work in kas-container setups because they need
root.
We once had patches allowing wic to retain all partition images instead
of throwing them away after disk assembly. Having a switch for wic to
say ... do those partitions ... later do the disk would be generic,
allow hooking in this and other things.

Isar also has a class that creates ext4 images without, after which such
a check should also be done.

Is ext4 the only fs we care about? We have some layers doing ubifs,
squashfs and all sorts of funny things.

Maybe the kernel does warn "on device" so we could have a systemd unit
warning for all filesystems ... which would probably best find its
place in the kernel and or debian.

Henning

> 
> > Jan
> >  
> > > Henning
> > >
> > > Am Thu, 11 Feb 2021 11:07:52 +0300
> > > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > >  
> > > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:  
> > > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > >
> > > > > Hi ISAR developers,
> > > > >
> > > > > this series is the summary of a nice journey through the file
> > > > > system jungle regarding Y2038 problem. It all began with a
> > > > > warning which is reported by kernels >= 5.4:
> > > > >
> > > > > ext4 filesystem being mounted at (mountpoint) supports
> > > > > timestamps until 2038 (0x7fffffff)
> > > > >
> > > > > I guess that most ISAR layers are using the Debian kernels,
> > > > > so that warning was not recognized yet or at least not very
> > > > > often.
> > > > >
> > > > > When reading this warning I was surprised. Shouldn't a modern
> > > > > file system like ext4 be Y2038-safe? As it turned out it
> > > > > depends on the inode size if an ext4 file system is safe or
> > > > > not. So why was the inode size not sufficient in my case?
> > > > >
> > > > > The inode size is chosen during file system generation and
> > > > > depends on the size of the file system that is going to be
> > > > > created. For details let's have a look at `man mke2fs`:
> > > > >
> > > > > -T usage-type[,...]
> > > > >      Specify how the filesystem is going to be used, so that
> > > > > mke2fs can choose optimal filesystem parameters for that use.
> > > > > The usage types that are supported are defined in the
> > > > > configuration file /etc/mke2fs.conf. The user may specify one
> > > > > or more usage types using a comma separated list.
> > > > >
> > > > >      If this option is is not specified, mke2fs will pick a
> > > > > single default usage type based on the size of the filesystem
> > > > > to be created. If the filesystem size is less than 3
> > > > > megabytes, mke2fs will use the filesystem type floppy. If the
> > > > > filesystem size is greater than or equal to 3 but less than
> > > > > 512 megabytes, mke2fs(8) will use the filesystem type small.
> > > > >
> > > > > The relevant parts from /etc/mke2fs.conf:
> > > > > [fs_types]
> > > > > ...
> > > > >          small = {
> > > > >                  blocksize = 1024
> > > > >                  inode_size = 128
> > > > >                  inode_ratio = 4096
> > > > >          }
> > > > > ...
> > > > >
> > > > > So whenever you create an ext4 file system with less than
> > > > > 512MB in size you will end up with 128 byte inodes and your
> > > > > file system is not Y2038-safe.
> > > > >
> > > > > The ISAR part:
> > > > > ext4 may often be used in combination with the
> > > > > expand-on-first-boot recipe / feature. So whenever creating a
> > > > > small partition (e.g. inside a wic file) and extending it
> > > > > later may result in a Y2038 affected ext4 file system.
> > > > >
> > > > > That is exactly what happened to me and I would like to make
> > > > > sure that all other ISAR users are aware of this situation.
> > > > >
> > > > > Valid workarounds found so far:
> > > > >   - Tell wic that an partition will grow:
> > > > >     Add `--mkfs-extraopts "-T ext4"` to your wic partition
> > > > > definition
> > > > >   - Set the inode size to 256 (for small ext4 partitions)
> > > > >     Add `--mkfs-extraopts "-I 256"` to your wic partition
> > > > > definition
> > > > >
> > > > > The upstream part:
> > > > > None of the following patches has been sent to any upstream
> > > > > (OE) mailing lists yet but hopefully that will happen soon.
> > > > > So far: Any comments welcome!
> > > > >
> > > > > Best regards,
> > > > > Florian
> > > > >
> > > > > Florian Bezdeka (2):
> > > > >    wic-img: Forward warnings from wic to bitbake
> > > > >    wic: Warn if an ext filesystem affected by the Y2038
> > > > > problem is used
> > > > >
> > > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > > >   scripts/lib/wic/partition.py | 38
> > > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > > insertions(+), 5 deletions(-)  
> > > > Applied to next, thanks.
> > > >  
> > >  
> >
> >  
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[florian.bezdeka]

On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:
> Am Thu, 11 Feb 2021 10:57:31 +0100
> schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> <florian.bezdeka@siemens.com>:
> 
> > On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:
> > > On 11.02.21 09:23, Henning Schild wrote:  
> > > > Hi all,
> > > > 
> > > > i never got around to reviewing this. But did we just fork wic?
> > > > These patches need to go into wic and we later backport them once
> > > > they are accepted upstream.
> > > > 
> > > > Maybe they are already ... did not check.
> > > > 
> > > > When it comes to changing bitbake or wic, we should really not
> > > > ... We have forks of some files, like the wic plugins and bitbake
> > > > config, those are fine but should also stay very close to
> > > > upstream.
> > > > 
> > > > The recently applied patch from Vijai also violates that. Since
> > > > the fork of the plugins was not updated with the wic bump and the
> > > > repair just takes a few bits of what we probably should take.
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > If you are referring to
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C48d6471d1d4341e4445d08d8ce778b07%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486360122035313%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=61t42JDRuSWYJF%2Ff6rE6A7A9o0%2BlDF7zKwN85LVo%2BiU%3D&amp;reserved=0:
> > > That one was "only" patching an isar version, though I agree that we
> > > should make sure to realign it with the original plugins if we are
> > > now imbalanced.
> > > 
> > > This one here is more critical as it changed a formerly vanilla wic
> > > file. That should be fixed quickly.
> > > 
> > > Florian, maybe you can propose a similar change to OE upstream? In
> > > the meantime, is there a chance to move the changes out of
> > > partition.py, to a file that is isar-specific?
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
> > I guess the "RFC" tag of this series has been overlooked. It was not
> > intended for merging (yet). Part one (forwarding wic warnings to
> > bitbake) is a pure ISAR change and could be taken as is (if no further
> > comments come up).
> 
> I guess that calls for a revert. And for more attention on the
> maintainers side.
> 
> > Sorry for the long description of the series, but if you read closely
> > I already mentioned that the second part should go to OE. I sent it
> > out for feedback collection only.
> > 
> > The upstreaming to OE will take some time due to internal
> > clarifications. I never contributed to OE before, so some kind of
> > approval process has to be followed first.
> > 
> > At first glance there was no easy way moving the warnings from wic to
> > ISAR. We would have to re-parse the wic template file again and check
> > all the partitions afterwards. wic has all the necessary information
> > at hand so I guess that's way easier.
> 
> I guess it can be moved into a task after wic. Here one would need to
> parse the partition table, which kind of sucks. "losetup" or "kpartx"
> might help but will not work in kas-container setups because they need
> root.
> We once had patches allowing wic to retain all partition images instead
> of throwing them away after disk assembly. Having a switch for wic to
> say ... do those partitions ... later do the disk would be generic,
> allow hooking in this and other things.
> 
> Isar also has a class that creates ext4 images without, after which such
> a check should also be done.

Yes. But instead of spreading the warnings around it would be nice to
have a single place where we could do the Y2038 checks. So maybe it
should be a base feature of "image.bbclass"? Or ext4-img.bbclass should
call wic instead of the mke2fs utilities directly?

BTW: The name ext4-img.bbclass is kind of misleading. You could simply
create ext{2,3} file systems by setting MKE2FS_ARGS to something like
"-t ext2".

> 
> Is ext4 the only fs we care about? We have some layers doing ubifs,
> squashfs and all sorts of funny things.

Up to now I cared about the filesystems supported by wic. So
ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106 (u32)
and btrfs will "never" overflow (u64). 

ubifs is similar to btrfs, so not affected by Y2038.

> 
> Maybe the kernel does warn "on device" so we could have a systemd unit
> warning for all filesystems ... which would probably best find its
> place in the kernel and or debian.

At least for affected ext file systems the kernel will warn (on mount).
But I considered that as "too late".

> 
> Henning
> 
> > 
> > > Jan
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > > Henning
> > > > 
> > > > Am Thu, 11 Feb 2021 11:07:52 +0300
> > > > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:  
> > > > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > > > 
> > > > > > Hi ISAR developers,
> > > > > > 
> > > > > > this series is the summary of a nice journey through the file
> > > > > > system jungle regarding Y2038 problem. It all began with a
> > > > > > warning which is reported by kernels >= 5.4:
> > > > > > 
> > > > > > ext4 filesystem being mounted at (mountpoint) supports
> > > > > > timestamps until 2038 (0x7fffffff)
> > > > > > 
> > > > > > I guess that most ISAR layers are using the Debian kernels,
> > > > > > so that warning was not recognized yet or at least not very
> > > > > > often.
> > > > > > 
> > > > > > When reading this warning I was surprised. Shouldn't a modern
> > > > > > file system like ext4 be Y2038-safe? As it turned out it
> > > > > > depends on the inode size if an ext4 file system is safe or
> > > > > > not. So why was the inode size not sufficient in my case?
> > > > > > 
> > > > > > The inode size is chosen during file system generation and
> > > > > > depends on the size of the file system that is going to be
> > > > > > created. For details let's have a look at `man mke2fs`:
> > > > > > 
> > > > > > -T usage-type[,...]
> > > > > >      Specify how the filesystem is going to be used, so that
> > > > > > mke2fs can choose optimal filesystem parameters for that use.
> > > > > > The usage types that are supported are defined in the
> > > > > > configuration file /etc/mke2fs.conf. The user may specify one
> > > > > > or more usage types using a comma separated list.
> > > > > > 
> > > > > >      If this option is is not specified, mke2fs will pick a
> > > > > > single default usage type based on the size of the filesystem
> > > > > > to be created. If the filesystem size is less than 3
> > > > > > megabytes, mke2fs will use the filesystem type floppy. If the
> > > > > > filesystem size is greater than or equal to 3 but less than
> > > > > > 512 megabytes, mke2fs(8) will use the filesystem type small.
> > > > > > 
> > > > > > The relevant parts from /etc/mke2fs.conf:
> > > > > > [fs_types]
> > > > > > ...
> > > > > >          small = {
> > > > > >                  blocksize = 1024
> > > > > >                  inode_size = 128
> > > > > >                  inode_ratio = 4096
> > > > > >          }
> > > > > > ...
> > > > > > 
> > > > > > So whenever you create an ext4 file system with less than
> > > > > > 512MB in size you will end up with 128 byte inodes and your
> > > > > > file system is not Y2038-safe.
> > > > > > 
> > > > > > The ISAR part:
> > > > > > ext4 may often be used in combination with the
> > > > > > expand-on-first-boot recipe / feature. So whenever creating a
> > > > > > small partition (e.g. inside a wic file) and extending it
> > > > > > later may result in a Y2038 affected ext4 file system.
> > > > > > 
> > > > > > That is exactly what happened to me and I would like to make
> > > > > > sure that all other ISAR users are aware of this situation.
> > > > > > 
> > > > > > Valid workarounds found so far:
> > > > > >   - Tell wic that an partition will grow:
> > > > > >     Add `--mkfs-extraopts "-T ext4"` to your wic partition
> > > > > > definition
> > > > > >   - Set the inode size to 256 (for small ext4 partitions)
> > > > > >     Add `--mkfs-extraopts "-I 256"` to your wic partition
> > > > > > definition
> > > > > > 
> > > > > > The upstream part:
> > > > > > None of the following patches has been sent to any upstream
> > > > > > (OE) mailing lists yet but hopefully that will happen soon.
> > > > > > So far: Any comments welcome!
> > > > > > 
> > > > > > Best regards,
> > > > > > Florian
> > > > > > 
> > > > > > Florian Bezdeka (2):
> > > > > >    wic-img: Forward warnings from wic to bitbake
> > > > > >    wic: Warn if an ext filesystem affected by the Y2038
> > > > > > problem is used
> > > > > > 
> > > > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > > > >   scripts/lib/wic/partition.py | 38
> > > > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > > > insertions(+), 5 deletions(-)  
> > > > > Applied to next, thanks.
> > > > >  
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[florian.bezdeka]

On Thu, 2021-02-11 at 12:47 +0000, [ext] florian.bezdeka@siemens.com
wrote:
> On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:
> > Am Thu, 11 Feb 2021 10:57:31 +0100
> > schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> > <florian.bezdeka@siemens.com>:
> > 
> > > On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:
> > > > On 11.02.21 09:23, Henning Schild wrote:  
> > > > > Hi all,
> > > > > 
> > > > > i never got around to reviewing this. But did we just fork wic?
> > > > > These patches need to go into wic and we later backport them once
> > > > > they are accepted upstream.
> > > > > 
> > > > > Maybe they are already ... did not check.
> > > > > 
> > > > > When it comes to changing bitbake or wic, we should really not
> > > > > ... We have forks of some files, like the wic plugins and bitbake
> > > > > config, those are fine but should also stay very close to
> > > > > upstream.
> > > > > 
> > > > > The recently applied patch from Vijai also violates that. Since
> > > > > the fork of the plugins was not updated with the wic bump and the
> > > > > repair just takes a few bits of what we probably should take.
> > > > >  
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > If you are referring to
> > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C070614d51b4b45045bdf08d8ce8b657f%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486445390373228%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3H1fH4EhWZMF%2BgUYpPcej9py24HSVAgIjwOwiELlOqs%3D&amp;reserved=0:
> > > > That one was "only" patching an isar version, though I agree that we
> > > > should make sure to realign it with the original plugins if we are
> > > > now imbalanced.
> > > > 
> > > > This one here is more critical as it changed a formerly vanilla wic
> > > > file. That should be fixed quickly.
> > > > 
> > > > Florian, maybe you can propose a similar change to OE upstream? In
> > > > the meantime, is there a chance to move the changes out of
> > > > partition.py, to a file that is isar-specific?
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > I guess the "RFC" tag of this series has been overlooked. It was not
> > > intended for merging (yet). Part one (forwarding wic warnings to
> > > bitbake) is a pure ISAR change and could be taken as is (if no further
> > > comments come up).
> > 
> > I guess that calls for a revert. And for more attention on the
> > maintainers side.
> > 
> > > Sorry for the long description of the series, but if you read closely
> > > I already mentioned that the second part should go to OE. I sent it
> > > out for feedback collection only.
> > > 
> > > The upstreaming to OE will take some time due to internal
> > > clarifications. I never contributed to OE before, so some kind of
> > > approval process has to be followed first.
> > > 
> > > At first glance there was no easy way moving the warnings from wic to
> > > ISAR. We would have to re-parse the wic template file again and check
> > > all the partitions afterwards. wic has all the necessary information
> > > at hand so I guess that's way easier.
> > 
> > I guess it can be moved into a task after wic. Here one would need to
> > parse the partition table, which kind of sucks. "losetup" or "kpartx"
> > might help but will not work in kas-container setups because they need
> > root.
> > We once had patches allowing wic to retain all partition images instead
> > of throwing them away after disk assembly. Having a switch for wic to
> > say ... do those partitions ... later do the disk would be generic,
> > allow hooking in this and other things.
> > 
> > Isar also has a class that creates ext4 images without, after which such
> > a check should also be done.
> 
> Yes. But instead of spreading the warnings around it would be nice to
> have a single place where we could do the Y2038 checks. So maybe it
> should be a base feature of "image.bbclass"? Or ext4-img.bbclass should
> call wic instead of the mke2fs utilities directly?
> 
> BTW: The name ext4-img.bbclass is kind of misleading. You could simply
> create ext{2,3} file systems by setting MKE2FS_ARGS to something like
> "-t ext2".
> 
> > 
> > Is ext4 the only fs we care about? We have some layers doing ubifs,
> > squashfs and all sorts of funny things.
> 
> Up to now I cared about the filesystems supported by wic. So
> ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106 (u32)
> and btrfs will "never" overflow (u64). 
> 
> ubifs is similar to btrfs, so not affected by Y2038.
> 
> > 
> > Maybe the kernel does warn "on device" so we could have a systemd unit
> > warning for all filesystems ... which would probably best find its
> > place in the kernel and or debian.
> 
> At least for affected ext file systems the kernel will warn (on mount).
> But I considered that as "too late".

To be more specific: Linux >= 5.4 warns. That's why I guess that many
projects did not realize that they are already affected by the Y2038
problem because of older kernel versions.

> 
> > 
> > Henning
> > 
> > > 
> > > > Jan
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > > Henning
> > > > > 
> > > > > Am Thu, 11 Feb 2021 11:07:52 +0300
> > > > > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > > > >  
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:  
> > > > > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > > > > 
> > > > > > > Hi ISAR developers,
> > > > > > > 
> > > > > > > this series is the summary of a nice journey through the file
> > > > > > > system jungle regarding Y2038 problem. It all began with a
> > > > > > > warning which is reported by kernels >= 5.4:
> > > > > > > 
> > > > > > > ext4 filesystem being mounted at (mountpoint) supports
> > > > > > > timestamps until 2038 (0x7fffffff)
> > > > > > > 
> > > > > > > I guess that most ISAR layers are using the Debian kernels,
> > > > > > > so that warning was not recognized yet or at least not very
> > > > > > > often.
> > > > > > > 
> > > > > > > When reading this warning I was surprised. Shouldn't a modern
> > > > > > > file system like ext4 be Y2038-safe? As it turned out it
> > > > > > > depends on the inode size if an ext4 file system is safe or
> > > > > > > not. So why was the inode size not sufficient in my case?
> > > > > > > 
> > > > > > > The inode size is chosen during file system generation and
> > > > > > > depends on the size of the file system that is going to be
> > > > > > > created. For details let's have a look at `man mke2fs`:
> > > > > > > 
> > > > > > > -T usage-type[,...]
> > > > > > >      Specify how the filesystem is going to be used, so that
> > > > > > > mke2fs can choose optimal filesystem parameters for that use.
> > > > > > > The usage types that are supported are defined in the
> > > > > > > configuration file /etc/mke2fs.conf. The user may specify one
> > > > > > > or more usage types using a comma separated list.
> > > > > > > 
> > > > > > >      If this option is is not specified, mke2fs will pick a
> > > > > > > single default usage type based on the size of the filesystem
> > > > > > > to be created. If the filesystem size is less than 3
> > > > > > > megabytes, mke2fs will use the filesystem type floppy. If the
> > > > > > > filesystem size is greater than or equal to 3 but less than
> > > > > > > 512 megabytes, mke2fs(8) will use the filesystem type small.
> > > > > > > 
> > > > > > > The relevant parts from /etc/mke2fs.conf:
> > > > > > > [fs_types]
> > > > > > > ...
> > > > > > >          small = {
> > > > > > >                  blocksize = 1024
> > > > > > >                  inode_size = 128
> > > > > > >                  inode_ratio = 4096
> > > > > > >          }
> > > > > > > ...
> > > > > > > 
> > > > > > > So whenever you create an ext4 file system with less than
> > > > > > > 512MB in size you will end up with 128 byte inodes and your
> > > > > > > file system is not Y2038-safe.
> > > > > > > 
> > > > > > > The ISAR part:
> > > > > > > ext4 may often be used in combination with the
> > > > > > > expand-on-first-boot recipe / feature. So whenever creating a
> > > > > > > small partition (e.g. inside a wic file) and extending it
> > > > > > > later may result in a Y2038 affected ext4 file system.
> > > > > > > 
> > > > > > > That is exactly what happened to me and I would like to make
> > > > > > > sure that all other ISAR users are aware of this situation.
> > > > > > > 
> > > > > > > Valid workarounds found so far:
> > > > > > >   - Tell wic that an partition will grow:
> > > > > > >     Add `--mkfs-extraopts "-T ext4"` to your wic partition
> > > > > > > definition
> > > > > > >   - Set the inode size to 256 (for small ext4 partitions)
> > > > > > >     Add `--mkfs-extraopts "-I 256"` to your wic partition
> > > > > > > definition
> > > > > > > 
> > > > > > > The upstream part:
> > > > > > > None of the following patches has been sent to any upstream
> > > > > > > (OE) mailing lists yet but hopefully that will happen soon.
> > > > > > > So far: Any comments welcome!
> > > > > > > 
> > > > > > > Best regards,
> > > > > > > Florian
> > > > > > > 
> > > > > > > Florian Bezdeka (2):
> > > > > > >    wic-img: Forward warnings from wic to bitbake
> > > > > > >    wic: Warn if an ext filesystem affected by the Y2038
> > > > > > > problem is used
> > > > > > > 
> > > > > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > > > > >   scripts/lib/wic/partition.py | 38
> > > > > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > > > > insertions(+), 5 deletions(-)  
> > > > > > Applied to next, thanks.
> > > > > >  
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > >  
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > 
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Henning Schild]

Am Thu, 11 Feb 2021 14:31:58 +0100
schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
<florian.bezdeka@siemens.com>:

> On Thu, 2021-02-11 at 12:47 +0000, [ext] florian.bezdeka@siemens.com
> wrote:
> > On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:  
> > > Am Thu, 11 Feb 2021 10:57:31 +0100
> > > schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> > > <florian.bezdeka@siemens.com>:
> > >  
> > > > On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:  
> > > > > On 11.02.21 09:23, Henning Schild wrote:  
> > > > > > Hi all,
> > > > > >
> > > > > > i never got around to reviewing this. But did we just fork
> > > > > > wic? These patches need to go into wic and we later
> > > > > > backport them once they are accepted upstream.
> > > > > >
> > > > > > Maybe they are already ... did not check.
> > > > > >
> > > > > > When it comes to changing bitbake or wic, we should really
> > > > > > not ... We have forks of some files, like the wic plugins
> > > > > > and bitbake config, those are fine but should also stay
> > > > > > very close to upstream.
> > > > > >
> > > > > > The recently applied patch from Vijai also violates that.
> > > > > > Since the fork of the plugins was not updated with the wic
> > > > > > bump and the repair just takes a few bits of what we
> > > > > > probably should take.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >  
> > > > >
> > > > > If you are referring to
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Ca81479e099ce4a32a67608d8ce916870%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486471194236324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=QvpGJS2QMLxBNb0EQZVilcyZr3CBN%2FZ48rSYlOUVisU%3D&amp;reserved=0:
> > > > > That one was "only" patching an isar version, though I agree
> > > > > that we should make sure to realign it with the original
> > > > > plugins if we are now imbalanced.
> > > > >
> > > > > This one here is more critical as it changed a formerly
> > > > > vanilla wic file. That should be fixed quickly.
> > > > >
> > > > > Florian, maybe you can propose a similar change to OE
> > > > > upstream? In the meantime, is there a chance to move the
> > > > > changes out of partition.py, to a file that is isar-specific?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  
> > > >
> > > > I guess the "RFC" tag of this series has been overlooked. It
> > > > was not intended for merging (yet). Part one (forwarding wic
> > > > warnings to bitbake) is a pure ISAR change and could be taken
> > > > as is (if no further comments come up).  
> > >
> > > I guess that calls for a revert. And for more attention on the
> > > maintainers side.
> > >  
> > > > Sorry for the long description of the series, but if you read
> > > > closely I already mentioned that the second part should go to
> > > > OE. I sent it out for feedback collection only.
> > > >
> > > > The upstreaming to OE will take some time due to internal
> > > > clarifications. I never contributed to OE before, so some kind
> > > > of approval process has to be followed first.
> > > >
> > > > At first glance there was no easy way moving the warnings from
> > > > wic to ISAR. We would have to re-parse the wic template file
> > > > again and check all the partitions afterwards. wic has all the
> > > > necessary information at hand so I guess that's way easier.  
> > >
> > > I guess it can be moved into a task after wic. Here one would
> > > need to parse the partition table, which kind of sucks. "losetup"
> > > or "kpartx" might help but will not work in kas-container setups
> > > because they need root.
> > > We once had patches allowing wic to retain all partition images
> > > instead of throwing them away after disk assembly. Having a
> > > switch for wic to say ... do those partitions ... later do the
> > > disk would be generic, allow hooking in this and other things.
> > >
> > > Isar also has a class that creates ext4 images without, after
> > > which such a check should also be done.  
> >
> > Yes. But instead of spreading the warnings around it would be nice
> > to have a single place where we could do the Y2038 checks. So maybe
> > it should be a base feature of "image.bbclass"? Or ext4-img.bbclass
> > should call wic instead of the mke2fs utilities directly?
> >
> > BTW: The name ext4-img.bbclass is kind of misleading. You could
> > simply create ext{2,3} file systems by setting MKE2FS_ARGS to
> > something like "-t ext2".
> >  
> > >
> > > Is ext4 the only fs we care about? We have some layers doing
> > > ubifs, squashfs and all sorts of funny things.  
> >
> > Up to now I cared about the filesystems supported by wic. So
> > ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106 (u32)
> > and btrfs will "never" overflow (u64).
> >
> > ubifs is similar to btrfs, so not affected by Y2038.
> >  
> > >
> > > Maybe the kernel does warn "on device" so we could have a systemd
> > > unit warning for all filesystems ... which would probably best
> > > find its place in the kernel and or debian.  
> >
> > At least for affected ext file systems the kernel will warn (on
> > mount). But I considered that as "too late".  
> 
> To be more specific: Linux >= 5.4 warns. That's why I guess that many
> projects did not realize that they are already affected by the Y2038
> problem because of older kernel versions.

Which sounds like that warning needs backporting into the debian10
kernel and maybe cip. Not sure Isar is the best place, but a valid one
that could help.

Maybe mkfs could warn ... as well.

Henning

> >  
> > >
> > > Henning
> > >  
> > > >  
> > > > > Jan
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  
> > > > > > Henning
> > > > > >
> > > > > > Am Thu, 11 Feb 2021 11:07:52 +0300
> > > > > > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >  
> > > > > > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:  
> > > > > > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > > > > >
> > > > > > > > Hi ISAR developers,
> > > > > > > >
> > > > > > > > this series is the summary of a nice journey through
> > > > > > > > the file system jungle regarding Y2038 problem. It all
> > > > > > > > began with a warning which is reported by kernels >=
> > > > > > > > 5.4:
> > > > > > > >
> > > > > > > > ext4 filesystem being mounted at (mountpoint) supports
> > > > > > > > timestamps until 2038 (0x7fffffff)
> > > > > > > >
> > > > > > > > I guess that most ISAR layers are using the Debian
> > > > > > > > kernels, so that warning was not recognized yet or at
> > > > > > > > least not very often.
> > > > > > > >
> > > > > > > > When reading this warning I was surprised. Shouldn't a
> > > > > > > > modern file system like ext4 be Y2038-safe? As it
> > > > > > > > turned out it depends on the inode size if an ext4 file
> > > > > > > > system is safe or not. So why was the inode size not
> > > > > > > > sufficient in my case?
> > > > > > > >
> > > > > > > > The inode size is chosen during file system generation
> > > > > > > > and depends on the size of the file system that is
> > > > > > > > going to be created. For details let's have a look at
> > > > > > > > `man mke2fs`:
> > > > > > > >
> > > > > > > > -T usage-type[,...]
> > > > > > > >      Specify how the filesystem is going to be used, so
> > > > > > > > that mke2fs can choose optimal filesystem parameters
> > > > > > > > for that use. The usage types that are supported are
> > > > > > > > defined in the configuration file /etc/mke2fs.conf. The
> > > > > > > > user may specify one or more usage types using a comma
> > > > > > > > separated list.
> > > > > > > >
> > > > > > > >      If this option is is not specified, mke2fs will
> > > > > > > > pick a single default usage type based on the size of
> > > > > > > > the filesystem to be created. If the filesystem size is
> > > > > > > > less than 3 megabytes, mke2fs will use the filesystem
> > > > > > > > type floppy. If the filesystem size is greater than or
> > > > > > > > equal to 3 but less than 512 megabytes, mke2fs(8) will
> > > > > > > > use the filesystem type small.
> > > > > > > >
> > > > > > > > The relevant parts from /etc/mke2fs.conf:
> > > > > > > > [fs_types]
> > > > > > > > ...
> > > > > > > >          small = {
> > > > > > > >                  blocksize = 1024
> > > > > > > >                  inode_size = 128
> > > > > > > >                  inode_ratio = 4096
> > > > > > > >          }
> > > > > > > > ...
> > > > > > > >
> > > > > > > > So whenever you create an ext4 file system with less
> > > > > > > > than 512MB in size you will end up with 128 byte inodes
> > > > > > > > and your file system is not Y2038-safe.
> > > > > > > >
> > > > > > > > The ISAR part:
> > > > > > > > ext4 may often be used in combination with the
> > > > > > > > expand-on-first-boot recipe / feature. So whenever
> > > > > > > > creating a small partition (e.g. inside a wic file) and
> > > > > > > > extending it later may result in a Y2038 affected ext4
> > > > > > > > file system.
> > > > > > > >
> > > > > > > > That is exactly what happened to me and I would like to
> > > > > > > > make sure that all other ISAR users are aware of this
> > > > > > > > situation.
> > > > > > > >
> > > > > > > > Valid workarounds found so far:
> > > > > > > >   - Tell wic that an partition will grow:
> > > > > > > >     Add `--mkfs-extraopts "-T ext4"` to your wic
> > > > > > > > partition definition
> > > > > > > >   - Set the inode size to 256 (for small ext4
> > > > > > > > partitions) Add `--mkfs-extraopts "-I 256"` to your wic
> > > > > > > > partition definition
> > > > > > > >
> > > > > > > > The upstream part:
> > > > > > > > None of the following patches has been sent to any
> > > > > > > > upstream (OE) mailing lists yet but hopefully that will
> > > > > > > > happen soon. So far: Any comments welcome!
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Florian
> > > > > > > >
> > > > > > > > Florian Bezdeka (2):
> > > > > > > >    wic-img: Forward warnings from wic to bitbake
> > > > > > > >    wic: Warn if an ext filesystem affected by the Y2038
> > > > > > > > problem is used
> > > > > > > >
> > > > > > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > > > > > >   scripts/lib/wic/partition.py | 38
> > > > > > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > > > > > insertions(+), 5 deletions(-)  
> > > > > > > Applied to next, thanks.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >  
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >  
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  
> > > >  
> > >  
> >  
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Jan Kiszka]

On 11.02.21 15:13, Henning Schild wrote:
> Am Thu, 11 Feb 2021 14:31:58 +0100
> schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> <florian.bezdeka@siemens.com>:
> 
>> On Thu, 2021-02-11 at 12:47 +0000, [ext] florian.bezdeka@siemens.com
>> wrote:
>>> On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:  
>>>> Am Thu, 11 Feb 2021 10:57:31 +0100
>>>> schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
>>>> <florian.bezdeka@siemens.com>:
>>>>  
>>>>> On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:  
>>>>>> On 11.02.21 09:23, Henning Schild wrote:  
>>>>>>> Hi all,
>>>>>>>
>>>>>>> i never got around to reviewing this. But did we just fork
>>>>>>> wic? These patches need to go into wic and we later
>>>>>>> backport them once they are accepted upstream.
>>>>>>>
>>>>>>> Maybe they are already ... did not check.
>>>>>>>
>>>>>>> When it comes to changing bitbake or wic, we should really
>>>>>>> not ... We have forks of some files, like the wic plugins
>>>>>>> and bitbake config, those are fine but should also stay
>>>>>>> very close to upstream.
>>>>>>>
>>>>>>> The recently applied patch from Vijai also violates that.
>>>>>>> Since the fork of the plugins was not updated with the wic
>>>>>>> bump and the repair just takes a few bits of what we
>>>>>>> probably should take.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  
>>>>>>
>>>>>> If you are referring to
>>>>>> https://groups.google.com/d/msgid/isar-users/20201126091750.28048-1-Vijaikumar_Kanagarajan%40mentor.com
>>>>>> That one was "only" patching an isar version, though I agree
>>>>>> that we should make sure to realign it with the original
>>>>>> plugins if we are now imbalanced.
>>>>>>
>>>>>> This one here is more critical as it changed a formerly
>>>>>> vanilla wic file. That should be fixed quickly.
>>>>>>
>>>>>> Florian, maybe you can propose a similar change to OE
>>>>>> upstream? In the meantime, is there a chance to move the
>>>>>> changes out of partition.py, to a file that is isar-specific?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  
>>>>>
>>>>> I guess the "RFC" tag of this series has been overlooked. It
>>>>> was not intended for merging (yet). Part one (forwarding wic
>>>>> warnings to bitbake) is a pure ISAR change and could be taken
>>>>> as is (if no further comments come up).  
>>>>
>>>> I guess that calls for a revert. And for more attention on the
>>>> maintainers side.
>>>>  
>>>>> Sorry for the long description of the series, but if you read
>>>>> closely I already mentioned that the second part should go to
>>>>> OE. I sent it out for feedback collection only.
>>>>>
>>>>> The upstreaming to OE will take some time due to internal
>>>>> clarifications. I never contributed to OE before, so some kind
>>>>> of approval process has to be followed first.
>>>>>
>>>>> At first glance there was no easy way moving the warnings from
>>>>> wic to ISAR. We would have to re-parse the wic template file
>>>>> again and check all the partitions afterwards. wic has all the
>>>>> necessary information at hand so I guess that's way easier.  
>>>>
>>>> I guess it can be moved into a task after wic. Here one would
>>>> need to parse the partition table, which kind of sucks. "losetup"
>>>> or "kpartx" might help but will not work in kas-container setups
>>>> because they need root.
>>>> We once had patches allowing wic to retain all partition images
>>>> instead of throwing them away after disk assembly. Having a
>>>> switch for wic to say ... do those partitions ... later do the
>>>> disk would be generic, allow hooking in this and other things.
>>>>
>>>> Isar also has a class that creates ext4 images without, after
>>>> which such a check should also be done.  
>>>
>>> Yes. But instead of spreading the warnings around it would be nice
>>> to have a single place where we could do the Y2038 checks. So maybe
>>> it should be a base feature of "image.bbclass"? Or ext4-img.bbclass
>>> should call wic instead of the mke2fs utilities directly?
>>>
>>> BTW: The name ext4-img.bbclass is kind of misleading. You could
>>> simply create ext{2,3} file systems by setting MKE2FS_ARGS to
>>> something like "-t ext2".
>>>  
>>>>
>>>> Is ext4 the only fs we care about? We have some layers doing
>>>> ubifs, squashfs and all sorts of funny things.  
>>>
>>> Up to now I cared about the filesystems supported by wic. So
>>> ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106 (u32)
>>> and btrfs will "never" overflow (u64).
>>>
>>> ubifs is similar to btrfs, so not affected by Y2038.
>>>  
>>>>
>>>> Maybe the kernel does warn "on device" so we could have a systemd
>>>> unit warning for all filesystems ... which would probably best
>>>> find its place in the kernel and or debian.  
>>>
>>> At least for affected ext file systems the kernel will warn (on
>>> mount). But I considered that as "too late".  
>>
>> To be more specific: Linux >= 5.4 warns. That's why I guess that many
>> projects did not realize that they are already affected by the Y2038
>> problem because of older kernel versions.
> 
> Which sounds like that warning needs backporting into the debian10
> kernel and maybe cip. Not sure Isar is the best place, but a valid one
> that could help.
> 
> Maybe mkfs could warn ... as well.
> 

Right, and we want such warnings seen at image *build time*, not only on
the target during runtime. That is the key idea behind this instrumentation.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Henning Schild]

Am Thu, 11 Feb 2021 18:57:24 +0100
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> On 11.02.21 15:13, Henning Schild wrote:
> > Am Thu, 11 Feb 2021 14:31:58 +0100
> > schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> > <florian.bezdeka@siemens.com>:
> >   
> >> On Thu, 2021-02-11 at 12:47 +0000, [ext]
> >> florian.bezdeka@siemens.com wrote:  
> >>> On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:    
> >>>> Am Thu, 11 Feb 2021 10:57:31 +0100
> >>>> schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> >>>> <florian.bezdeka@siemens.com>:
> >>>>    
> >>>>> On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:    
> >>>>>> On 11.02.21 09:23, Henning Schild wrote:    
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> i never got around to reviewing this. But did we just fork
> >>>>>>> wic? These patches need to go into wic and we later
> >>>>>>> backport them once they are accepted upstream.
> >>>>>>>
> >>>>>>> Maybe they are already ... did not check.
> >>>>>>>
> >>>>>>> When it comes to changing bitbake or wic, we should really
> >>>>>>> not ... We have forks of some files, like the wic plugins
> >>>>>>> and bitbake config, those are fine but should also stay
> >>>>>>> very close to upstream.
> >>>>>>>
> >>>>>>> The recently applied patch from Vijai also violates that.
> >>>>>>> Since the fork of the plugins was not updated with the wic
> >>>>>>> bump and the repair just takes a few bits of what we
> >>>>>>> probably should take.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>    
> >>>>>>
> >>>>>> If you are referring to
> >>>>>> https://groups.google.com/d/msgid/isar-users/20201126091750.28048-1-Vijaikumar_Kanagarajan%40mentor.com
> >>>>>> That one was "only" patching an isar version, though I agree
> >>>>>> that we should make sure to realign it with the original
> >>>>>> plugins if we are now imbalanced.
> >>>>>>
> >>>>>> This one here is more critical as it changed a formerly
> >>>>>> vanilla wic file. That should be fixed quickly.
> >>>>>>
> >>>>>> Florian, maybe you can propose a similar change to OE
> >>>>>> upstream? In the meantime, is there a chance to move the
> >>>>>> changes out of partition.py, to a file that is isar-specific?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>    
> >>>>>
> >>>>> I guess the "RFC" tag of this series has been overlooked. It
> >>>>> was not intended for merging (yet). Part one (forwarding wic
> >>>>> warnings to bitbake) is a pure ISAR change and could be taken
> >>>>> as is (if no further comments come up).    
> >>>>
> >>>> I guess that calls for a revert. And for more attention on the
> >>>> maintainers side.

Florian, maybe you send a revert series. Not your fault but maybe your
call.

Henning

> >>>>> Sorry for the long description of the series, but if you read
> >>>>> closely I already mentioned that the second part should go to
> >>>>> OE. I sent it out for feedback collection only.
> >>>>>
> >>>>> The upstreaming to OE will take some time due to internal
> >>>>> clarifications. I never contributed to OE before, so some kind
> >>>>> of approval process has to be followed first.
> >>>>>
> >>>>> At first glance there was no easy way moving the warnings from
> >>>>> wic to ISAR. We would have to re-parse the wic template file
> >>>>> again and check all the partitions afterwards. wic has all the
> >>>>> necessary information at hand so I guess that's way easier.    
> >>>>
> >>>> I guess it can be moved into a task after wic. Here one would
> >>>> need to parse the partition table, which kind of sucks. "losetup"
> >>>> or "kpartx" might help but will not work in kas-container setups
> >>>> because they need root.
> >>>> We once had patches allowing wic to retain all partition images
> >>>> instead of throwing them away after disk assembly. Having a
> >>>> switch for wic to say ... do those partitions ... later do the
> >>>> disk would be generic, allow hooking in this and other things.
> >>>>
> >>>> Isar also has a class that creates ext4 images without, after
> >>>> which such a check should also be done.    
> >>>
> >>> Yes. But instead of spreading the warnings around it would be nice
> >>> to have a single place where we could do the Y2038 checks. So
> >>> maybe it should be a base feature of "image.bbclass"? Or
> >>> ext4-img.bbclass should call wic instead of the mke2fs utilities
> >>> directly?
> >>>
> >>> BTW: The name ext4-img.bbclass is kind of misleading. You could
> >>> simply create ext{2,3} file systems by setting MKE2FS_ARGS to
> >>> something like "-t ext2".
> >>>    
> >>>>
> >>>> Is ext4 the only fs we care about? We have some layers doing
> >>>> ubifs, squashfs and all sorts of funny things.    
> >>>
> >>> Up to now I cared about the filesystems supported by wic. So
> >>> ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106
> >>> (u32) and btrfs will "never" overflow (u64).
> >>>
> >>> ubifs is similar to btrfs, so not affected by Y2038.
> >>>    
> >>>>
> >>>> Maybe the kernel does warn "on device" so we could have a systemd
> >>>> unit warning for all filesystems ... which would probably best
> >>>> find its place in the kernel and or debian.    
> >>>
> >>> At least for affected ext file systems the kernel will warn (on
> >>> mount). But I considered that as "too late".    
> >>
> >> To be more specific: Linux >= 5.4 warns. That's why I guess that
> >> many projects did not realize that they are already affected by
> >> the Y2038 problem because of older kernel versions.  
> > 
> > Which sounds like that warning needs backporting into the debian10
> > kernel and maybe cip. Not sure Isar is the best place, but a valid
> > one that could help.
> > 
> > Maybe mkfs could warn ... as well.
> >   
> 
> Right, and we want such warnings seen at image *build time*, not only
> on the target during runtime. That is the key idea behind this
> instrumentation.
> 
> Jan
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Baurzhan Ismagulov]

On Thu, Feb 11, 2021 at 07:01:50PM +0100, Henning Schild wrote:
> Florian, maybe you send a revert series. Not your fault but maybe your
> call.

For that matter, we can discuss reverting. That said, I'd like to understand
the situation first.

I know that you invested much effort for integrating wic without changes and
keeping it unmodified; this prevents maintenance effort. Upstreaming the
changes is also good for the same reason; Florian is doing that. If the changes
are accepted, we update wic -- everything fine. If not, we still can decide
what to do with that -- no doors are closed. Currently, Isar warns users about
the problem -- added value. I personally fail to see what value should
reverting have in this situation.

On the maintainer side, I think we could test the following additions:

* Even if the maintainer thinks an RFC patch is good enough as is, it's advised
  to sync with the list.

* If a patch changes upstream copies (bitbake, wic; anything else?), double
  checking is advised.

With kind regards,
Baurzhan.

[PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[Florian Bezdeka]

This is the backport for upstream (openembedded-core)
eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038 problem is used")

We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.

ext2 and ext3 filesystems are always affected by the time overflow, so
let's warn the user if these filesystems are still being used.

If ext4 is affected depends on the inode size chosen during filesystem
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended
later) this might be an issue for many users.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 scripts/lib/wic/misc.py      |  1 +
 scripts/lib/wic/partition.py | 15 +++++++--------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 4b08d64..c4332d5 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
 
 # executable -> recipe pairs for exec_native_cmd
 NATIVE_RECIPES = {"bmaptool": "bmap-tools",
+                  "dumpe2fs": "e2fsprogs",
                   "grub-mkimage": "grub-efi",
                   "isohybrid": "syslinux",
                   "mcopy": "mtools",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 4a5a31e..e6bcc9e 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -432,26 +432,25 @@ class Partition():
             if part.mountpoint:
                 args = [part.fstype, "mounted at %s" % part.mountpoint]
             elif part.label:
-                args = [part.fstype, "labeled %s" % part.label]
+                args = [part.fstype, "labeled '%s'" % part.label]
             elif part.part_name:
-                args = [part.fstype, "in partition %s" % part.part_name]
+                args = [part.fstype, "in partition '%s'" % part.part_name]
             else:
-                args = [part.fstype, ""]
+                args = [part.fstype, "in partition %s" % part.num]
             return err.format(*args)
 
-        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
-
         # ext2 and ext3 are always affected by the Y2038 problem
         if self.fstype in ["ext2", "ext3"]:
             logger.warn(get_err_str(self))
             return
 
+        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
+
         # if ext4 is affected by the Y2038 problem depends on the inode size
-        # Remember: inode size depends on the file system size
         for line in out.splitlines():
             if line.startswith("Inode size:"):
                 size = int(line.split(":")[1].strip())
                 if size < 256:
-                    logger.warn("%s Inodes (of size %d) are too small." % \
+                    logger.warn("%s Inodes (of size %d) are too small." %
                                 (get_err_str(self), size))
-                break
\ No newline at end of file
+                break
-- 
2.29.2

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[vijaikumar....@gmail.com]

[-- Attachment #1.1: Type: text/plain, Size: 3176 bytes --]



On Monday, March 1, 2021 at 8:49:34 PM UTC+5:30 Florian Bezdeka wrote:

> This is the backport for upstream (openembedded-core) 
> eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038 
> problem is used") 
>

I believe we could uprev wic to a version that includes this patch, instead 
of cherry-picking this.

Thanks,
Vijai Kumar K
 

>
> We are getting closer and closer to the year 2038 where the 32 bit 
> time_t overflow will happen. While products (= embedded systems) with an 
> expected life time of 15 years are still save the situation may change 
> if your system has to survive the next 20 years. 
>
> ext2 and ext3 filesystems are always affected by the time overflow, so 
> let's warn the user if these filesystems are still being used. 
>
> If ext4 is affected depends on the inode size chosen during filesystem 
> creation. At least 256 bytes are necessary to be safe. As ext4 is 
> used very often (and partitions may be created small first and extended 
> later) this might be an issue for many users. 
>
> Signed-off-by: Florian Bezdeka <florian...@siemens.com> 
> --- 
> scripts/lib/wic/misc.py | 1 + 
> scripts/lib/wic/partition.py | 15 +++++++-------- 
> 2 files changed, 8 insertions(+), 8 deletions(-) 
>
> diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py 
> index 4b08d64..c4332d5 100644 
> --- a/scripts/lib/wic/misc.py 
> +++ b/scripts/lib/wic/misc.py 
> @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') 
>
> # executable -> recipe pairs for exec_native_cmd 
> NATIVE_RECIPES = {"bmaptool": "bmap-tools", 
> + "dumpe2fs": "e2fsprogs", 
> "grub-mkimage": "grub-efi", 
> "isohybrid": "syslinux", 
> "mcopy": "mtools", 
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py 
> index 4a5a31e..e6bcc9e 100644 
> --- a/scripts/lib/wic/partition.py 
> +++ b/scripts/lib/wic/partition.py 
> @@ -432,26 +432,25 @@ class Partition(): 
> if part.mountpoint: 
> args = [part.fstype, "mounted at %s" % part.mountpoint] 
> elif part.label: 
> - args = [part.fstype, "labeled %s" % part.label] 
> + args = [part.fstype, "labeled '%s'" % part.label] 
> elif part.part_name: 
> - args = [part.fstype, "in partition %s" % part.part_name] 
> + args = [part.fstype, "in partition '%s'" % part.part_name] 
> else: 
> - args = [part.fstype, ""] 
> + args = [part.fstype, "in partition %s" % part.num] 
> return err.format(*args) 
>
> - ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) 
> - 
> # ext2 and ext3 are always affected by the Y2038 problem 
> if self.fstype in ["ext2", "ext3"]: 
> logger.warn(get_err_str(self)) 
> return 
>
> + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) 
> + 
> # if ext4 is affected by the Y2038 problem depends on the inode size 
> - # Remember: inode size depends on the file system size 
> for line in out.splitlines(): 
> if line.startswith("Inode size:"): 
> size = int(line.split(":")[1].strip()) 
> if size < 256: 
> - logger.warn("%s Inodes (of size %d) are too small." % \ 
> + logger.warn("%s Inodes (of size %d) are too small." % 
> (get_err_str(self), size)) 
> - break 
> \ No newline at end of file 
> + break 
> -- 
> 2.29.2 
>
>

[-- Attachment #1.2: Type: text/html, Size: 4242 bytes --]

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

On Mon, 2021-03-01 at 07:23 -0800, vijaikumar....@gmail.com wrote:
> 
> 
> On Monday, March 1, 2021 at 8:49:34 PM UTC+5:30 Florian Bezdeka
> wrote:
> > This is the backport for upstream (openembedded-core)
> > eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038
> > problem is used")
> > 
> 
> 
> I believe we could uprev wic to a version that includes this patch,
> instead of cherry-picking this.

It's not a cherry-pick because the RFC series has already been merged
into ISAR before it was merged upstream. It's a real backport. 

wic of ISAR and upstream (openembedded-core) have diverged, so just
picking the recent upstream version would overwrite all the changes
that were never upstreamed. I can't even test that, so someone else has
to do that.

> 
> Thanks,
> Vijai Kumar K
>  
> > 
> > We are getting closer and closer to the year 2038 where the 32 bit
> > time_t overflow will happen. While products (= embedded systems)
> > with
> > an
> > expected life time of 15 years are still save the situation may
> > change
> > if your system has to survive the next 20 years.
> > 
> > ext2 and ext3 filesystems are always affected by the time overflow,
> > so
> > let's warn the user if these filesystems are still being used.
> > 
> > If ext4 is affected depends on the inode size chosen during
> > filesystem
> > creation. At least 256 bytes are necessary to be safe. As ext4 is
> > used very often (and partitions may be created small first and
> > extended
> > later) this might be an issue for many users.
> > 
> > Signed-off-by: Florian Bezdeka <florian...@siemens.com>
> > ---
> >  scripts/lib/wic/misc.py | 1 +
> >  scripts/lib/wic/partition.py | 15 +++++++--------
> >  2 files changed, 8 insertions(+), 8 deletions(-)
> > 
> > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> > index 4b08d64..c4332d5 100644
> > --- a/scripts/lib/wic/misc.py
> > +++ b/scripts/lib/wic/misc.py
> > @@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
> >  
> >  # executable -> recipe pairs for exec_native_cmd
> >  NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> > + "dumpe2fs": "e2fsprogs",
> >  "grub-mkimage": "grub-efi",
> >  "isohybrid": "syslinux",
> >  "mcopy": "mtools",
> > diff --git a/scripts/lib/wic/partition.py
> > b/scripts/lib/wic/partition.py
> > index 4a5a31e..e6bcc9e 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -432,26 +432,25 @@ class Partition():
> >  if part.mountpoint:
> >  args = [part.fstype, "mounted at %s" % part.mountpoint]
> >  elif part.label:
> > - args = [part.fstype, "labeled %s" % part.label]
> > + args = [part.fstype, "labeled '%s'" % part.label]
> >  elif part.part_name:
> > - args = [part.fstype, "in partition %s" % part.part_name]
> > + args = [part.fstype, "in partition '%s'" % part.part_name]
> >  else:
> > - args = [part.fstype, ""]
> > + args = [part.fstype, "in partition %s" % part.num]
> >  return err.format(*args)
> >  
> > - ret, out = exec_native_cmd("dumpe2fs %s" % rootfs,
> > native_sysroot)
> > -
> >  # ext2 and ext3 are always affected by the Y2038 problem
> >  if self.fstype in ["ext2", "ext3"]:
> >  logger.warn(get_err_str(self))
> >  return
> >  
> > + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs,
> > native_sysroot)
> > +
> >  # if ext4 is affected by the Y2038 problem depends on the inode
> > size
> > - # Remember: inode size depends on the file system size
> >  for line in out.splitlines():
> >  if line.startswith("Inode size:"):
> >  size = int(line.split(":")[1].strip())
> >  if size < 256:
> > - logger.warn("%s Inodes (of size %d) are too small." % \
> > + logger.warn("%s Inodes (of size %d) are too small." %
> >  (get_err_str(self), size))
> > - break
> > \ No newline at end of file
> > + break
> > -- 
> > 2.29.2
> > 

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[vijaikumar....@gmail.com]

[-- Attachment #1.1: Type: text/plain, Size: 4402 bytes --]



On Monday, March 1, 2021 at 9:08:26 PM UTC+5:30 florian...@siemens.com 
wrote:

> On Mon, 2021-03-01 at 07:23 -0800, vijaikumar....@gmail.com wrote: 
> > 
> > 
> > On Monday, March 1, 2021 at 8:49:34 PM UTC+5:30 Florian Bezdeka 
> > wrote: 
> > > This is the backport for upstream (openembedded-core) 
> > > eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038 
> > > problem is used") 
> > > 
> > 
> > 
> > I believe we could uprev wic to a version that includes this patch, 
> > instead of cherry-picking this. 
>
> It's not a cherry-pick because the RFC series has already been merged 
> into ISAR before it was merged upstream. It's a real backport.  
>

As I see, these are the missing pieces of the RFC patch which got merged 
recently.
The backport is actually spread across 2 commits.

Maybe we could modify the commit message to reflect that.

Thanks,
Vijai Kumar K


> wic of ISAR and upstream (openembedded-core) have diverged, so just 
> picking the recent upstream version would overwrite all the changes 
> that were never upstreamed. I can't even test that, so someone else has 
> to do that. 
>
> > 
> > Thanks, 
> > Vijai Kumar K 
> >   
> > > 
> > > We are getting closer and closer to the year 2038 where the 32 bit 
> > > time_t overflow will happen. While products (= embedded systems) 
> > > with 
> > > an 
> > > expected life time of 15 years are still save the situation may 
> > > change 
> > > if your system has to survive the next 20 years. 
> > > 
> > > ext2 and ext3 filesystems are always affected by the time overflow, 
> > > so 
> > > let's warn the user if these filesystems are still being used. 
> > > 
> > > If ext4 is affected depends on the inode size chosen during 
> > > filesystem 
> > > creation. At least 256 bytes are necessary to be safe. As ext4 is 
> > > used very often (and partitions may be created small first and 
> > > extended 
> > > later) this might be an issue for many users. 
> > > 
> > > Signed-off-by: Florian Bezdeka <florian...@siemens.com> 
> > > --- 
> > >  scripts/lib/wic/misc.py | 1 + 
> > >  scripts/lib/wic/partition.py | 15 +++++++-------- 
> > >  2 files changed, 8 insertions(+), 8 deletions(-) 
> > > 
> > > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py 
> > > index 4b08d64..c4332d5 100644 
> > > --- a/scripts/lib/wic/misc.py 
> > > +++ b/scripts/lib/wic/misc.py 
> > > @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') 
> > >   
> > >  # executable -> recipe pairs for exec_native_cmd 
> > >  NATIVE_RECIPES = {"bmaptool": "bmap-tools", 
> > > + "dumpe2fs": "e2fsprogs", 
> > >  "grub-mkimage": "grub-efi", 
> > >  "isohybrid": "syslinux", 
> > >  "mcopy": "mtools", 
> > > diff --git a/scripts/lib/wic/partition.py 
> > > b/scripts/lib/wic/partition.py 
> > > index 4a5a31e..e6bcc9e 100644 
> > > --- a/scripts/lib/wic/partition.py 
> > > +++ b/scripts/lib/wic/partition.py 
> > > @@ -432,26 +432,25 @@ class Partition(): 
> > >  if part.mountpoint: 
> > >  args = [part.fstype, "mounted at %s" % part.mountpoint] 
> > >  elif part.label: 
> > > - args = [part.fstype, "labeled %s" % part.label] 
> > > + args = [part.fstype, "labeled '%s'" % part.label] 
> > >  elif part.part_name: 
> > > - args = [part.fstype, "in partition %s" % part.part_name] 
> > > + args = [part.fstype, "in partition '%s'" % part.part_name] 
> > >  else: 
> > > - args = [part.fstype, ""] 
> > > + args = [part.fstype, "in partition %s" % part.num] 
> > >  return err.format(*args) 
> > >   
> > > - ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, 
> > > native_sysroot) 
> > > - 
> > >  # ext2 and ext3 are always affected by the Y2038 problem 
> > >  if self.fstype in ["ext2", "ext3"]: 
> > >  logger.warn(get_err_str(self)) 
> > >  return 
> > >   
> > > + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, 
> > > native_sysroot) 
> > > + 
> > >  # if ext4 is affected by the Y2038 problem depends on the inode 
> > > size 
> > > - # Remember: inode size depends on the file system size 
> > >  for line in out.splitlines(): 
> > >  if line.startswith("Inode size:"): 
> > >  size = int(line.split(":")[1].strip()) 
> > >  if size < 256: 
> > > - logger.warn("%s Inodes (of size %d) are too small." % \ 
> > > + logger.warn("%s Inodes (of size %d) are too small." % 
> > >  (get_err_str(self), size)) 
> > > - break 
> > > \ No newline at end of file 
> > > + break 
> > > -- 
> > > 2.29.2 
> > > 
>
>

[-- Attachment #1.2: Type: text/html, Size: 5815 bytes --]

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[Jan Kiszka]

On 01.03.21 16:18, Florian Bezdeka wrote:
> This is the backport for upstream (openembedded-core)
> eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038 problem is used")
> 

We need to sync to a specific commit of upstream, rather than expanding
the fork.

Jan

> We are getting closer and closer to the year 2038 where the 32 bit
> time_t overflow will happen. While products (= embedded systems) with an
> expected life time of 15 years are still save the situation may change
> if your system has to survive the next 20 years.
> 
> ext2 and ext3 filesystems are always affected by the time overflow, so
> let's warn the user if these filesystems are still being used.
> 
> If ext4 is affected depends on the inode size chosen during filesystem
> creation. At least 256 bytes are necessary to be safe. As ext4 is
> used very often (and partitions may be created small first and extended
> later) this might be an issue for many users.
> 
> Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> ---
>  scripts/lib/wic/misc.py      |  1 +
>  scripts/lib/wic/partition.py | 15 +++++++--------
>  2 files changed, 8 insertions(+), 8 deletions(-)
> 
> diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> index 4b08d64..c4332d5 100644
> --- a/scripts/lib/wic/misc.py
> +++ b/scripts/lib/wic/misc.py
> @@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
>  
>  # executable -> recipe pairs for exec_native_cmd
>  NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> +                  "dumpe2fs": "e2fsprogs",
>                    "grub-mkimage": "grub-efi",
>                    "isohybrid": "syslinux",
>                    "mcopy": "mtools",
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index 4a5a31e..e6bcc9e 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -432,26 +432,25 @@ class Partition():
>              if part.mountpoint:
>                  args = [part.fstype, "mounted at %s" % part.mountpoint]
>              elif part.label:
> -                args = [part.fstype, "labeled %s" % part.label]
> +                args = [part.fstype, "labeled '%s'" % part.label]
>              elif part.part_name:
> -                args = [part.fstype, "in partition %s" % part.part_name]
> +                args = [part.fstype, "in partition '%s'" % part.part_name]
>              else:
> -                args = [part.fstype, ""]
> +                args = [part.fstype, "in partition %s" % part.num]
>              return err.format(*args)
>  
> -        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
> -
>          # ext2 and ext3 are always affected by the Y2038 problem
>          if self.fstype in ["ext2", "ext3"]:
>              logger.warn(get_err_str(self))
>              return
>  
> +        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
> +
>          # if ext4 is affected by the Y2038 problem depends on the inode size
> -        # Remember: inode size depends on the file system size
>          for line in out.splitlines():
>              if line.startswith("Inode size:"):
>                  size = int(line.split(":")[1].strip())
>                  if size < 256:
> -                    logger.warn("%s Inodes (of size %d) are too small." % \
> +                    logger.warn("%s Inodes (of size %d) are too small." %
>                                  (get_err_str(self), size))
> -                break
> \ No newline at end of file
> +                break
> 

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[florian.bezdeka]

On Mon, 2021-03-01 at 18:22 +0100, Jan Kiszka wrote:
> On 01.03.21 16:18, Florian Bezdeka wrote:
> > This is the backport for upstream (openembedded-core)
> > eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038 problem is used")
> > 
> 
> We need to sync to a specific commit of upstream, rather than expanding
> the fork.

Repeating from the other thread:

wic in ISAR has diverged from upstream, so updating or synchronizing it
to the upstream version is not doable for me. I can't even test that.

As the RFC series was merged by accident the situation got even worse.
I can take the responsibility to synchronize that part with the
upstream version, but I can't take responsibility for a full
synchronization. 

As already suggested, I would update the commit message again, but no
way for me to do the full synchronization.

> 
> Jan
> 
> > We are getting closer and closer to the year 2038 where the 32 bit
> > time_t overflow will happen. While products (= embedded systems) with an
> > expected life time of 15 years are still save the situation may change
> > if your system has to survive the next 20 years.
> > 
> > ext2 and ext3 filesystems are always affected by the time overflow, so
> > let's warn the user if these filesystems are still being used.
> > 
> > If ext4 is affected depends on the inode size chosen during filesystem
> > creation. At least 256 bytes are necessary to be safe. As ext4 is
> > used very often (and partitions may be created small first and extended
> > later) this might be an issue for many users.
> > 
> > Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> > ---
> >  scripts/lib/wic/misc.py      |  1 +
> >  scripts/lib/wic/partition.py | 15 +++++++--------
> >  2 files changed, 8 insertions(+), 8 deletions(-)
> > 
> > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> > index 4b08d64..c4332d5 100644
> > --- a/scripts/lib/wic/misc.py
> > +++ b/scripts/lib/wic/misc.py
> > @@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
> >  
> > 
> > 
> > 
> >  # executable -> recipe pairs for exec_native_cmd
> >  NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> > +                  "dumpe2fs": "e2fsprogs",
> >                    "grub-mkimage": "grub-efi",
> >                    "isohybrid": "syslinux",
> >                    "mcopy": "mtools",
> > diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> > index 4a5a31e..e6bcc9e 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -432,26 +432,25 @@ class Partition():
> >              if part.mountpoint:
> >                  args = [part.fstype, "mounted at %s" % part.mountpoint]
> >              elif part.label:
> > -                args = [part.fstype, "labeled %s" % part.label]
> > +                args = [part.fstype, "labeled '%s'" % part.label]
> >              elif part.part_name:
> > -                args = [part.fstype, "in partition %s" % part.part_name]
> > +                args = [part.fstype, "in partition '%s'" % part.part_name]
> >              else:
> > -                args = [part.fstype, ""]
> > +                args = [part.fstype, "in partition %s" % part.num]
> >              return err.format(*args)
> >  
> > 
> > 
> > 
> > -        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
> > -
> >          # ext2 and ext3 are always affected by the Y2038 problem
> >          if self.fstype in ["ext2", "ext3"]:
> >              logger.warn(get_err_str(self))
> >              return
> >  
> > 
> > 
> > 
> > +        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot)
> > +
> >          # if ext4 is affected by the Y2038 problem depends on the inode size
> > -        # Remember: inode size depends on the file system size
> >          for line in out.splitlines():
> >              if line.startswith("Inode size:"):
> >                  size = int(line.split(":")[1].strip())
> >                  if size < 256:
> > -                    logger.warn("%s Inodes (of size %d) are too small." % \
> > +                    logger.warn("%s Inodes (of size %d) are too small." %
> >                                  (get_err_str(self), size))
> > -                break
> > \ No newline at end of file
> > +                break
> > 
> 

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[vijaikumar....@gmail.com]

[-- Attachment #1.1: Type: text/plain, Size: 4712 bytes --]



On Monday, March 1, 2021 at 11:15:08 PM UTC+5:30 florian...@siemens.com 
wrote:

> On Mon, 2021-03-01 at 18:22 +0100, Jan Kiszka wrote: 
> > On 01.03.21 16:18, Florian Bezdeka wrote: 
> > > This is the backport for upstream (openembedded-core) 
> > > eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038 
> problem is used") 
> > > 
> > 
> > We need to sync to a specific commit of upstream, rather than expanding 
> > the fork. 
>
> Repeating from the other thread: 
>
> wic in ISAR has diverged from upstream, so updating or synchronizing it 
> to the upstream version is not doable for me. I can't even test that. 
>
> As the RFC series was merged by accident the situation got even worse. 
> I can take the responsibility to synchronize that part with the 
> upstream version, but I can't take responsibility for a full 
> synchronization.  
>
> As already suggested, I would update the commit message again, but no 
> way for me to do the full synchronization. 
>

I could do that. And probable those pending ISAR wic plugin alignment as 
well. But not immediately. It will have to wait till next week, probably 
weekend. I have some stuff to clear from my plate first.  

Thanks,
Vijai Kumar K


> > 
> > Jan 
> > 
> > > We are getting closer and closer to the year 2038 where the 32 bit 
> > > time_t overflow will happen. While products (= embedded systems) with 
> an 
> > > expected life time of 15 years are still save the situation may change 
> > > if your system has to survive the next 20 years. 
> > > 
> > > ext2 and ext3 filesystems are always affected by the time overflow, so 
> > > let's warn the user if these filesystems are still being used. 
> > > 
> > > If ext4 is affected depends on the inode size chosen during filesystem 
> > > creation. At least 256 bytes are necessary to be safe. As ext4 is 
> > > used very often (and partitions may be created small first and 
> extended 
> > > later) this might be an issue for many users. 
> > > 
> > > Signed-off-by: Florian Bezdeka <florian...@siemens.com> 
> > > --- 
> > >  scripts/lib/wic/misc.py | 1 + 
> > >  scripts/lib/wic/partition.py | 15 +++++++-------- 
> > >  2 files changed, 8 insertions(+), 8 deletions(-) 
> > > 
> > > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py 
> > > index 4b08d64..c4332d5 100644 
> > > --- a/scripts/lib/wic/misc.py 
> > > +++ b/scripts/lib/wic/misc.py 
> > > @@ -26,6 +26,7 @@ logger = logging.getLogger('wic') 
> > >   
> > > 
> > > 
> > > 
> > >  # executable -> recipe pairs for exec_native_cmd 
> > >  NATIVE_RECIPES = {"bmaptool": "bmap-tools", 
> > > + "dumpe2fs": "e2fsprogs", 
> > >                    "grub-mkimage": "grub-efi", 
> > >                    "isohybrid": "syslinux", 
> > >                    "mcopy": "mtools", 
> > > diff --git a/scripts/lib/wic/partition.py 
> b/scripts/lib/wic/partition.py 
> > > index 4a5a31e..e6bcc9e 100644 
> > > --- a/scripts/lib/wic/partition.py 
> > > +++ b/scripts/lib/wic/partition.py 
> > > @@ -432,26 +432,25 @@ class Partition(): 
> > >              if part.mountpoint: 
> > >                  args = [part.fstype, "mounted at %s" % 
> part.mountpoint] 
> > >              elif part.label: 
> > > - args = [part.fstype, "labeled %s" % part.label] 
> > > + args = [part.fstype, "labeled '%s'" % part.label] 
> > >              elif part.part_name: 
> > > - args = [part.fstype, "in partition %s" % part.part_name] 
> > > + args = [part.fstype, "in partition '%s'" % part.part_name] 
> > >              else: 
> > > - args = [part.fstype, ""] 
> > > + args = [part.fstype, "in partition %s" % part.num] 
> > >              return err.format(*args) 
> > >   
> > > 
> > > 
> > > 
> > > - ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) 
> > > - 
> > >          # ext2 and ext3 are always affected by the Y2038 problem 
> > >          if self.fstype in ["ext2", "ext3"]: 
> > >              logger.warn(get_err_str(self)) 
> > >              return 
> > >   
> > > 
> > > 
> > > 
> > > + ret, out = exec_native_cmd("dumpe2fs %s" % rootfs, native_sysroot) 
> > > + 
> > >          # if ext4 is affected by the Y2038 problem depends on the 
> inode size 
> > > - # Remember: inode size depends on the file system size 
> > >          for line in out.splitlines(): 
> > >              if line.startswith("Inode size:"): 
> > >                  size = int(line.split(":")[1].strip()) 
> > >                  if size < 256: 
> > > - logger.warn("%s Inodes (of size %d) are too small." % \ 
> > > + logger.warn("%s Inodes (of size %d) are too small." % 
> > >                                  (get_err_str(self), size)) 
> > > - break 
> > > \ No newline at end of file 
> > > + break 
> > > 
> > 
>
>

[-- Attachment #1.2: Type: text/html, Size: 7498 bytes --]

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[Henning Schild]

We usually try to not backport but bump the whole bitbake. There have
been exceptions, but usually because maintainer did not enforce that,
not because commits have been "super important".
While this one looks good, i would say it does not justify such forking
and will need to wait for the next bitbake version bump.

But feel free to bump all of bitbake, might be smooth or a significant
amount of work.

Henning

Am Mon, 1 Mar 2021 16:18:23 +0100
schrieb Florian Bezdeka <florian.bezdeka@siemens.com>:

> This is the backport for upstream (openembedded-core)
> eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038
> problem is used")
> 
> We are getting closer and closer to the year 2038 where the 32 bit
> time_t overflow will happen. While products (= embedded systems) with
> an expected life time of 15 years are still save the situation may
> change if your system has to survive the next 20 years.
> 
> ext2 and ext3 filesystems are always affected by the time overflow, so
> let's warn the user if these filesystems are still being used.
> 
> If ext4 is affected depends on the inode size chosen during filesystem
> creation. At least 256 bytes are necessary to be safe. As ext4 is
> used very often (and partitions may be created small first and
> extended later) this might be an issue for many users.
> 
> Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> ---
>  scripts/lib/wic/misc.py      |  1 +
>  scripts/lib/wic/partition.py | 15 +++++++--------
>  2 files changed, 8 insertions(+), 8 deletions(-)
> 
> diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> index 4b08d64..c4332d5 100644
> --- a/scripts/lib/wic/misc.py
> +++ b/scripts/lib/wic/misc.py
> @@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
>  
>  # executable -> recipe pairs for exec_native_cmd
>  NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> +                  "dumpe2fs": "e2fsprogs",
>                    "grub-mkimage": "grub-efi",
>                    "isohybrid": "syslinux",
>                    "mcopy": "mtools",
> diff --git a/scripts/lib/wic/partition.py
> b/scripts/lib/wic/partition.py index 4a5a31e..e6bcc9e 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -432,26 +432,25 @@ class Partition():
>              if part.mountpoint:
>                  args = [part.fstype, "mounted at %s" %
> part.mountpoint] elif part.label:
> -                args = [part.fstype, "labeled %s" % part.label]
> +                args = [part.fstype, "labeled '%s'" % part.label]
>              elif part.part_name:
> -                args = [part.fstype, "in partition %s" %
> part.part_name]
> +                args = [part.fstype, "in partition '%s'" %
> part.part_name] else:
> -                args = [part.fstype, ""]
> +                args = [part.fstype, "in partition %s" % part.num]
>              return err.format(*args)
>  
> -        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs,
> native_sysroot) -
>          # ext2 and ext3 are always affected by the Y2038 problem
>          if self.fstype in ["ext2", "ext3"]:
>              logger.warn(get_err_str(self))
>              return
>  
> +        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs,
> native_sysroot) +
>          # if ext4 is affected by the Y2038 problem depends on the
> inode size
> -        # Remember: inode size depends on the file system size
>          for line in out.splitlines():
>              if line.startswith("Inode size:"):
>                  size = int(line.split(":")[1].strip())
>                  if size < 256:
> -                    logger.warn("%s Inodes (of size %d) are too
> small." % \
> +                    logger.warn("%s Inodes (of size %d) are too
> small." % (get_err_str(self), size))
> -                break
> \ No newline at end of file
> +                break

Re: [PATCH] wic: Warn if an ext filesystem affected by the Y2038 problem is used

[Jan Kiszka]

On 02.03.21 10:20, Henning Schild wrote:
> We usually try to not backport but bump the whole bitbake. There have
> been exceptions, but usually because maintainer did not enforce that,
> not because commits have been "super important".
> While this one looks good, i would say it does not justify such forking
> and will need to wait for the next bitbake version bump.
> 
> But feel free to bump all of bitbake, might be smooth or a significant
> amount of work.

s/bitbake/wic/g, I suspect...

Jan

> 
> Henning
> 
> Am Mon, 1 Mar 2021 16:18:23 +0100
> schrieb Florian Bezdeka <florian.bezdeka@siemens.com>:
> 
>> This is the backport for upstream (openembedded-core)
>> eecbe6255584 ("wic: Warn if an ext filesystem affected by the Y2038
>> problem is used")
>>
>> We are getting closer and closer to the year 2038 where the 32 bit
>> time_t overflow will happen. While products (= embedded systems) with
>> an expected life time of 15 years are still save the situation may
>> change if your system has to survive the next 20 years.
>>
>> ext2 and ext3 filesystems are always affected by the time overflow, so
>> let's warn the user if these filesystems are still being used.
>>
>> If ext4 is affected depends on the inode size chosen during filesystem
>> creation. At least 256 bytes are necessary to be safe. As ext4 is
>> used very often (and partitions may be created small first and
>> extended later) this might be an issue for many users.
>>
>> Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
>> ---
>>  scripts/lib/wic/misc.py      |  1 +
>>  scripts/lib/wic/partition.py | 15 +++++++--------
>>  2 files changed, 8 insertions(+), 8 deletions(-)
>>
>> diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
>> index 4b08d64..c4332d5 100644
>> --- a/scripts/lib/wic/misc.py
>> +++ b/scripts/lib/wic/misc.py
>> @@ -26,6 +26,7 @@ logger = logging.getLogger('wic')
>>  
>>  # executable -> recipe pairs for exec_native_cmd
>>  NATIVE_RECIPES = {"bmaptool": "bmap-tools",
>> +                  "dumpe2fs": "e2fsprogs",
>>                    "grub-mkimage": "grub-efi",
>>                    "isohybrid": "syslinux",
>>                    "mcopy": "mtools",
>> diff --git a/scripts/lib/wic/partition.py
>> b/scripts/lib/wic/partition.py index 4a5a31e..e6bcc9e 100644
>> --- a/scripts/lib/wic/partition.py
>> +++ b/scripts/lib/wic/partition.py
>> @@ -432,26 +432,25 @@ class Partition():
>>              if part.mountpoint:
>>                  args = [part.fstype, "mounted at %s" %
>> part.mountpoint] elif part.label:
>> -                args = [part.fstype, "labeled %s" % part.label]
>> +                args = [part.fstype, "labeled '%s'" % part.label]
>>              elif part.part_name:
>> -                args = [part.fstype, "in partition %s" %
>> part.part_name]
>> +                args = [part.fstype, "in partition '%s'" %
>> part.part_name] else:
>> -                args = [part.fstype, ""]
>> +                args = [part.fstype, "in partition %s" % part.num]
>>              return err.format(*args)
>>  
>> -        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs,
>> native_sysroot) -
>>          # ext2 and ext3 are always affected by the Y2038 problem
>>          if self.fstype in ["ext2", "ext3"]:
>>              logger.warn(get_err_str(self))
>>              return
>>  
>> +        ret, out = exec_native_cmd("dumpe2fs %s" % rootfs,
>> native_sysroot) +
>>          # if ext4 is affected by the Y2038 problem depends on the
>> inode size
>> -        # Remember: inode size depends on the file system size
>>          for line in out.splitlines():
>>              if line.startswith("Inode size:"):
>>                  size = int(line.split(":")[1].strip())
>>                  if size < 256:
>> -                    logger.warn("%s Inodes (of size %d) are too
>> small." % \
>> +                    logger.warn("%s Inodes (of size %d) are too
>> small." % (get_err_str(self), size))
>> -                break
>> \ No newline at end of file
>> +                break
> 

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Jan Kiszka]

On 01.02.21 19:58, Bezdeka, Florian (T RDA IOT SES-DE) wrote:
> Valid workarounds found so far:
>  - Tell wic that an partition will grow:
>    Add `--mkfs-extraopts "-T ext4"` to your wic partition definition
>  - Set the inode size to 256 (for small ext4 partitions)
>    Add `--mkfs-extraopts "-I 256"` to your wic partition definition

Is this check is currently part of Isar (triggering on one of my layers
at least) and will eventually come back via OE, concrete help how to
resolve the warning should become part of Isar as well.

Apparently, --mkfs-extraopts "-T default" is the common pattern now, right?

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Florian Bezdeka]

On 27.03.21 08:20, Jan Kiszka wrote:
> On 01.02.21 19:58, Bezdeka, Florian (T RDA IOT SES-DE) wrote:
>> Valid workarounds found so far:
>>  - Tell wic that an partition will grow:
>>    Add `--mkfs-extraopts "-T ext4"` to your wic partition definition
>>  - Set the inode size to 256 (for small ext4 partitions)
>>    Add `--mkfs-extraopts "-I 256"` to your wic partition definition
> 
> Is this check is currently part of Isar (triggering on one of my layers
> at least) and will eventually come back via OE, concrete help how to
> resolve the warning should become part of Isar as well.>
> Apparently, --mkfs-extraopts "-T default" is the common pattern now, right?

Right. At least as long as you're not setting up a very tiny or giant FS.

> 
> Jan
> 


-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux