Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

["florian.bezdeka@siemens.com" <florian.bezdeka@siemens.com>]

On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:
> Am Thu, 11 Feb 2021 10:57:31 +0100
> schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> <florian.bezdeka@siemens.com>:
> 
> > On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:
> > > On 11.02.21 09:23, Henning Schild wrote:  
> > > > Hi all,
> > > > 
> > > > i never got around to reviewing this. But did we just fork wic?
> > > > These patches need to go into wic and we later backport them once
> > > > they are accepted upstream.
> > > > 
> > > > Maybe they are already ... did not check.
> > > > 
> > > > When it comes to changing bitbake or wic, we should really not
> > > > ... We have forks of some files, like the wic plugins and bitbake
> > > > config, those are fine but should also stay very close to
> > > > upstream.
> > > > 
> > > > The recently applied patch from Vijai also violates that. Since
> > > > the fork of the plugins was not updated with the wic bump and the
> > > > repair just takes a few bits of what we probably should take.
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > If you are referring to
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7C48d6471d1d4341e4445d08d8ce778b07%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486360122035313%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=61t42JDRuSWYJF%2Ff6rE6A7A9o0%2BlDF7zKwN85LVo%2BiU%3D&amp;reserved=0:
> > > That one was "only" patching an isar version, though I agree that we
> > > should make sure to realign it with the original plugins if we are
> > > now imbalanced.
> > > 
> > > This one here is more critical as it changed a formerly vanilla wic
> > > file. That should be fixed quickly.
> > > 
> > > Florian, maybe you can propose a similar change to OE upstream? In
> > > the meantime, is there a chance to move the changes out of
> > > partition.py, to a file that is isar-specific?
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
> > I guess the "RFC" tag of this series has been overlooked. It was not
> > intended for merging (yet). Part one (forwarding wic warnings to
> > bitbake) is a pure ISAR change and could be taken as is (if no further
> > comments come up).
> 
> I guess that calls for a revert. And for more attention on the
> maintainers side.
> 
> > Sorry for the long description of the series, but if you read closely
> > I already mentioned that the second part should go to OE. I sent it
> > out for feedback collection only.
> > 
> > The upstreaming to OE will take some time due to internal
> > clarifications. I never contributed to OE before, so some kind of
> > approval process has to be followed first.
> > 
> > At first glance there was no easy way moving the warnings from wic to
> > ISAR. We would have to re-parse the wic template file again and check
> > all the partitions afterwards. wic has all the necessary information
> > at hand so I guess that's way easier.
> 
> I guess it can be moved into a task after wic. Here one would need to
> parse the partition table, which kind of sucks. "losetup" or "kpartx"
> might help but will not work in kas-container setups because they need
> root.
> We once had patches allowing wic to retain all partition images instead
> of throwing them away after disk assembly. Having a switch for wic to
> say ... do those partitions ... later do the disk would be generic,
> allow hooking in this and other things.
> 
> Isar also has a class that creates ext4 images without, after which such
> a check should also be done.

Yes. But instead of spreading the warnings around it would be nice to
have a single place where we could do the Y2038 checks. So maybe it
should be a base feature of "image.bbclass"? Or ext4-img.bbclass should
call wic instead of the mke2fs utilities directly?

BTW: The name ext4-img.bbclass is kind of misleading. You could simply
create ext{2,3} file systems by setting MKE2FS_ARGS to something like
"-t ext2".

> 
> Is ext4 the only fs we care about? We have some layers doing ubifs,
> squashfs and all sorts of funny things.

Up to now I cared about the filesystems supported by wic. So
ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106 (u32)
and btrfs will "never" overflow (u64). 

ubifs is similar to btrfs, so not affected by Y2038.

> 
> Maybe the kernel does warn "on device" so we could have a systemd unit
> warning for all filesystems ... which would probably best find its
> place in the kernel and or debian.

At least for affected ext file systems the kernel will warn (on mount).
But I considered that as "too late".

> 
> Henning
> 
> > 
> > > Jan
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > > Henning
> > > > 
> > > > Am Thu, 11 Feb 2021 11:07:52 +0300
> > > > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:  
> > > > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > > > 
> > > > > > Hi ISAR developers,
> > > > > > 
> > > > > > this series is the summary of a nice journey through the file
> > > > > > system jungle regarding Y2038 problem. It all began with a
> > > > > > warning which is reported by kernels >= 5.4:
> > > > > > 
> > > > > > ext4 filesystem being mounted at (mountpoint) supports
> > > > > > timestamps until 2038 (0x7fffffff)
> > > > > > 
> > > > > > I guess that most ISAR layers are using the Debian kernels,
> > > > > > so that warning was not recognized yet or at least not very
> > > > > > often.
> > > > > > 
> > > > > > When reading this warning I was surprised. Shouldn't a modern
> > > > > > file system like ext4 be Y2038-safe? As it turned out it
> > > > > > depends on the inode size if an ext4 file system is safe or
> > > > > > not. So why was the inode size not sufficient in my case?
> > > > > > 
> > > > > > The inode size is chosen during file system generation and
> > > > > > depends on the size of the file system that is going to be
> > > > > > created. For details let's have a look at `man mke2fs`:
> > > > > > 
> > > > > > -T usage-type[,...]
> > > > > >      Specify how the filesystem is going to be used, so that
> > > > > > mke2fs can choose optimal filesystem parameters for that use.
> > > > > > The usage types that are supported are defined in the
> > > > > > configuration file /etc/mke2fs.conf. The user may specify one
> > > > > > or more usage types using a comma separated list.
> > > > > > 
> > > > > >      If this option is is not specified, mke2fs will pick a
> > > > > > single default usage type based on the size of the filesystem
> > > > > > to be created. If the filesystem size is less than 3
> > > > > > megabytes, mke2fs will use the filesystem type floppy. If the
> > > > > > filesystem size is greater than or equal to 3 but less than
> > > > > > 512 megabytes, mke2fs(8) will use the filesystem type small.
> > > > > > 
> > > > > > The relevant parts from /etc/mke2fs.conf:
> > > > > > [fs_types]
> > > > > > ...
> > > > > >          small = {
> > > > > >                  blocksize = 1024
> > > > > >                  inode_size = 128
> > > > > >                  inode_ratio = 4096
> > > > > >          }
> > > > > > ...
> > > > > > 
> > > > > > So whenever you create an ext4 file system with less than
> > > > > > 512MB in size you will end up with 128 byte inodes and your
> > > > > > file system is not Y2038-safe.
> > > > > > 
> > > > > > The ISAR part:
> > > > > > ext4 may often be used in combination with the
> > > > > > expand-on-first-boot recipe / feature. So whenever creating a
> > > > > > small partition (e.g. inside a wic file) and extending it
> > > > > > later may result in a Y2038 affected ext4 file system.
> > > > > > 
> > > > > > That is exactly what happened to me and I would like to make
> > > > > > sure that all other ISAR users are aware of this situation.
> > > > > > 
> > > > > > Valid workarounds found so far:
> > > > > >   - Tell wic that an partition will grow:
> > > > > >     Add `--mkfs-extraopts "-T ext4"` to your wic partition
> > > > > > definition
> > > > > >   - Set the inode size to 256 (for small ext4 partitions)
> > > > > >     Add `--mkfs-extraopts "-I 256"` to your wic partition
> > > > > > definition
> > > > > > 
> > > > > > The upstream part:
> > > > > > None of the following patches has been sent to any upstream
> > > > > > (OE) mailing lists yet but hopefully that will happen soon.
> > > > > > So far: Any comments welcome!
> > > > > > 
> > > > > > Best regards,
> > > > > > Florian
> > > > > > 
> > > > > > Florian Bezdeka (2):
> > > > > >    wic-img: Forward warnings from wic to bitbake
> > > > > >    wic: Warn if an ext filesystem affected by the Y2038
> > > > > > problem is used
> > > > > > 
> > > > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > > > >   scripts/lib/wic/partition.py | 38
> > > > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > > > insertions(+), 5 deletions(-)  
> > > > > Applied to next, thanks.
> > > > >  
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
>