Re: [RFC PATCH 0/2] wic: warn on usage of Y2038 affected file systems

[Henning Schild <henning.schild@siemens.com>]

Am Thu, 11 Feb 2021 14:31:58 +0100
schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
<florian.bezdeka@siemens.com>:

> On Thu, 2021-02-11 at 12:47 +0000, [ext] florian.bezdeka@siemens.com
> wrote:
> > On Thu, 2021-02-11 at 11:21 +0100, Henning Schild wrote:  
> > > Am Thu, 11 Feb 2021 10:57:31 +0100
> > > schrieb "Bezdeka, Florian (T RDA IOT SES-DE)"
> > > <florian.bezdeka@siemens.com>:
> > >  
> > > > On Thu, 2021-02-11 at 10:09 +0100, Jan Kiszka wrote:  
> > > > > On 11.02.21 09:23, Henning Schild wrote:  
> > > > > > Hi all,
> > > > > >
> > > > > > i never got around to reviewing this. But did we just fork
> > > > > > wic? These patches need to go into wic and we later
> > > > > > backport them once they are accepted upstream.
> > > > > >
> > > > > > Maybe they are already ... did not check.
> > > > > >
> > > > > > When it comes to changing bitbake or wic, we should really
> > > > > > not ... We have forks of some files, like the wic plugins
> > > > > > and bitbake config, those are fine but should also stay
> > > > > > very close to upstream.
> > > > > >
> > > > > > The recently applied patch from Vijai also violates that.
> > > > > > Since the fork of the plugins was not updated with the wic
> > > > > > bump and the repair just takes a few bits of what we
> > > > > > probably should take.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >  
> > > > >
> > > > > If you are referring to
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2F20201126091750.28048-1-Vijaikumar_Kanagarajan%2540mentor.com&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Ca81479e099ce4a32a67608d8ce916870%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637486471194236324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=QvpGJS2QMLxBNb0EQZVilcyZr3CBN%2FZ48rSYlOUVisU%3D&amp;reserved=0:
> > > > > That one was "only" patching an isar version, though I agree
> > > > > that we should make sure to realign it with the original
> > > > > plugins if we are now imbalanced.
> > > > >
> > > > > This one here is more critical as it changed a formerly
> > > > > vanilla wic file. That should be fixed quickly.
> > > > >
> > > > > Florian, maybe you can propose a similar change to OE
> > > > > upstream? In the meantime, is there a chance to move the
> > > > > changes out of partition.py, to a file that is isar-specific?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  
> > > >
> > > > I guess the "RFC" tag of this series has been overlooked. It
> > > > was not intended for merging (yet). Part one (forwarding wic
> > > > warnings to bitbake) is a pure ISAR change and could be taken
> > > > as is (if no further comments come up).  
> > >
> > > I guess that calls for a revert. And for more attention on the
> > > maintainers side.
> > >  
> > > > Sorry for the long description of the series, but if you read
> > > > closely I already mentioned that the second part should go to
> > > > OE. I sent it out for feedback collection only.
> > > >
> > > > The upstreaming to OE will take some time due to internal
> > > > clarifications. I never contributed to OE before, so some kind
> > > > of approval process has to be followed first.
> > > >
> > > > At first glance there was no easy way moving the warnings from
> > > > wic to ISAR. We would have to re-parse the wic template file
> > > > again and check all the partitions afterwards. wic has all the
> > > > necessary information at hand so I guess that's way easier.  
> > >
> > > I guess it can be moved into a task after wic. Here one would
> > > need to parse the partition table, which kind of sucks. "losetup"
> > > or "kpartx" might help but will not work in kas-container setups
> > > because they need root.
> > > We once had patches allowing wic to retain all partition images
> > > instead of throwing them away after disk assembly. Having a
> > > switch for wic to say ... do those partitions ... later do the
> > > disk would be generic, allow hooking in this and other things.
> > >
> > > Isar also has a class that creates ext4 images without, after
> > > which such a check should also be done.  
> >
> > Yes. But instead of spreading the warnings around it would be nice
> > to have a single place where we could do the Y2038 checks. So maybe
> > it should be a base feature of "image.bbclass"? Or ext4-img.bbclass
> > should call wic instead of the mke2fs utilities directly?
> >
> > BTW: The name ext4-img.bbclass is kind of misleading. You could
> > simply create ext{2,3} file systems by setting MKE2FS_ARGS to
> > something like "-t ext2".
> >  
> > >
> > > Is ext4 the only fs we care about? We have some layers doing
> > > ubifs, squashfs and all sorts of funny things.  
> >
> > Up to now I cared about the filesystems supported by wic. So
> > ext{2,3,4}, btrfs and squashfs. squashfs will overflow in 2106 (u32)
> > and btrfs will "never" overflow (u64).
> >
> > ubifs is similar to btrfs, so not affected by Y2038.
> >  
> > >
> > > Maybe the kernel does warn "on device" so we could have a systemd
> > > unit warning for all filesystems ... which would probably best
> > > find its place in the kernel and or debian.  
> >
> > At least for affected ext file systems the kernel will warn (on
> > mount). But I considered that as "too late".  
> 
> To be more specific: Linux >= 5.4 warns. That's why I guess that many
> projects did not realize that they are already affected by the Y2038
> problem because of older kernel versions.

Which sounds like that warning needs backporting into the debian10
kernel and maybe cip. Not sure Isar is the best place, but a valid one
that could help.

Maybe mkfs could warn ... as well.

Henning

> >  
> > >
> > > Henning
> > >  
> > > >  
> > > > > Jan
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  
> > > > > > Henning
> > > > > >
> > > > > > Am Thu, 11 Feb 2021 11:07:52 +0300
> > > > > > schrieb Anton Mikanovich <amikan@ilbers.de>:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >  
> > > > > > > 01.02.2021 21:58, florian.bezdeka@siemens.com wrote:  
> > > > > > > > From: Florian Bezdeka <florian.bezdeka@siemens.com>
> > > > > > > >
> > > > > > > > Hi ISAR developers,
> > > > > > > >
> > > > > > > > this series is the summary of a nice journey through
> > > > > > > > the file system jungle regarding Y2038 problem. It all
> > > > > > > > began with a warning which is reported by kernels >=
> > > > > > > > 5.4:
> > > > > > > >
> > > > > > > > ext4 filesystem being mounted at (mountpoint) supports
> > > > > > > > timestamps until 2038 (0x7fffffff)
> > > > > > > >
> > > > > > > > I guess that most ISAR layers are using the Debian
> > > > > > > > kernels, so that warning was not recognized yet or at
> > > > > > > > least not very often.
> > > > > > > >
> > > > > > > > When reading this warning I was surprised. Shouldn't a
> > > > > > > > modern file system like ext4 be Y2038-safe? As it
> > > > > > > > turned out it depends on the inode size if an ext4 file
> > > > > > > > system is safe or not. So why was the inode size not
> > > > > > > > sufficient in my case?
> > > > > > > >
> > > > > > > > The inode size is chosen during file system generation
> > > > > > > > and depends on the size of the file system that is
> > > > > > > > going to be created. For details let's have a look at
> > > > > > > > `man mke2fs`:
> > > > > > > >
> > > > > > > > -T usage-type[,...]
> > > > > > > >      Specify how the filesystem is going to be used, so
> > > > > > > > that mke2fs can choose optimal filesystem parameters
> > > > > > > > for that use. The usage types that are supported are
> > > > > > > > defined in the configuration file /etc/mke2fs.conf. The
> > > > > > > > user may specify one or more usage types using a comma
> > > > > > > > separated list.
> > > > > > > >
> > > > > > > >      If this option is is not specified, mke2fs will
> > > > > > > > pick a single default usage type based on the size of
> > > > > > > > the filesystem to be created. If the filesystem size is
> > > > > > > > less than 3 megabytes, mke2fs will use the filesystem
> > > > > > > > type floppy. If the filesystem size is greater than or
> > > > > > > > equal to 3 but less than 512 megabytes, mke2fs(8) will
> > > > > > > > use the filesystem type small.
> > > > > > > >
> > > > > > > > The relevant parts from /etc/mke2fs.conf:
> > > > > > > > [fs_types]
> > > > > > > > ...
> > > > > > > >          small = {
> > > > > > > >                  blocksize = 1024
> > > > > > > >                  inode_size = 128
> > > > > > > >                  inode_ratio = 4096
> > > > > > > >          }
> > > > > > > > ...
> > > > > > > >
> > > > > > > > So whenever you create an ext4 file system with less
> > > > > > > > than 512MB in size you will end up with 128 byte inodes
> > > > > > > > and your file system is not Y2038-safe.
> > > > > > > >
> > > > > > > > The ISAR part:
> > > > > > > > ext4 may often be used in combination with the
> > > > > > > > expand-on-first-boot recipe / feature. So whenever
> > > > > > > > creating a small partition (e.g. inside a wic file) and
> > > > > > > > extending it later may result in a Y2038 affected ext4
> > > > > > > > file system.
> > > > > > > >
> > > > > > > > That is exactly what happened to me and I would like to
> > > > > > > > make sure that all other ISAR users are aware of this
> > > > > > > > situation.
> > > > > > > >
> > > > > > > > Valid workarounds found so far:
> > > > > > > >   - Tell wic that an partition will grow:
> > > > > > > >     Add `--mkfs-extraopts "-T ext4"` to your wic
> > > > > > > > partition definition
> > > > > > > >   - Set the inode size to 256 (for small ext4
> > > > > > > > partitions) Add `--mkfs-extraopts "-I 256"` to your wic
> > > > > > > > partition definition
> > > > > > > >
> > > > > > > > The upstream part:
> > > > > > > > None of the following patches has been sent to any
> > > > > > > > upstream (OE) mailing lists yet but hopefully that will
> > > > > > > > happen soon. So far: Any comments welcome!
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Florian
> > > > > > > >
> > > > > > > > Florian Bezdeka (2):
> > > > > > > >    wic-img: Forward warnings from wic to bitbake
> > > > > > > >    wic: Warn if an ext filesystem affected by the Y2038
> > > > > > > > problem is used
> > > > > > > >
> > > > > > > >   meta/classes/wic-img.bbclass | 20 ++++++++++++++-----
> > > > > > > >   scripts/lib/wic/partition.py | 38
> > > > > > > > ++++++++++++++++++++++++++++++++++++ 2 files changed, 53
> > > > > > > > insertions(+), 5 deletions(-)  
> > > > > > > Applied to next, thanks.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >  
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >  
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >  
> > > >  
> > >  
> >  
>