Re: [PATCH v2] classes/rootfs: remove content of /dev, /run and vmlinux old

["MOESSBAUER, Felix" <felix.moessbauer@siemens.com>]

On Thu, 2024-05-02 at 17:16 +0200, Baurzhan Ismagulov wrote:
> On 2024-04-26 12:13, MOESSBAUER, Felix wrote:
> > On Fri, 2024-04-26 at 10:55 +0200, Baurzhan Ismagulov wrote:
> > > On 2024-04-25 09:13, 'Quirin Gylstorff' via isar-users wrote:
> > > > `/dev` and `/run` contain artifacts from the ISAR build clean
> > > > them
> > > > up for reproducability.
> > > 
> > > Suggest "Isar" and "reproducibility".
> > > 
> > > 
> > > > diff --git a/meta/classes/image.bbclass
> > > > b/meta/classes/image.bbclass
> > > > index 98741da0..4af0d448 100644
> > > > --- a/meta/classes/image.bbclass
> > > > +++ b/meta/classes/image.bbclass
> > > > @@ -427,12 +427,17 @@ do_rootfs_finalize() {
> > > >              rmdir --ignore-fail-on-non-empty
> > > > ${ROOTFSDIR}/base-apt
> > > >  
> > > >          mountpoint -q '${ROOTFSDIR}/dev' && \
> > > > -            umount -l ${ROOTFSDIR}/dev
> > > > +            umount -l -R ${ROOTFSDIR}/dev && \
> > > > +            rm -rf ${ROOTFSDIR}/dev/*
> > > > +
> > > >          mountpoint -q '${ROOTFSDIR}/proc' && \
> > > >              umount -l ${ROOTFSDIR}/proc
> > > >          mountpoint -q '${ROOTFSDIR}/sys' && \
> > > >              umount -l ${ROOTFSDIR}/sys
> > > >  
> > > > +        rm -rf ${ROOTFSDIR}/run/*
> > > > +        rm -f ${ROOTFSDIR}/vmlinuz.old
> > > > +
> > > 
> > > Could you provide the list of the specific files that you want to
> > > have removed?
> > > 
> > > Debian normally has /dev populated even though devtmpfs is
> > > mounted
> > > later on top
> > 
> > This is probably just an artifact of the bootstrapping, but not
> > written
> > in any debian policy. As debian uses systemd (and we anyways only
> > support this init system), we should follow the "man file-
> > hierarchy".
> > There a devtmpfs is always mounted on /dev. What's the point then
> > to
> > still keep the dirs and files there?
> > 
> > > of it. If the problem is that the entries have different
> > > timestamps
> > > on
> > > different builds, then the question is not limited to /dev but
> > > applies to any
> > > files created in postinstall scripts. I don't think the general
> > > approach should
> > > be to remove such files at the meta level.
> > 
> > The timestamps are already clamped by the rootfs postprocessing,
> > however this is a leak of host resources into the rootfs. Which
> > files
> > are under /dev depends on the devtmpfs implementation - which
> > depends
> > on the host kernel.
> > 
> > I propose to just clear these directories.
> 
> Thanks Felix for the details,
> 
> 1. Regarding /dev, I agree that it looks like an artifact of
> bootstrapping and
>    installation. However, I don't see that the contents are copied
> from
>    devtmpfs; could you please provide a sample command line and the
> list of
>    files to confirm the host leak?

I checked again and even if I add files / dirs to the hosts (or kas
containers) /dev, these do not show up in the bootstrapped rootfs. By
that, leaving the files should be fine from a reproducible-builds
perspective.

> 
>    After both debootstrap and d-i, I see the fixed list of console,
> fd, full,
>    null, ptmx, pts, random, shm, stderr, stdin, stdout, tty, urandom,
> zero. I
>    do think we can safely remove /dev/* for our use cases (although
>    file-hierarchy(7) says devtmpfs is only *usually* mounted), but
> I'd like to
>    document the reason for removal if we later find out that some
> software has
>    different expectations.
> 
>    We see different tools expecting at least /dev/null, /proc,
> /dev/pts and
>    /dev/shm and I also couldn't find any documentation on this. In
> the good
>    case, they fail (sometimes with cryptic messages), but often they
> crash or
>    hog the CPU (which is fun to debug from under e.g. CI + qemu).
> Some
>    indirectly related examples and discussion:
>   
> https://salsa.debian.org/installer-team/debootstrap/-/commit/a10e577675896b693c66ac77eace7ff76199da2c
>    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=571136
> 
> 2. For /run, I'd still like to see the list of the files, because I'd
> expect
>    Isar to leave it in a clean (= empty) state; maybe we have
> something that we
>    should clean up elsewhere in Isar.

For /run, the list also seems to be stable and it only contains
directories. However I don't really know where this is from. The
bootstrap is left with just /run/lock , /run/mount.

tar -tvf isar-image-base-debian-bookworm-qemuamd64.tar | grep "./run/"
lrwxrwxrwx root/root         0 2024-03-04 16:14 ./var/lock -> /run/lock
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/sendsigs.omit.d/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/log/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/user/
-rw-r--r-- root/root         0 2024-03-04 16:14 ./run/adduser
drwxrwxrwt root/root         0 2024-03-04 16:14 ./run/lock/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/lock/subsys/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/mount/
lrwxrwxrwx root/root         0 2024-03-04 16:14 ./run/shm -> /dev/shm
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/runit/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/runit/supervise/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/machines/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/ask-
password/
drwxr-xr-x 998/998           0 2024-03-04 16:14 ./run/systemd/netif/
drwxr-xr-x 998/998           0 2024-03-04 16:14
./run/systemd/netif/links/
drwxr-xr-x 998/998           0 2024-03-04 16:14
./run/systemd/netif/lldp/
drwxr-xr-x 998/998           0 2024-03-04 16:14
./run/systemd/netif/leases/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/seats/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/sessions/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/users/
drwxr-xr-x root/root         0 2024-03-04 16:14 ./run/systemd/shutdown/

Felix

> 
> 3. Regarding "vmlinux old", I assume it's about /vmlinuz.old (and I
> suggest to
>    update the commit title for better understanding -- or we can do
> that when
>    applying if it's ok for you) -- that should disappear by itself
> when we
>    multistrap the final rootfs directly from multiple apt repos. I'm
> fine
>    applying this part.
> 
> With kind regards,
> Baurzhan

-- 
Siemens AG, Technology
Linux Expert Center