Re: [PATCH 0/3] Detect systemd services ordering cycle

[Uladzimir Bely <ubely@ilbers.de>]

On Sat, 2024-08-31 at 10:45 +0200, Jan Kiszka wrote:
> On 21.08.24 06:49, Uladzimir Bely wrote:
> > On Fri, 2024-08-16 at 17:38 +0300, Uladzimir Bely wrote:
> > > Merging v3 of "expand-on-first-boot: Ensure that /tmp is
> > > writable"
> > > led to systemd services ordering cycle. Systemd randomly drops
> > > one
> > > of sycled services and sometimes it's "expand-on-first-boot" one.
> > > 
> > > This patchset reverts version 3 of the patch and returns to
> > > previous version.
> > > 
> > > Additionally, we extend CI so that qemu run tests could always
> > > detect
> > > any systemd services ordering cycles.
> > > 
> > > Clara Kowalsky (1):
> > >   expand-on-first-boot: Add /tmp to ConditionPathIsReadWrite
> > > 
> > > Uladzimir Bely (2):
> > >   Revert "expand-on-first-boot: Ensure that /tmp is writable"
> > >   testsuite: Detect systemd services ordering cycle
> > > 
> > >  .../expand-on-first-boot/files/expand-on-first-boot.service  | 3
> > > +--
> > >  testsuite/cibuilder.py                                       | 5
> > > +++++
> > >  2 files changed, 6 insertions(+), 2 deletions(-)
> > > 
> > > -- 
> > > 2.44.2
> > > 
> > 
> > Applied to next.
> > 
> 
> Still broken:
> 
> root@isar:~# journalctl -u expand-on-first-boot
> -- Boot 336ca3ef86c74a06ab8e11e40a321c84 --
> Jun 16 09:44:32 isar systemd[1]: expand-on-first-boot.service -
> Expand last partition was skipped because of an unmet condition check
> (ConditionPathIsReadWrite=/etc /tmp).
> root@isar:~# cat /proc/cmdline 
> BOOT_IMAGE=/vmlinuz-6.1.0-23-amd64 root=PARTLABEL=platform rw
> rootwait console=ttyS0,115200 console=tty0
> 
> Patch will follow.
> 
> Why didn't you test this?
> 

Hello.

The patchset was autotested in CI several times, and, according to VM
logs (for example, with bullseye/amd64 it previusly was always failing)
everything is OK:

```
[    8.433834] systemd[1]: Starting Expand last partition...
[    8.467179] systemd[1]: Condition check resulted in Rebuild Hardware
Database being skipped.
[    8.481797] systemd[1]: Condition check resulted in Platform
Persistent Storage Archival being skipped.
[    8.492728] random: systemd: uninitialized urandom read (16 bytes
read)
[    8.520001] systemd[1]: Starting Load/Save Random Seed...
[    8.581835] systemd[1]: Starting Apply Kernel Variables...
[    8.655061] systemd[1]: Starting Create System Users...
[    8.738683] systemd[1]: Mounted FUSE Control File System.
[    8.786544] systemd[1]: Mounted Kernel Configuration File System.
[    8.837270] systemd[1]: Finished Apply Kernel Variables.
[    8.934155] systemd[1]: Started Journal Service.
[    9.096109] random: crng init done
[    9.098673] random: 46 urandom warning(s) missed due to ratelimiting
[    9.151174] systemd-journald[195]: Received client request to flush
runtime journal.
[    9.175233] systemd-journald[195]: Creating journal file
/var/log/journal/2cada36492d746358164b901eda86292/system.journal on a
btrfs file system, and copy-on-write is enabled. This is likely to slow
down journal access substantially, please consider turning off the
copy-on-write file attribute on the journal directory, using chattr +C.
[   10.292661] BTRFS info (device sda2): resize device /dev/sda2 (devid
1) from 1305477120 to 1573912576
```

Now I looked to all logs and found for "bookworm/arm" the following:

```
[   24.219166] systemd[1]: expand-on-first-boot.service - Expand last
partition was skipped because of an unmet condition check
(ConditionPathIsReadWrite=/etc /tmp).
```

We probably should to extend checks in testsuite, while currenaly only
images using "sdimage-efi-sd" and "sdimage-efi-btrfs" only are tested. 

Probably, this depend on systemd version and bullseye and lower
versions are not affected.

> Jan
> 

-- 
Best regards,
Uladzimir.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/58d072e0ac650f6480aedf69071a12efc92ff8f0.camel%40ilbers.de.