[PATCH 0/1] fix on-first-boot for systemd >= 247

[PATCH 0/1] fix on-first-boot for systemd >= 247

[Felix Moessbauer]

This patch should fix the missing machine-id, as well as the first-boot target.
Please review carefully and test this on various layers.
Feedback is highly appreciated, especially for more complex images like SWU, ro root+overlay, no initrd, ubuntu and debian buster.

Snippet from journal on first boot of the image:

Jul 21 21:52:26 test-image systemd[1]: Detected architecture x86-64.
Jul 21 21:52:26 test-image systemd[1]: Detected first boot.
Jul 21 21:52:26 test-image systemd[1]: Set hostname to <test-image>.
Jul 21 21:52:26 test-image systemd[1]: Initializing machine ID from KVM UUID.
Jul 21 21:52:26 test-image systemd[1]: Failed to populate /etc with preset unit settings, ignoring: Read-only file system
Jul 21 21:52:26 test-image systemd[1]: Queued start job for default target Graphical Interface.

Snipped from second boot:

Jul 21 21:55:54 test-image systemd[1]: Detected architecture x86-64.
Jul 21 21:55:54 test-image systemd[1]: Set hostname to <test-image>.
Jul 21 21:55:54 test-image systemd[1]: Queued start job for default target Graphical Interface.

cat /proc/cmdline
initrd=\initrd.img-5.10.0-16-amd64 LABEL=Boot root=PARTUUID=5e078448-eeb1-4fc4-acd3-d3db9c93a430 rootwait console=tty0 console=ttyS0,115200

Best regards,
Felix

Felix Moessbauer (1):
  fix on-first-boot for systemd > 247

 meta/classes/image-postproc-extension.bbclass | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

-- 
2.30.2

[PATCH 1/1] fix on-first-boot for systemd >= 247

[Felix Moessbauer]

This patch fixes the bug that /etc/machine-id is never
generated when running from within an initrd with an ro rootfs.

In this case, the magic value "uninitialized" has to be used
instead of deleting the file.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/image-postproc-extension.bbclass | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 0c412c0d..98c697b2 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -56,8 +56,15 @@ image_postprocess_mark() {
 ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
 image_postprocess_machine_id() {
     # systemd(1) takes care of recreating the machine-id on first boot
+    # for systemd < v247, remove the file, else set to uninitialized
+    # (required if initramfs with ro root is used)
+    SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd )
+    if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
+        sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
+    else
+        echo "uninitialized" | sudo tee '${IMAGE_ROOTFS}/etc/machine-id'
+    fi
     sudo rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
-    sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
 }
 
 ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_sshd_key_regen"
-- 
2.30.2

Re: [PATCH 1/1] fix on-first-boot for systemd >= 247

[Henning Schild]

Am Fri, 22 Jul 2022 00:04:04 +0200
schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:

> This patch fixes the bug that /etc/machine-id is never
> generated when running from within an initrd with an ro rootfs.
> 
> In this case, the magic value "uninitialized" has to be used
> instead of deleting the file.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  meta/classes/image-postproc-extension.bbclass | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image-postproc-extension.bbclass
> b/meta/classes/image-postproc-extension.bbclass index
> 0c412c0d..98c697b2 100644 ---
> a/meta/classes/image-postproc-extension.bbclass +++
> b/meta/classes/image-postproc-extension.bbclass @@ -56,8 +56,15 @@
> image_postprocess_mark() { ROOTFS_POSTPROCESS_COMMAND =+
> "image_postprocess_machine_id" image_postprocess_machine_id() {
>      # systemd(1) takes care of recreating the machine-id on first
> boot
> +    # for systemd < v247, remove the file, else set to uninitialized
> +    # (required if initramfs with ro root is used)
> +    SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query
> --showformat='${source:Upstream-Version}' --show systemd )
> +    if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
> +        sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
> +    else
> +        echo "uninitialized" | sudo tee
> '${IMAGE_ROOTFS}/etc/machine-id'
> +    fi
>      sudo rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
> -    sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'

I would keep the rm and follow up with a conditional write. Also i
found that /etc/machine-id seems to be 444 at least at the end of the
day. If it already is at that point, the tee might fail. And it might
be that we need to 444 it when we create it.

Henning

>  }
>  
>  ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_sshd_key_regen"

RE: [PATCH 1/1] fix on-first-boot for systemd >= 247

[Moessbauer, Felix]

> -----Original Message-----
> From: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> Sent: Friday, July 22, 2022 9:34 AM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>
> Cc: isar-users@googlegroups.com; Schmidl, Tobias (SI EP MF GWA EA 4)
> <tobias.schmidl@siemens.com>; Kiszka, Jan (T CED) <jan.kiszka@siemens.com>
> Subject: Re: [PATCH 1/1] fix on-first-boot for systemd >= 247
> 
> Am Fri, 22 Jul 2022 00:04:04 +0200
> schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
> 
> > This patch fixes the bug that /etc/machine-id is never generated when
> > running from within an initrd with an ro rootfs.
> >
> > In this case, the magic value "uninitialized" has to be used instead
> > of deleting the file.
> >
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >  meta/classes/image-postproc-extension.bbclass | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/image-postproc-extension.bbclass
> > b/meta/classes/image-postproc-extension.bbclass index
> > 0c412c0d..98c697b2 100644 ---
> > a/meta/classes/image-postproc-extension.bbclass +++
> > b/meta/classes/image-postproc-extension.bbclass @@ -56,8 +56,15 @@
> > image_postprocess_mark() { ROOTFS_POSTPROCESS_COMMAND =+
> > "image_postprocess_machine_id" image_postprocess_machine_id() {
> >      # systemd(1) takes care of recreating the machine-id on first
> > boot
> > +    # for systemd < v247, remove the file, else set to uninitialized
> > +    # (required if initramfs with ro root is used)
> > +    SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query
> > --showformat='${source:Upstream-Version}' --show systemd )
> > +    if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
> > +        sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
> > +    else
> > +        echo "uninitialized" | sudo tee
> > '${IMAGE_ROOTFS}/etc/machine-id'
> > +    fi
> >      sudo rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
> > -    sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
> 
> I would keep the rm and follow up with a conditional write. Also i found that
> /etc/machine-id seems to be 444 at least at the end of the day. If it already is at
> that point, the tee might fail. And it might be that we need to 444 it when we
> create it.

I thought about that as well, but to keep the access rights (which are not 444 initially), I decided to not remove the file.

Felix

> 
> Henning
> 
> >  }
> >
> >  ROOTFS_POSTPROCESS_COMMAND =+
> "image_postprocess_sshd_key_regen"

RE: [PATCH 1/1] fix on-first-boot for systemd >= 247

[Moessbauer, Felix]

> > Am Fri, 22 Jul 2022 00:04:04 +0200
> > schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
> >
> > > This patch fixes the bug that /etc/machine-id is never generated
> > > when running from within an initrd with an ro rootfs.
> > >
> > > In this case, the magic value "uninitialized" has to be used instead
> > > of deleting the file.
> > >
> > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > > ---
> > >  meta/classes/image-postproc-extension.bbclass | 9 ++++++++-
> > >  1 file changed, 8 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/image-postproc-extension.bbclass
> > > b/meta/classes/image-postproc-extension.bbclass index
> > > 0c412c0d..98c697b2 100644 ---
> > > a/meta/classes/image-postproc-extension.bbclass +++
> > > b/meta/classes/image-postproc-extension.bbclass @@ -56,8 +56,15 @@
> > > image_postprocess_mark() { ROOTFS_POSTPROCESS_COMMAND =+
> > > "image_postprocess_machine_id" image_postprocess_machine_id() {
> > >      # systemd(1) takes care of recreating the machine-id on first
> > > boot
> > > +    # for systemd < v247, remove the file, else set to uninitialized
> > > +    # (required if initramfs with ro root is used)
> > > +    SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query
> > > --showformat='${source:Upstream-Version}' --show systemd )
> > > +    if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
> > > +        sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
> > > +    else
> > > +        echo "uninitialized" | sudo tee
> > > '${IMAGE_ROOTFS}/etc/machine-id'
> > > +    fi
> > >      sudo rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
> > > -    sudo rm -f '${IMAGE_ROOTFS}/etc/machine-id'
> >
> > I would keep the rm and follow up with a conditional write. Also i
> > found that /etc/machine-id seems to be 444 at least at the end of the
> > day. If it already is at that point, the tee might fail. And it might
> > be that we need to 444 it when we create it.
> 
> I thought about that as well, but to keep the access rights (which are not 444
> initially), I decided to not remove the file.

Just found out that systemd is not in all images we generate (e.g. the SDK).
This breaks the patch.
Will send out a v2 soon, with a detection if systemd is installed.

Felix

> 
> Felix
> 
> >
> > Henning
> >
> > >  }
> > >
> > >  ROOTFS_POSTPROCESS_COMMAND =+
> > "image_postprocess_sshd_key_regen"
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-
> users%2FAM9PR10MB4869C2A9537643BF92AB960589909%2540AM9PR10MB4
> 869.EURPRD10.PROD.OUTLOOK.COM&amp;data=05%7C01%7Cfelix.moessbau
> er%40siemens.com%7Cbff2e2353511402d995b08da6bbd768e%7C38ae3bcd957
> 94fd4addab42e1495d55a%7C1%7C0%7C637940758721837057%7CUnknown%7
> CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
> XVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=rSoTRCqgcsncXBFPUX27Nkpj
> T8rM0mr%2FH6l3sDDu1fY%3D&amp;reserved=0.

Re: [PATCH 1/1] fix on-first-boot for systemd >= 247

[Schmidl, Tobias]

Hi Felix,

Am Freitag, dem 22.07.2022 um 00:04 +0200 schrieb Felix Moessbauer:
> 
> This patch fixes the bug that /etc/machine-id is never
> generated when running from within an initrd with an ro rootfs.
> 
> In this case, the magic value "uninitialized" has to be used
> instead of deleting the file.
> 

Unfortunately, this doesn't seem to work on debian-buster (for
qemuamd64):

```
Jul 22 07:52:21 localhost systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
Jul 22 07:52:21 localhost systemd[1]: Booting up is supported only when:
Jul 22 07:52:21 localhost systemd[1]: 1) /etc/machine-id exists and is populated.
Jul 22 07:52:21 localhost systemd[1]: 2) /etc/machine-id exists and is empty.
```

Kind regards,

Tobias

Re: [PATCH 1/1] fix on-first-boot for systemd >= 247

[Henning Schild]

Am Fri, 22 Jul 2022 10:02:35 +0200
schrieb "Schmidl, Tobias (T CED SES-DE)" <tobiasschmidl@siemens.com>:

> Hi Felix,
> 
> Am Freitag, dem 22.07.2022 um 00:04 +0200 schrieb Felix Moessbauer:
> > 
> > This patch fixes the bug that /etc/machine-id is never
> > generated when running from within an initrd with an ro rootfs.
> > 
> > In this case, the magic value "uninitialized" has to be used
> > instead of deleting the file.
> >   
> 
> Unfortunately, this doesn't seem to work on debian-buster (for
> qemuamd64):
> 
> ```
> Jul 22 07:52:21 localhost systemd[1]: System cannot boot: Missing
> /etc/machine-id and /etc is mounted read-only. Jul 22 07:52:21
> localhost systemd[1]: Booting up is supported only when: Jul 22
> 07:52:21 localhost systemd[1]: 1) /etc/machine-id exists and is
> populated. Jul 22 07:52:21 localhost systemd[1]: 2) /etc/machine-id
> exists and is empty. ```

Sounds like one more >= cases to cover, and another number to find
for when it changed in systemd. Ideally the numbers would be the ones
where systemd really changed things, not the numbers we find in debian
releases ... who knows whether ubuntu uses the same ones, or what is in
some inbetween package feeds like backports.

Henning

> Kind regards,
> 
> Tobias

RE: [PATCH 1/1] fix on-first-boot for systemd >= 247

[Moessbauer, Felix]

> -----Original Message-----
> From: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> Sent: Friday, July 22, 2022 10:27 AM
> To: Schmidl, Tobias (T CED SES-DE) <tobiasschmidl@siemens.com>
> Cc: isar-users@googlegroups.com; Moessbauer, Felix (T CED SES-DE)
> <felix.moessbauer@siemens.com>; Kiszka, Jan (T CED)
> <jan.kiszka@siemens.com>; Schmidl, Tobias (SI EP MF GWA EA 4)
> <tobias.schmidl@siemens.com>
> Subject: Re: [PATCH 1/1] fix on-first-boot for systemd >= 247
> 
> Am Fri, 22 Jul 2022 10:02:35 +0200
> schrieb "Schmidl, Tobias (T CED SES-DE)" <tobiasschmidl@siemens.com>:
> 
> > Hi Felix,
> >
> > Am Freitag, dem 22.07.2022 um 00:04 +0200 schrieb Felix Moessbauer:
> > >
> > > This patch fixes the bug that /etc/machine-id is never generated
> > > when running from within an initrd with an ro rootfs.
> > >
> > > In this case, the magic value "uninitialized" has to be used instead
> > > of deleting the file.
> > >
> >
> > Unfortunately, this doesn't seem to work on debian-buster (for
> > qemuamd64):
> >
> > ```
> > Jul 22 07:52:21 localhost systemd[1]: System cannot boot: Missing
> > /etc/machine-id and /etc is mounted read-only. Jul 22 07:52:21
> > localhost systemd[1]: Booting up is supported only when: Jul 22
> > 07:52:21 localhost systemd[1]: 1) /etc/machine-id exists and is
> > populated. Jul 22 07:52:21 localhost systemd[1]: 2) /etc/machine-id
> > exists and is empty. ```
> 
> Sounds like one more >= cases to cover, and another number to find for when it
> changed in systemd.

I don't know if we will find a solution for buster as the bug only has been fixed for system >= 247.
For the technical reasoning, have a look here at [1].

> Ideally the numbers would be the ones where systemd
> really changed things, not the numbers we find in debian releases ... who knows
> whether ubuntu uses the same ones, or what is in some inbetween package
> feeds like backports.

The corresponding commit in systemd is a48627ef and I just checked which git tags contain that commit to get the version number.
So that should be fine.

PS: I just added the "right" Tobias and removed the other one.
Please apology for the confusion.

Felix

[1] https://github.com/systemd/systemd/pull/16939

> 
> Henning
> 
> > Kind regards,
> >
> > Tobias