[PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Henning Schild]

Diff to v5:
- keep fallback path for debian stretch simply using old code and rdeps

Original Authors log:

Diff to v1:
- Rewording on patch message ("root partition" --> "last partition",
  added note to systemd-dependency and its impact)
- Added systemd-version dependency
- Removed runtime if case that allowed for non-systemd systems

Diff to v2:
- Version bump to v1.2
- Removed e2fsprogs as dependency

Diff to v3:
- Omit the explicit call to systemd-growfs if a fstab option has been
  found
- Be able to deal with both mounted and unmounted mountpoints

Diff to v4:
- Mount the last partition unconditionally to a tmp dir and let
  systemd-growfs operate there.

Tobias Schmidl (1):
  expand-on-first-boot: Switch from resize2fs to systemd-growfs

 ...oot_1.1.bb => expand-on-first-boot_1.2.bb} |  3 +-
 .../files/expand-last-partition.sh            | 31 ++++++++++++++++---
 2 files changed, 28 insertions(+), 6 deletions(-)
 rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (88%)

-- 
2.37.4

[PATCH v6 1/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Henning Schild]

From: Tobias Schmidl <tobiasschmidl@siemens.com>

We want to be more versatile in our approach of resizing the last
partition. Therefore we switch from resize2fs to systemd-growfs.

This allows for ext4, btrfs, xfs, and dm-crypt partitions to be resized.

Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 ...oot_1.1.bb => expand-on-first-boot_1.2.bb} |  3 +-
 .../files/expand-last-partition.sh            | 31 ++++++++++++++++---
 2 files changed, 28 insertions(+), 6 deletions(-)
 rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb => expand-on-first-boot_1.2.bb} (88%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
similarity index 88%
rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
index 1703a648ca27..8d11aef00927 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -1,13 +1,14 @@
 # Resize last partition to full medium size on fist boot
 #
 # This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
 #
 # SPDX-License-Identifier: MIT
 
 inherit dpkg-raw
 
 DESCRIPTION = "This service grows the last partition to the full medium during first boot"
+MAINTAINER = "isar-users <isar-users@googlegroups.com>"
 
 DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk, util-linux"
 
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 759ff8b61ccd..046a1394d213 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -3,7 +3,7 @@
 # Resize last partition to full medium size
 #
 # This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2022
 #
 # SPDX-License-Identifier: MIT
 
@@ -57,8 +57,29 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
 # Inform the kernel about the partitioning change
 partx -u "${LAST_PART}"
 
-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
+# this is for debian stretch or systemd < 236
+if [ ! -x /lib/systemd/systemd-growfs ]; then
+	# Do not fail resize2fs if no mtab entry is found, e.g.,
+	# when using systemd mount units.
+	export EXT2FS_NO_MTAB_OK=1
 
-resize2fs "${LAST_PART}"
+	resize2fs "${LAST_PART}"
+	exit 0
+if
+
+if grep -q x-systemd.growfs /etc/fstab; then
+	echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
+	exit 0
+fi
+
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
+if [ ! -d "${MOUNT_POINT}" ]; then
+	echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
+	exit 1
+fi
+
+mount "${LAST_PART}" "${MOUNT_POINT}"
+/lib/systemd/systemd-growfs "${MOUNT_POINT}"
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"
-- 
2.37.4

Re: [PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Henning Schild]

I am taking this over from Tobias. The last remaining point was keeping
the support for stretch, which i implemented here.

Henning

Am Tue, 25 Oct 2022 15:47:19 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Diff to v5:
> - keep fallback path for debian stretch simply using old code and
> rdeps
> 
> Original Authors log:
> 
> Diff to v1:
> - Rewording on patch message ("root partition" --> "last partition",
>   added note to systemd-dependency and its impact)
> - Added systemd-version dependency
> - Removed runtime if case that allowed for non-systemd systems
> 
> Diff to v2:
> - Version bump to v1.2
> - Removed e2fsprogs as dependency
> 
> Diff to v3:
> - Omit the explicit call to systemd-growfs if a fstab option has been
>   found
> - Be able to deal with both mounted and unmounted mountpoints
> 
> Diff to v4:
> - Mount the last partition unconditionally to a tmp dir and let
>   systemd-growfs operate there.
> 
> Tobias Schmidl (1):
>   expand-on-first-boot: Switch from resize2fs to systemd-growfs
> 
>  ...oot_1.1.bb => expand-on-first-boot_1.2.bb} |  3 +-
>  .../files/expand-last-partition.sh            | 31
> ++++++++++++++++--- 2 files changed, 28 insertions(+), 6 deletions(-)
>  rename
> meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb
> => expand-on-first-boot_1.2.bb} (88%)
> 

Re: [PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Bezdeka, Florian]

On Tue, 2022-10-25 at 15:49 +0200, Henning Schild wrote:
> I am taking this over from Tobias. The last remaining point was keeping
> the support for stretch, which i implemented here.

There were complains on the list that we have a possible race-window
here. A /data partition was unmounted after the resize took place.

That is likely not directly related to this patch, as this is "not in
yet" but I assume we still would have the same race.

See [1].

[1] https://groups.google.com/g/isar-users/c/o-iZbN7hPRw

> 
> Henning
> 
> Am Tue, 25 Oct 2022 15:47:19 +0200
> schrieb Henning Schild <henning.schild@siemens.com>:
> 
> > Diff to v5:
> > - keep fallback path for debian stretch simply using old code and
> > rdeps
> > 
> > Original Authors log:
> > 
> > Diff to v1:
> > - Rewording on patch message ("root partition" --> "last partition",
> >   added note to systemd-dependency and its impact)
> > - Added systemd-version dependency
> > - Removed runtime if case that allowed for non-systemd systems
> > 
> > Diff to v2:
> > - Version bump to v1.2
> > - Removed e2fsprogs as dependency
> > 
> > Diff to v3:
> > - Omit the explicit call to systemd-growfs if a fstab option has been
> >   found
> > - Be able to deal with both mounted and unmounted mountpoints
> > 
> > Diff to v4:
> > - Mount the last partition unconditionally to a tmp dir and let
> >   systemd-growfs operate there.
> > 
> > Tobias Schmidl (1):
> >   expand-on-first-boot: Switch from resize2fs to systemd-growfs
> > 
> >  ...oot_1.1.bb => expand-on-first-boot_1.2.bb} |  3 +-
> >  .../files/expand-last-partition.sh            | 31
> > ++++++++++++++++--- 2 files changed, 28 insertions(+), 6 deletions(-)
> >  rename
> > meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb
> > => expand-on-first-boot_1.2.bb} (88%)
> > 
> 

Re: [PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Henning Schild]

Am Tue, 25 Oct 2022 15:55:00 +0200
schrieb "Bezdeka, Florian (T CED SES-DE)" <florian.bezdeka@siemens.com>:

> On Tue, 2022-10-25 at 15:49 +0200, Henning Schild wrote:
> > I am taking this over from Tobias. The last remaining point was
> > keeping the support for stretch, which i implemented here.  
> 
> There were complains on the list that we have a possible race-window
> here. A /data partition was unmounted after the resize took place.
> 
> That is likely not directly related to this patch, as this is "not in
> yet" but I assume we still would have the same race.
> 
> See [1].

I would hope that the race will be gone when doing things with systemd,
but i do not know. In the stretch case it might be still in, but i
really do not care ... we should drop stretch from Isar or maintain it
on the best effort basis suggested here. Make sure it still works ...
but not much more.

I have read that thread and pointed out there that systemd resize will
be coming, and that people can try it already to see if it solves their
problem. Will take Raphael in Cc.

Raphael please have a look at the following patch, it might just solve
your resize race.

https://patchwork.isar-build.org/project/isar/patch/20221025134720.22571-2-henning.schild@siemens.com/

Henning

> 
> [1] https://groups.google.com/g/isar-users/c/o-iZbN7hPRw
> 
> > 
> > Henning
> > 
> > Am Tue, 25 Oct 2022 15:47:19 +0200
> > schrieb Henning Schild <henning.schild@siemens.com>:
> >   
> > > Diff to v5:
> > > - keep fallback path for debian stretch simply using old code and
> > > rdeps
> > > 
> > > Original Authors log:
> > > 
> > > Diff to v1:
> > > - Rewording on patch message ("root partition" --> "last
> > > partition", added note to systemd-dependency and its impact)
> > > - Added systemd-version dependency
> > > - Removed runtime if case that allowed for non-systemd systems
> > > 
> > > Diff to v2:
> > > - Version bump to v1.2
> > > - Removed e2fsprogs as dependency
> > > 
> > > Diff to v3:
> > > - Omit the explicit call to systemd-growfs if a fstab option has
> > > been found
> > > - Be able to deal with both mounted and unmounted mountpoints
> > > 
> > > Diff to v4:
> > > - Mount the last partition unconditionally to a tmp dir and let
> > >   systemd-growfs operate there.
> > > 
> > > Tobias Schmidl (1):
> > >   expand-on-first-boot: Switch from resize2fs to systemd-growfs
> > > 
> > >  ...oot_1.1.bb => expand-on-first-boot_1.2.bb} |  3 +-
> > >  .../files/expand-last-partition.sh            | 31
> > > ++++++++++++++++--- 2 files changed, 28 insertions(+), 6
> > > deletions(-) rename
> > > meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb
> > > => expand-on-first-boot_1.2.bb} (88%)  
> > >   
> >   
> 

Re: [PATCH v6 1/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Henning Schild]

Am Tue, 25 Oct 2022 15:47:20 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> From: Tobias Schmidl <tobiasschmidl@siemens.com>
> 
> We want to be more versatile in our approach of resizing the last
> partition. Therefore we switch from resize2fs to systemd-growfs.
> 
> This allows for ext4, btrfs, xfs, and dm-crypt partitions to be
> resized.
> 
> Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  ...oot_1.1.bb => expand-on-first-boot_1.2.bb} |  3 +-
>  .../files/expand-last-partition.sh            | 31
> ++++++++++++++++--- 2 files changed, 28 insertions(+), 6 deletions(-)
>  rename
> meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.1.bb
> => expand-on-first-boot_1.2.bb} (88%)
> 
> diff --git
> a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
> b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
> similarity index 88% rename from
> meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
> rename to
> meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
> index 1703a648ca27..8d11aef00927 100644 ---
> a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.1.bb
> +++
> b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
> @@ -1,13 +1,14 @@ # Resize last partition to full medium size on fist
> boot # # This software is a part of ISAR. -# Copyright (c) Siemens
> AG, 2018 +# Copyright (c) Siemens AG, 2018-2022 #
>  # SPDX-License-Identifier: MIT
>  
>  inherit dpkg-raw
>  
>  DESCRIPTION = "This service grows the last partition to the full
> medium during first boot" +MAINTAINER = "isar-users
> <isar-users@googlegroups.com>" 
>  DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs,
> fdisk, util-linux" 
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 759ff8b61ccd..046a1394d213 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -3,7 +3,7 @@ # Resize last partition to full medium size # # This
> software is a part of ISAR. -# Copyright (c) Siemens AG, 2018
> +# Copyright (c) Siemens AG, 2018-2022
>  #
>  # SPDX-License-Identifier: MIT
>  
> @@ -57,8 +57,29 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
>  # Inform the kernel about the partitioning change
>  partx -u "${LAST_PART}"
>  
> -# Do not fail resize2fs if no mtab entry is found, e.g.,
> -# when using systemd mount units.
> -export EXT2FS_NO_MTAB_OK=1
> +# this is for debian stretch or systemd < 236
> +if [ ! -x /lib/systemd/systemd-growfs ]; then
> +	# Do not fail resize2fs if no mtab entry is found, e.g.,
> +	# when using systemd mount units.
> +	export EXT2FS_NO_MTAB_OK=1
>  
> -resize2fs "${LAST_PART}"
> +	resize2fs "${LAST_PART}"
> +	exit 0
> +if
> +
> +if grep -q x-systemd.growfs /etc/fstab; then
> +	echo "Found x-systemd.growfs option in /etc/fstab, won't
> call it explicitly." >&2
> +	exit 0
> +fi

syntax error ... stay tuned ...

> +
> +# mount $LAST_PART out of tree, so we won't conflict with other
> mounts +MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
> +if [ ! -d "${MOUNT_POINT}" ]; then
> +	echo "Cannot create temporary mount point ${MOUNT_POINT}."
> >&2
> +	exit 1
> +fi
> +
> +mount "${LAST_PART}" "${MOUNT_POINT}"
> +/lib/systemd/systemd-growfs "${MOUNT_POINT}"
> +umount "${MOUNT_POINT}"
> +rmdir "${MOUNT_POINT}"

Re: [PATCH v6 0/1] expand-on-first-boot: Switch from resize2fs to systemd-growfs

[Raphael Lisicki]

On 25.10.22 16:31, Henning Schild wrote:
> 
> I have read that thread and pointed out there that systemd resize will
> be coming, and that people can try it already to see if it solves their
> problem. Will take Raphael in Cc.
> 
> Raphael please have a look at the following patch, it might just solve
> your resize race.
> 

I did not manage to test this yet, but I found quite some documentation
by systemd on the very same issue [1] and its shortcomings. I would be
moderately confident that systemd resize and systemd-udev will not clash.

The mentioned udevadm lock command is not available with debian bullseye.



[1] https://systemd.io/BLOCK_DEVICE_LOCKING/


best regards
raphael