[PATCH] expand-on-first-boot: support resizing a btrfs root

[PATCH] expand-on-first-boot: support resizing a btrfs root

[Joe MacDonald]

resize2fs fails if the build sets the root filesystem for an image to
btrfs though the partition is expanded properly. Since the rest of the
script works as intended, simply checking the last partition type with
lsblk and adding `btrfs resize` instaed of `resize2fs` in the appropriate
circumstance allows everything to function as expected.

Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
---
 .../files/expand-last-partition.sh              | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 7175dfd..23cfd86 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -36,8 +36,15 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
 # Inform the kernel about the partitioning change
 partx -u "${LAST_PART}"
 
-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
-
-resize2fs "${LAST_PART}"
+# Determine the filesystem type and perform the appropriate resize function
+ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
+case ${ROOTFS_TYPE} in
+   ext* )
+      # Do not fail resize2fs if no mtab entry is found, e.g.,
+      # when using systemd mount units.
+      export EXT2FS_NO_MTAB_OK=1
+      resize2fs "${LAST_PART}"
+      ;;
+   btrfs ) btrfs filesystem resize max / ;;
+   * ) exit 1 ;;
+esac
-- 
2.30.2

Re: [PATCH] expand-on-first-boot: support resizing a btrfs root

[Jan Kiszka]

On 22.10.21 20:01, Joe MacDonald wrote:
> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instaed of `resize2fs` in the appropriate

typo: instead

> circumstance allows everything to function as expected.
> 
> Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> ---
>  .../files/expand-last-partition.sh              | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..23cfd86 100755
> --- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,15 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
>  # Inform the kernel about the partitioning change
>  partx -u "${LAST_PART}"
>  
> -# Do not fail resize2fs if no mtab entry is found, e.g.,
> -# when using systemd mount units.
> -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize function
> +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> +case ${ROOTFS_TYPE} in
> +   ext* )
> +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> +      # when using systemd mount units.
> +      export EXT2FS_NO_MTAB_OK=1
> +      resize2fs "${LAST_PART}"
> +      ;;
> +   btrfs ) btrfs filesystem resize max / ;;

Could we reformat this to use multiple line, like the block above?

> +   * ) exit 1 ;;
> +esac
> 

Good extension!

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH] expand-on-first-boot: support resizing a btrfs root

[Joe MacDonald]

[-- Attachment #1: Type: text/plain, Size: 2380 bytes --]

[Re: [PATCH] expand-on-first-boot: support resizing a btrfs root] On 21.10.22 (Fri 20:19) Jan Kiszka wrote:

> On 22.10.21 20:01, Joe MacDonald wrote:
> > resize2fs fails if the build sets the root filesystem for an image to
> > btrfs though the partition is expanded properly. Since the rest of the
> > script works as intended, simply checking the last partition type with
> > lsblk and adding `btrfs resize` instaed of `resize2fs` in the appropriate
> 
> typo: instead

Oop, fixed.  Thanks.

> > circumstance allows everything to function as expected.
> > 
> > Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> > ---
> >  .../files/expand-last-partition.sh              | 17 ++++++++++++-----
> >  1 file changed, 12 insertions(+), 5 deletions(-)
> > 
> > diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > index 7175dfd..23cfd86 100755
> > --- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > +++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > @@ -36,8 +36,15 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
> >  # Inform the kernel about the partitioning change
> >  partx -u "${LAST_PART}"
> >  
> > -# Do not fail resize2fs if no mtab entry is found, e.g.,
> > -# when using systemd mount units.
> > -export EXT2FS_NO_MTAB_OK=1
> > -
> > -resize2fs "${LAST_PART}"
> > +# Determine the filesystem type and perform the appropriate resize function
> > +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> > +case ${ROOTFS_TYPE} in
> > +   ext* )
> > +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> > +      # when using systemd mount units.
> > +      export EXT2FS_NO_MTAB_OK=1
> > +      resize2fs "${LAST_PART}"
> > +      ;;
> > +   btrfs ) btrfs filesystem resize max / ;;
> 
> Could we reformat this to use multiple line, like the block above?

Yes, good suggestion.  v2 also includes an informational message on the
fall-through condition rather than silently exiting with a failure.

-Joe.

> 
> > +   * ) exit 1 ;;
> > +esac
> > 
> 
> Good extension!
> 
> Jan
> 
> -- 
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux

-- 
-Joe MacDonald.
Linux Architect | Mentor® A Siemens Business
:wq

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Joe MacDonald]

resize2fs fails if the build sets the root filesystem for an image to
btrfs though the partition is expanded properly. Since the rest of the
script works as intended, simply checking the last partition type with
lsblk and adding `btrfs resize` instead of `resize2fs` in the appropriate
circumstance allows everything to function as expected.

Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
---
 .../files/expand-last-partition.sh            | 22 ++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 7175dfd..6975d9c 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
 # Inform the kernel about the partitioning change
 partx -u "${LAST_PART}"
 
-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
-
-resize2fs "${LAST_PART}"
+# Determine the filesystem type and perform the appropriate resize function
+ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
+case ${ROOTFS_TYPE} in
+   ext* )
+      # Do not fail resize2fs if no mtab entry is found, e.g.,
+      # when using systemd mount units.
+      export EXT2FS_NO_MTAB_OK=1
+      resize2fs "${LAST_PART}"
+      ;;
+   btrfs )
+      btrfs filesystem resize max /
+      ;;
+   * )
+      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize performed"
+      exit 1
+      ;;
+esac
-- 
2.30.2

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Henning Schild]

Yes, good addition! But it would be even better if we had it running in
CI. We could switch one target over to btrfs by providing an example
wks file, one that boots an image in qemu.
That way we would have a btrfs example in the tree and get confident
that wic will do things as i would hope for.

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_macdonald@mentor.com>:

> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instead of `resize2fs` in the
> appropriate circumstance allows everything to function as expected.
> 
> Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> ---
>  .../files/expand-last-partition.sh            | 22
> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..6975d9c 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> the kernel about the partitioning change partx -u "${LAST_PART}" -#
> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize
> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> +case ${ROOTFS_TYPE} in
> +   ext* )
> +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> +      # when using systemd mount units.
> +      export EXT2FS_NO_MTAB_OK=1
> +      resize2fs "${LAST_PART}"
> +      ;;
> +   btrfs )
> +      btrfs filesystem resize max /

Need to do something to "DEBIAN_DEPENDS", maybe "e2fsprogs" ->
"e2fsprogs | btrfs-progs".

> +      ;;
> +   * )
> +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize
> performed"
> +      exit 1
> +      ;;
> +esac

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Henning Schild]

Am Mon, 25 Oct 2021 10:10:54 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Yes, good addition! But it would be even better if we had it running
> in CI. We could switch one target over to btrfs by providing an
> example wks file, one that boots an image in qemu.
> That way we would have a btrfs example in the tree and get confident
> that wic will do things as i would hope for.
> 
> Am Fri, 22 Oct 2021 15:50:47 -0400
> schrieb Joe MacDonald <joe_macdonald@mentor.com>:
> 
> > resize2fs fails if the build sets the root filesystem for an image
> > to btrfs though the partition is expanded properly. Since the rest
> > of the script works as intended, simply checking the last partition
> > type with lsblk and adding `btrfs resize` instead of `resize2fs` in
> > the appropriate circumstance allows everything to function as
> > expected.
> > 
> > Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> > ---
> >  .../files/expand-last-partition.sh            | 22
> > ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
> > 
> > diff --git
> > a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > index 7175dfd..6975d9c 100755 ---
> > a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > +++
> > b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> > @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> > the kernel about the partitioning change partx -u "${LAST_PART}" -#
> > Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> > systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> > -
> > -resize2fs "${LAST_PART}"
> > +# Determine the filesystem type and perform the appropriate resize
> > function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> > +case ${ROOTFS_TYPE} in
> > +   ext* )
> > +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> > +      # when using systemd mount units.
> > +      export EXT2FS_NO_MTAB_OK=1
> > +      resize2fs "${LAST_PART}"
> > +      ;;
> > +   btrfs )
> > +      btrfs filesystem resize max /  
> 
> Need to do something to "DEBIAN_DEPENDS", maybe "e2fsprogs" ->
> "e2fsprogs | btrfs-progs".

It is in fact more complicated than that. The two "progs" should not
both be depended on, because we would install too much stuff. We can
not use "|" because we will not know which one we get. Say one was
installed for any other reason, the one we need for rootfs might not
get pulled in.
Recommends would be better, but isar will not pull them. (still would
be an idea to be "correct")

We could also spit out two packages "expand-on-first-boot-ext|btrfs"
but that does not scale well for the next fs to cover.

So i would say "Recommends: e2fsprogs, btrfs-progs" and make that
script "command -v" and cry. Maybe keeping e2fsprogs in a first patch,
and removing it from Depends in a second interface-breaking patch.

regards,
Henning

> > +      ;;
> > +   * )
> > +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize
> > performed"
> > +      exit 1
> > +      ;;
> > +esac  
> 

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Henning Schild]

Hey Joe,

will you follow up with a v3? We are having another patch for
expand-on-first-boot coming up and discussions on how we could maybe
eventually switch to systemd-repart.

https://groups.google.com/g/isar-users/c/p5QADbRC3EA

Henning

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_macdonald@mentor.com>:

> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instead of `resize2fs` in the
> appropriate circumstance allows everything to function as expected.
> 
> Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> ---
>  .../files/expand-last-partition.sh            | 22
> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..6975d9c 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> the kernel about the partitioning change partx -u "${LAST_PART}" -#
> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize
> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> +case ${ROOTFS_TYPE} in
> +   ext* )
> +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> +      # when using systemd mount units.
> +      export EXT2FS_NO_MTAB_OK=1
> +      resize2fs "${LAST_PART}"
> +      ;;
> +   btrfs )
> +      btrfs filesystem resize max /
> +      ;;
> +   * )
> +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize
> performed"
> +      exit 1
> +      ;;
> +esac

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Jan Kiszka]

On 22.04.22 09:57, Henning Schild wrote:
> Hey Joe,
> 
> will you follow up with a v3? We are having another patch for
> expand-on-first-boot coming up and discussions on how we could maybe
> eventually switch to systemd-repart.

systemd-growfs. Only this would be relevant for adding more fs types
like here. repart has nothing to do with that.

Jan

> 
> https://groups.google.com/g/isar-users/c/p5QADbRC3EA
> 
> Henning
> 
> Am Fri, 22 Oct 2021 15:50:47 -0400
> schrieb Joe MacDonald <joe_macdonald@mentor.com>:
> 
>> resize2fs fails if the build sets the root filesystem for an image to
>> btrfs though the partition is expanded properly. Since the rest of the
>> script works as intended, simply checking the last partition type with
>> lsblk and adding `btrfs resize` instead of `resize2fs` in the
>> appropriate circumstance allows everything to function as expected.
>>
>> Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
>> ---
>>  .../files/expand-last-partition.sh            | 22
>> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
>>
>> diff --git
>> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
>> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
>> index 7175dfd..6975d9c 100755 ---
>> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
>> +++
>> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
>> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
>> the kernel about the partitioning change partx -u "${LAST_PART}" -#
>> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
>> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
>> -
>> -resize2fs "${LAST_PART}"
>> +# Determine the filesystem type and perform the appropriate resize
>> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
>> +case ${ROOTFS_TYPE} in
>> +   ext* )
>> +      # Do not fail resize2fs if no mtab entry is found, e.g.,
>> +      # when using systemd mount units.
>> +      export EXT2FS_NO_MTAB_OK=1
>> +      resize2fs "${LAST_PART}"
>> +      ;;
>> +   btrfs )
>> +      btrfs filesystem resize max /
>> +      ;;
>> +   * )
>> +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize
>> performed"
>> +      exit 1
>> +      ;;
>> +esac
> 

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Henning Schild]

It all looks like we want to revert the systemd-growfs patch and go
back to using plain tools for each partition type. So i looked at this
again and will likely propose another round, keeping the author but
changing it quite a bit.

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_macdonald@mentor.com>:

> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instead of `resize2fs` in the
> appropriate circumstance allows everything to function as expected.

It it not about "rootfs" it is about the last partition.

> Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> ---
>  .../files/expand-last-partition.sh            | 22
> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..6975d9c 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> the kernel about the partitioning change partx -u "${LAST_PART}" -#
> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize
> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> +case ${ROOTFS_TYPE} in

not rootfs but last partition

> +   ext* )

indentation is inconsistent with the rest of the script

> +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> +      # when using systemd mount units.
> +      export EXT2FS_NO_MTAB_OK=1
> +      resize2fs "${LAST_PART}"
> +      ;;
> +   btrfs )
> +      btrfs filesystem resize max /

Here it is again not about root. We will need the temporary mount bits
from the systemd-growfs patch and recycle them here. Because btrfs
wants to be mounted and we happen to know how that works from the other
patch.

Henning

> +      ;;
> +   * )
> +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize
> performed"
> +      exit 1
> +      ;;
> +esac

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Roberto A. Foglietta]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

On Fri, 9 Dec 2022 at 16:40, Henning Schild <henning.schild@siemens.com> wrote:
>
> It all looks like we want to revert the systemd-growfs patch and go
> back to using plain tools for each partition type. So i looked at this
> again and will likely propose another round, keeping the author but
> changing it quite a bit.
>
> Am Fri, 22 Oct 2021 15:50:47 -0400
> schrieb Joe MacDonald <joe_macdonald@mentor.com>:
>
> > resize2fs fails if the build sets the root filesystem for an image to
> > btrfs though the partition is expanded properly. Since the rest of the
> > script works as intended, simply checking the last partition type with
> > lsblk and adding `btrfs resize` instead of `resize2fs` in the
> > appropriate circumstance allows everything to function as expected.
>
> It it not about "rootfs" it is about the last partition.

This is the code you need to integrate btrfs filesystem

case $(lsblk -fno FSTYPE "${LAST_PART}") in
        ext4)   resize2fs "${LAST_PART}"
                ;;
        btrfs)  mkdir -p /tmp/btrfs
                mount "${LAST_PART}" /tmp/btrfs
                btrfs filesystem resize max /tmp/btrfs
                umount /tmp/btrfs && rmdir /tmp/btrfs
                ;;
esac

In attachment the patch against the current next branch.
It has been sent by elastic mail but went in moderation, as usual.

Best regards, R-

[-- Attachment #2: 0001-In-expand-last-partition-script-btrfs-support-added.patch --]
[-- Type: text/x-patch, Size: 3052 bytes --]

From 77c85b611d3da5712b8c883e7f97af72201e11b7 Mon Sep 17 00:00:00 2001
From: "Roberto A. Foglietta" <roberto.foglietta@gmail.com>
Date: Sat, 10 Dec 2022 04:13:42 +0100
Subject: [PATCH] In expand last partition script btrfs support added

improvement, expand-on-first-boot: support for btrfs added

Signed-off-by: Roberto A. Foglietta <roberto.foglietta@gmail.com>
---
 .../expand-on-first-boot_1.2.bb               |  2 +-
 .../files/expand-last-partition.sh            | 37 ++++++-------------
 2 files changed, 13 insertions(+), 26 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
index 0996000..fe7b72b 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -10,7 +10,7 @@ inherit dpkg-raw
 DESCRIPTION = "This service grows the last partition to the full medium during first boot"
 MAINTAINER = "isar-users <isar-users@googlegroups.com>"
 
-DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux"
+DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux, btrfs-progs"
 
 SRC_URI = " \
     file://expand-on-first-boot.service \
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 57055cc..acfef8a 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -57,29 +57,16 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
 # Inform the kernel about the partitioning change
 partx -u "${LAST_PART}"
 
-# this is for debian stretch or systemd < 236
-if [ ! -x /lib/systemd/systemd-growfs ]; then
-	# Do not fail resize2fs if no mtab entry is found, e.g.,
-	# when using systemd mount units.
-	export EXT2FS_NO_MTAB_OK=1
+# Do not fail resize2fs if no mtab entry is found, e.g.,
+# when using systemd mount units.
+export EXT2FS_NO_MTAB_OK=1
 
-	resize2fs "${LAST_PART}"
-	exit 0
-fi
-
-if grep -q x-systemd.growfs /etc/fstab; then
-	echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
-	exit 0
-fi
-
-# mount $LAST_PART out of tree, so we won't conflict with other mounts
-MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
-if [ ! -d "${MOUNT_POINT}" ]; then
-	echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
-	exit 1
-fi
-
-mount "${LAST_PART}" "${MOUNT_POINT}"
-/lib/systemd/systemd-growfs "${MOUNT_POINT}"
-umount "${MOUNT_POINT}"
-rmdir "${MOUNT_POINT}"
+case $(lsblk -fno FSTYPE "${LAST_PART}") in
+	ext4) 	resize2fs "${LAST_PART}"
+		;;
+	btrfs) 	mkdir -p /tmp/btrfs
+		mount "${LAST_PART}" /tmp/btrfs
+		btrfs filesystem resize max /tmp/btrfs
+		umount /tmp/btrfs && rmdir /tmp/btrfs
+		;;
+esac
-- 
2.34.1

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Henning Schild]

Am Sat, 10 Dec 2022 04:28:48 +0100
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> On Fri, 9 Dec 2022 at 16:40, Henning Schild
> <henning.schild@siemens.com> wrote:
> >
> > It all looks like we want to revert the systemd-growfs patch and go
> > back to using plain tools for each partition type. So i looked at
> > this again and will likely propose another round, keeping the
> > author but changing it quite a bit.
> >
> > Am Fri, 22 Oct 2021 15:50:47 -0400
> > schrieb Joe MacDonald <joe_macdonald@mentor.com>:
> >  
> > > resize2fs fails if the build sets the root filesystem for an
> > > image to btrfs though the partition is expanded properly. Since
> > > the rest of the script works as intended, simply checking the
> > > last partition type with lsblk and adding `btrfs resize` instead
> > > of `resize2fs` in the appropriate circumstance allows everything
> > > to function as expected.  
> >
> > It it not about "rootfs" it is about the last partition.  
> 
> This is the code you need to integrate btrfs filesystem
> 
> case $(lsblk -fno FSTYPE "${LAST_PART}") in
>         ext4)   resize2fs "${LAST_PART}"
>                 ;;
>         btrfs)  mkdir -p /tmp/btrfs
>                 mount "${LAST_PART}" /tmp/btrfs
>                 btrfs filesystem resize max /tmp/btrfs
>                 umount /tmp/btrfs && rmdir /tmp/btrfs
>                 ;;
> esac

I have an updated version of Joes patch looking very similar but still
needing some more testing. Feel free to review that once it comes.

As for this one, it is a clumsy duplicate and i will not even review.

Henning

> In attachment the patch against the current next branch.
> It has been sent by elastic mail but went in moderation, as usual.
> 
> Best regards, R-

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Roberto A. Foglietta]

On Sat, 10 Dec 2022 at 17:29, Henning Schild <henning.schild@siemens.com> wrote:

> As for this one, it is a clumsy duplicate and I will not even review.

The ancestor of this my "clumsy duplicate" works decently since September 30th

https://groups.google.com/g/isar-users/c/dxVJqRvnWjw/m/Y5g5mbIPBQAJ

while the alternative based on systemd-resizefs shown to be not viable
but was hanging on because Stretch support which is going to be
dismissed because EOL since July 2020

Please Henning, play nicely ;-)

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Henning Schild]

Am Fri, 22 Oct 2021 15:50:47 -0400
schrieb Joe MacDonald <joe_macdonald@mentor.com>:

> resize2fs fails if the build sets the root filesystem for an image to
> btrfs though the partition is expanded properly. Since the rest of the
> script works as intended, simply checking the last partition type with
> lsblk and adding `btrfs resize` instead of `resize2fs` in the
> appropriate circumstance allows everything to function as expected.
> 
> Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> ---
>  .../files/expand-last-partition.sh            | 22
> ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 7175dfd..6975d9c 100755 ---
> a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++
> b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> the kernel about the partitioning change partx -u "${LAST_PART}" -#
> Do not fail resize2fs if no mtab entry is found, e.g., -# when using
> systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> -
> -resize2fs "${LAST_PART}"
> +# Determine the filesystem type and perform the appropriate resize
> function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})

Calling "lsblk" suffers from the same udev race we have seen with
systemd-growfs ... only very subtle where we get like "" ... no
filesystem.

It seems to help to mount here and findmnt the mntpoint, we need the
mounting for btrfs anyhow.

Also lsblk can return multiple filesystems, like luks or lvm layers
underneath. Where findmnt only returns the top.

Joe, i had to rewrite your patch considerably. So i think i will change
the authorship to myself. I would really like to give you the credits
but am afraid to give you blame ... this whole expand story seems never
ending ...
So the patch will mention you, but likely not as author.

Henning

> +case ${ROOTFS_TYPE} in
> +   ext* )
> +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> +      # when using systemd mount units.
> +      export EXT2FS_NO_MTAB_OK=1
> +      resize2fs "${LAST_PART}"
> +      ;;
> +   btrfs )
> +      btrfs filesystem resize max /
> +      ;;
> +   * )
> +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no resize
> performed"
> +      exit 1
> +      ;;
> +esac

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Roberto A. Foglietta]

On Mon, 12 Dec 2022 at 16:24, Henning Schild <henning.schild@siemens.com> wrote:

>
> Joe, i had to rewrite your patch considerably. So i think i will change
> the authorship to myself. I would really like to give you the credits
> but am afraid to give you blame ... this whole expand story seems never
> ending ...
> So the patch will mention you, but likely not as author.
>

Thank you in advance for the mention. I wish to point out to you the
fact that supporting the btrfs is completely another issue than
triggering the udev to link the device and frankly speaking I do not
see the reason to keep the two things together. So, my ancestor patch
and the following development correctly address the btrfs support but
inherit the problem of failing when "${LAST_PART}" == "" which is a
completely another story because it depends on udev. Also the code to
trigger udev in order to create the devices links has been presented
here from another project of mine. So, the mention makes sense.

As a mere recap, this story should never have started at all. Because
there were two separate problems, no one of them was my duty to solve.
Moreover, the team got stuck into the idea of supporting every
filesystem on every platform: wrong approach. First separate legacy
from development, second fix a problem at time, third follows a
continuous improvement and continuous delivery model.

The btrfs support was not mandatory but suggested for the future due
to its intrinsic characteristics and the performance improvements
delivered into kernel v6.1. Your mandatory problem was to deal with
udev on a specific hardware platform and this was confused with a ext4
resizing failure - sometimes fails sometimes not depending the udev
timings (as far as I can remember) - so the btrfs got into the scene
as a suggestion to discard a software failure in resizing. If both
fail the problem is somewhere, in fact.

> but am afraid to give you blame

The only blame you can give to me is not being in charge of this at no
any time but trying to help without even having a piece of hardware to
make a test. In fact, the udev tests/code I did - I did on my
Christmas present for my nephew!

Best regards, R-

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Moessbauer, Felix]

On Mon, 2022-12-12 at 16:24 +0100, Henning Schild wrote:
> Am Fri, 22 Oct 2021 15:50:47 -0400
> schrieb Joe MacDonald <joe_macdonald@mentor.com>:
> 
> > resize2fs fails if the build sets the root filesystem for an image
> > to
> > btrfs though the partition is expanded properly. Since the rest of
> > the
> > script works as intended, simply checking the last partition type
> > with
> > lsblk and adding `btrfs resize` instead of `resize2fs` in the
> > appropriate circumstance allows everything to function as expected.
> > 
> > Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> > ---
> >  .../files/expand-last-partition.sh            | 22
> > ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-
> > )
> > 
> > diff --git
> > a/meta/recipes-support/expand-on-first-boot/files/expand-last-
> > partition.sh
> > b/meta/recipes-support/expand-on-first-boot/files/expand-last-
> > partition.sh
> > index 7175dfd..6975d9c 100755 ---
> > a/meta/recipes-support/expand-on-first-boot/files/expand-last-
> > partition.sh
> > +++
> > b/meta/recipes-support/expand-on-first-boot/files/expand-last-
> > partition.sh
> > @@ -36,8 +36,20 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ # Inform
> > the kernel about the partitioning change partx -u "${LAST_PART}" -#
> > Do not fail resize2fs if no mtab entry is found, e.g., -# when
> > using
> > systemd mount units. -export EXT2FS_NO_MTAB_OK=1
> > -
> > -resize2fs "${LAST_PART}"
> > +# Determine the filesystem type and perform the appropriate resize
> > function +ROOTFS_TYPE=$(lsblk -fno FSTYPE ${ROOT_DEV})
> 
> Calling "lsblk" suffers from the same udev race we have seen with
> systemd-growfs ... only very subtle where we get like "" ... no
> filesystem.
> 
> It seems to help to mount here and findmnt the mntpoint, we need the
> mounting for btrfs anyhow.

One problem remains: Where to mount.
If we mount on /mnt this will not work on an RO rootfs out of the box.
We currently have this issue in CIP core.

We could mount below /tmp which should be writable, but I don't know
which side effects this as the tmpfs itself will be mounted / created
by systemd.

Felix

> 
> Also lsblk can return multiple filesystems, like luks or lvm layers
> underneath. Where findmnt only returns the top.
> 
> Joe, i had to rewrite your patch considerably. So i think i will
> change
> the authorship to myself. I would really like to give you the credits
> but am afraid to give you blame ... this whole expand story seems
> never
> ending ...
> So the patch will mention you, but likely not as author.
> 
> Henning
> 
> > +case ${ROOTFS_TYPE} in
> > +   ext* )
> > +      # Do not fail resize2fs if no mtab entry is found, e.g.,
> > +      # when using systemd mount units.
> > +      export EXT2FS_NO_MTAB_OK=1
> > +      resize2fs "${LAST_PART}"
> > +      ;;
> > +   btrfs )
> > +      btrfs filesystem resize max /
> > +      ;;
> > +   * )
> > +      echo "Unrecognized filesystem type ${ROOTFS_TYPE} - no
> > resize
> > performed"
> > +      exit 1
> > +      ;;
> > +esac
> 

Re: [PATCH v2] expand-on-first-boot: support resizing a btrfs root

[Roberto A. Foglietta]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

On Tue, 13 Dec 2022 at 03:10, Moessbauer, Felix
<felix.moessbauer@siemens.com> wrote:

>
> One problem remains: Where to mount.
> If we mount on /mnt this will not work on an RO rootfs out of the box.
> We currently have this issue in CIP core.
>
> We could mount below /tmp which should be writable, but I don't know
> which side effects this as the tmpfs itself will be mounted / created
> by systemd.
>

I have updated my patch to use /dev/shm but the path could be modified
setting a variable into the running environment.
The patch has been sent with git send-email using elastic smtp, so it
went into moderation. Here attached for evaluation.

Best regards, -R

[-- Attachment #2: v2-0001-In-expand-last-partition-script-btrfs-support-add.patch --]
[-- Type: text/x-patch, Size: 3448 bytes --]

From 07a98a77f5c16b3da6c1a5e9d4fb7608fa333179 Mon Sep 17 00:00:00 2001
From: "Roberto A. Foglietta" <roberto.foglietta@gmail.com>
Date: Sat, 10 Dec 2022 04:13:42 +0100
Subject: [PATCH v2] In expand last partition script btrfs support added

Improvement, expand-on-first-boot: support for btrfs added
The mount point for btrfs filesystem to resizes is under /dev/shm
by default but a variable defined in the running enviroment can
change the temporary directory path. The /dev/shm has been choosen
because in some system /tmp can be on a RO root filesystem and
/tmp not yet mounted with tmpfs. This requires the system has
/dev/shm configured into the kernel.

Signed-off-by: Roberto A. Foglietta <roberto.foglietta@gmail.com>
---
 .../expand-on-first-boot_1.2.bb               |  2 +-
 .../files/expand-last-partition.sh            | 38 +++++++------------
 2 files changed, 14 insertions(+), 26 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
index 0996000..fe7b72b 100644
--- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
+++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb
@@ -10,7 +10,7 @@ inherit dpkg-raw
 DESCRIPTION = "This service grows the last partition to the full medium during first boot"
 MAINTAINER = "isar-users <isar-users@googlegroups.com>"
 
-DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux"
+DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux, btrfs-progs"
 
 SRC_URI = " \
     file://expand-on-first-boot.service \
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 4d55645..e314f70 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -59,29 +59,17 @@ sfdisk -d "${BOOT_DEV}" 2>/dev/null | \
 # Inform the kernel about the partitioning change
 partx -u "${LAST_PART}"
 
-# this is for debian stretch or systemd < 236
-if [ ! -x /lib/systemd/systemd-growfs ]; then
-	# Do not fail resize2fs if no mtab entry is found, e.g.,
-	# when using systemd mount units.
-	export EXT2FS_NO_MTAB_OK=1
+# Do not fail resize2fs if no mtab entry is found, e.g.,
+# when using systemd mount units.
+export EXT2FS_NO_MTAB_OK=1
 
-	resize2fs "${LAST_PART}"
-	exit 0
-fi
-
-if grep -q x-systemd.growfs /etc/fstab; then
-	echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2
-	exit 0
-fi
-
-# mount $LAST_PART out of tree, so we won't conflict with other mounts
-MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX")
-if [ ! -d "${MOUNT_POINT}" ]; then
-	echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
-	exit 1
-fi
-
-mount "${LAST_PART}" "${MOUNT_POINT}"
-/lib/systemd/systemd-growfs "${MOUNT_POINT}"
-umount "${MOUNT_POINT}"
-rmdir "${MOUNT_POINT}"
+tmp=${BOOT_DEV_TMPDIR:-/dev/shm}
+case $(lsblk -fno FSTYPE "${LAST_PART}") in
+	ext4) 	resize2fs "${LAST_PART}"
+		;;
+	btrfs) 	mkdir -p $tmp/btrfs
+		mount "${LAST_PART}" $tmp/btrfs
+		btrfs filesystem resize max $tmp/btrfs
+		umount $tmp/btrfs && rmdir $tmp/btrfs
+		;;
+esac
-- 
2.34.1