[PATCH v7 0/6] expand-on-first-boot btrfs and CI testing

[PATCH v7 0/6] expand-on-first-boot btrfs and CI testing

[Henning Schild]

changes since v6:
 - make sure CI only expects resize output when booting wic

changes since v5:
 - drop dead code around mktemp subshell
 - revert cosmetic mktemp stuff back to v3
 - remove "Suggested-by" tag to hopefully improve signal to noise ratio
   in review process

changes since v4:
 - cosmetic fix in the way we call mktemp
 - add a patch that makes sure the service is run only once, also on
   error
 - bump package version after changes are done

changes since v3:
 - rebased on "expand-on-first-boot: switch back away from systemd-growfs"
 - implement btrfs resizing and CI testing it
 - put the padding wks entry into an include

changes since v2:
 - add python namespace prefix for the bitbake get functions
 - install only in images that use WKS_FILEs that have the "gap"

changes since v1:
 - complete rewrite to not focus only on the tests but rather the
   feature for all examples

This first was only about testing but meanwhile has a focus on enabling
the expand feature in all example images anyhow, and later look for
traces of that happening in CI test where we can.
We install the application in any image and in the wic images add some
scratch space at the end. That will enable improved interactive use of
all those wic images, so makes sense apart from any testing.
Later use the log reading feature of the qemu tests to look for traces
of successful resizing for non ubuntu images. In ubuntu the
console_loglevel is not verbose enough to find traces.


Henning Schild (6):
  expand-on-first-boot: really only do that once, especially on failure
  expand-on-first-boot: support resizing a btrfs
  meta-isar: introduce an example to use btrfs
  CI: improve cibuilder readability
  meta-isar: install expand-on-first-boot in most images and add space
  CI: expect a message about filesystem resize vom expand script

 meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
 meta-isar/conf/machine/virtualbox.conf        |  1 +
 meta-isar/conf/machine/vmware.conf            |  1 +
 .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
 .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
 .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 +++++++
 .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
 .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
 ...oot_1.3.bb => expand-on-first-boot_1.4.bb} |  0
 .../files/expand-last-partition.sh            | 31 ++++++++++++++++---
 .../files/expand-on-first-boot.service        |  1 +
 testsuite/cibuilder.py                        | 31 ++++++++++++++++---
 12 files changed, 83 insertions(+), 9 deletions(-)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
 rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%)

-- 
2.38.2

[PATCH v7 1/6] expand-on-first-boot: really only do that once, especially on failure

[Henning Schild]

We do hope the expand will work but we also know very well that it can
fail. The script is written in a way that currently only supports one
try. And not a retry if it failed in an earlier boot.
So make sure to disable the service also on failure.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 .../expand-on-first-boot/files/expand-on-first-boot.service      | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service b/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service
index fda50016e9fe..90c92a396b27 100644
--- a/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service
@@ -15,6 +15,7 @@ ConditionPathIsReadWrite=/etc
 Type=oneshot
 ExecStart=/usr/share/expand-on-first-boot/expand-last-partition.sh
 ExecStartPost=-/bin/systemctl disable expand-on-first-boot.service
+ExecStopPost=-/bin/systemctl disable expand-on-first-boot.service
 
 [Install]
 WantedBy=sysinit.target
-- 
2.38.2

[PATCH v7 2/6] expand-on-first-boot: support resizing a btrfs

[Henning Schild]

This adds support for resizing btrfs filesystems if they are in that
last partition. It also prepares for potentially other filesystems to
come in the future by introducing a switch-case.

The mounting logic is taken from the systemd-growfs patches we had to
revert again. Some filesystems need to online for resizing, but in order
to find the filesystem of a partition (without udev) mounting it and
letting the kernel detect seems a good idea.

Suggested-by: Joe MacDonald <joe_macdonald@mentor.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 ...oot_1.3.bb => expand-on-first-boot_1.4.bb} |  0
 .../files/expand-last-partition.sh            | 31 ++++++++++++++++---
 2 files changed, 27 insertions(+), 4 deletions(-)
 rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.3.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.4.bb
similarity index 100%
rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.3.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.4.bb
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 93eddda2a3b7..7d658864924b 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -62,8 +62,31 @@ if grep -q x-systemd.growfs /etc/fstab; then
 	exit 0
 fi
 
-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
+# some filesystems need to be mounted i.e. btrfs, but mounting also helps
+# detect the filesystem type without having to wait for udev
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")
+mount "${LAST_PART}" "${MOUNT_POINT}"
 
-resize2fs "${LAST_PART}"
+ret=0
+# Determine the filesystem type and perform the appropriate resize function
+FS_TYPE=$(findmnt -fno FSTYPE "${MOUNT_POINT}" )
+case ${FS_TYPE} in
+ext*)
+	# Do not fail resize2fs if no mtab entry is found, e.g.,
+	# when using systemd mount units.
+	export EXT2FS_NO_MTAB_OK=1
+	resize2fs "${LAST_PART}"
+	;;
+btrfs)
+	btrfs filesystem resize max "${MOUNT_POINT}"
+	;;
+*)
+	echo "Unrecognized filesystem type ${FS_TYPE} - no resize performed"
+	ret=1
+	;;
+esac
+
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"
+exit $ret
-- 
2.38.2

[PATCH v7 3/6] meta-isar: introduce an example to use btrfs

[Henning Schild]

Switch debian bullseye over to btrfs rootfs and also to btrfs in last
partition to have a target where we test expand-on-first-boot for btrfs.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf                    | 5 +++++
 .../scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks     | 9 +++++++++
 2 files changed, 14 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index eca2628c4ae5..6b40cb50702e 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -13,6 +13,11 @@ IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 WKS_FILE_ubuntu-focal ?= "sdimage-efi-sd"
 IMAGER_INSTALL_remove_ubuntu-focal = "${GRUB_BOOTLOADER_INSTALL}"
 
+WKS_FILE_debian-bullseye ?= "sdimage-efi-btrfs"
+IMAGER_INSTALL_remove_debian-bullseye = "${GRUB_BOOTLOADER_INSTALL}"
+IMAGER_INSTALL_append_debian-bullseye = " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-progs"
+IMAGE_PREINSTALL_append_debian-bullseye = " btrfs-progs"
+
 IMAGE_INSTALL += "sshd-regen-keys"
 
 QEMU_ARCH ?= "x86_64"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
new file mode 100644
index 000000000000..014ecca61f0c
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
@@ -0,0 +1,9 @@
+# short-description: Create an EFI disk image without any swap and btrfs
+# long-description: Creates a partitioned EFI disk image without any swap that
+# the user can directly dd to boot media, where the last partition is btrfs.
+
+part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondisk sda --label efi --part-type EF00 --align 1024
+
+part / --source rootfs --ondisk sda --fstype btrfs --label platform --align 1024 --use-uuid --exclude-path boot/
+
+bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.38.2

[PATCH v7 4/6] CI: improve cibuilder readability

[Henning Schild]

What was called "servive_prompt" is not a prompt nor a service. Document
what that is and rename the variable.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 testsuite/cibuilder.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index d5c83b222d50..07a9edc56d9c 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -229,7 +229,8 @@ class CIBuilder(Test):
         self.log.info('QEMU boot line: ' + str(cmdline))
 
         login_prompt = b'isar login:'
-        service_prompt = b'Just an example'
+        # the printk of recipes-kernel/example-module
+        module_output = b'Just an example'
 
         timeout = time.time() + int(time_to_wait)
 
@@ -263,7 +264,7 @@ class CIBuilder(Test):
         if os.path.exists(output_file) and os.path.getsize(output_file) > 0:
             with open(output_file, "rb") as f1:
                 data = f1.read()
-                if service_prompt in data and login_prompt in data:
+                if module_output in data and login_prompt in data:
                     return
                 else:
                     app_log.error(data.decode(errors='replace'))
-- 
2.38.2

[PATCH v7 5/6] meta-isar: install expand-on-first-boot in most images and add space

[Henning Schild]

Here we take the most common and generic images and add some empty space
at the end of then.
This will enable better interactive use of our example images since
people will have some space to install some more packages. While the
space seems fixed it really is open end if the mass storage happens to be
bigger, because we install expand-on-first-boot as well.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf                       | 1 +
 meta-isar/conf/machine/virtualbox.conf                      | 1 +
 meta-isar/conf/machine/vmware.conf                          | 1 +
 meta-isar/conf/multiconfig/qemuarm-bookworm.conf            | 1 +
 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc | 5 +++++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks     | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks        | 2 ++
 8 files changed, 15 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index 6b40cb50702e..a1be5e5b113b 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -19,6 +19,7 @@ IMAGER_INSTALL_append_debian-bullseye = " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-pr
 IMAGE_PREINSTALL_append_debian-bullseye = " btrfs-progs"
 
 IMAGE_INSTALL += "sshd-regen-keys"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 QEMU_ARCH ?= "x86_64"
 QEMU_MACHINE ?= "q35"
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
index de3d04c3c4b4..8c3e9bacbf9b 100644
--- a/meta-isar/conf/machine/virtualbox.conf
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -10,6 +10,7 @@ KERNEL_NAME ?= "amd64"
 WKS_FILE ?= "sdimage-efi"
 
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 VMDK_SUBFORMAT = "monolithicSparse"
 IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
index fba639b19350..42503374742c 100644
--- a/meta-isar/conf/machine/vmware.conf
+++ b/meta-isar/conf/machine/vmware.conf
@@ -12,6 +12,7 @@ WKS_FILE ?= "sdimage-efi"
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 
 OVF_TEMPLATE_FILE ?= "vm-img-vmware.ovf.tmpl"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 VMDK_SUBFORMAT = "streamOptimized"
 IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/multiconfig/qemuarm-bookworm.conf b/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
index 126935b06fda..0b9ecb664545 100644
--- a/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
@@ -7,3 +7,4 @@ IMAGE_FSTYPES_append = " wic"
 WKS_FILE ?= "sdimage-efi-sd"
 
 IMAGER_INSTALL += "${SYSTEMD_BOOTLOADER_INSTALL}"
+IMAGE_INSTALL += "expand-on-first-boot"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
new file mode 100644
index 000000000000..701fba764176
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
@@ -0,0 +1,5 @@
+# some extra space we put in the end after the last partition
+# that allows testing expand-on-first-boot while at the same time
+# making our images more useful because they have space to play around
+
+part --source empty --no-table --ondisk sda --size 256M
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
index 014ecca61f0c..e22dba2f83f8 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondi
 
 part / --source rootfs --ondisk sda --fstype btrfs --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
index 754fbc46f8e4..7fe2953f5ff2 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondi
 
 part / --source rootfs --ondisk sda --fstype ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
index f3addbc7515d..80c3a5359b1f 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=grub-efi" --ondisk s
 
 part / --source rootfs --ondisk sda --fstype ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.38.2

[PATCH v7 6/6] CI: expect a message about filesystem resize vom expand script

[Henning Schild]

We have added the expand-on-first-boot recipe and some space to grow to
our example images. So now any image using wic should "expand" on its
first boot. And the kernel actually leaves a message for us to read, at
least on debian but not on ubuntu.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 testsuite/cibuilder.py | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 07a9edc56d9c..09c6ab9e535f 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -231,6 +231,23 @@ class CIBuilder(Test):
         login_prompt = b'isar login:'
         # the printk of recipes-kernel/example-module
         module_output = b'Just an example'
+        resize_output = None
+
+        bb_output = start_vm.get_bitbake_env(arch, distro).decode()
+        image_fstypes = start_vm.get_bitbake_var(bb_output, 'IMAGE_FSTYPES')
+        wks_file = start_vm.get_bitbake_var(bb_output, 'WKS_FILE')
+        # only the first type will be tested in start_vm.py
+        if image_fstypes.split()[0] == 'wic':
+            if wks_file:
+                bbdistro = start_vm.get_bitbake_var(bb_output, 'DISTRO')
+                # ubuntu is less verbose so we do not see the message
+                # /etc/sysctl.d/10-console-messages.conf
+                if bbdistro and "ubuntu" not in bbdistro:
+                    if "sdimage-efi-sd" in wks_file:
+                        # output we see when expand-on-first-boot runs on ext4
+                        resize_output = b'resized filesystem to'
+                    if "sdimage-efi-btrfs" in wks_file:
+                        resize_output = b': resize device '
 
         timeout = time.time() + int(time_to_wait)
 
@@ -265,8 +282,11 @@ class CIBuilder(Test):
             with open(output_file, "rb") as f1:
                 data = f1.read()
                 if module_output in data and login_prompt in data:
-                    return
-                else:
-                    app_log.error(data.decode(errors='replace'))
+                    if resize_output:
+                        if resize_output in data:
+                            return
+                    else:
+                        return
+                app_log.error(data.decode(errors='replace'))
 
         self.fail('Log ' + output_file)
-- 
2.38.2

Re: [PATCH v7 0/6] expand-on-first-boot btrfs and CI testing

[Henning Schild]

This now passed ilbers CI. To double check that last patch i triggered
another run in ilbers CI which i expect to fail because it looks for
strings that should no be there and takes the looking for strings path.

Henning

On Fri, 23 Dec 2022 09:55:23 +0100
Henning Schild <henning.schild@siemens.com> wrote:

> changes since v6:
>  - make sure CI only expects resize output when booting wic
> 
> changes since v5:
>  - drop dead code around mktemp subshell
>  - revert cosmetic mktemp stuff back to v3
>  - remove "Suggested-by" tag to hopefully improve signal to noise
> ratio in review process
> 
> changes since v4:
>  - cosmetic fix in the way we call mktemp
>  - add a patch that makes sure the service is run only once, also on
>    error
>  - bump package version after changes are done
> 
> changes since v3:
>  - rebased on "expand-on-first-boot: switch back away from
> systemd-growfs"
>  - implement btrfs resizing and CI testing it
>  - put the padding wks entry into an include
> 
> changes since v2:
>  - add python namespace prefix for the bitbake get functions
>  - install only in images that use WKS_FILEs that have the "gap"
> 
> changes since v1:
>  - complete rewrite to not focus only on the tests but rather the
>    feature for all examples
> 
> This first was only about testing but meanwhile has a focus on
> enabling the expand feature in all example images anyhow, and later
> look for traces of that happening in CI test where we can.
> We install the application in any image and in the wic images add some
> scratch space at the end. That will enable improved interactive use of
> all those wic images, so makes sense apart from any testing.
> Later use the log reading feature of the qemu tests to look for traces
> of successful resizing for non ubuntu images. In ubuntu the
> console_loglevel is not verbose enough to find traces.
> 
> 
> Henning Schild (6):
>   expand-on-first-boot: really only do that once, especially on
> failure expand-on-first-boot: support resizing a btrfs
>   meta-isar: introduce an example to use btrfs
>   CI: improve cibuilder readability
>   meta-isar: install expand-on-first-boot in most images and add space
>   CI: expect a message about filesystem resize vom expand script
> 
>  meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
>  meta-isar/conf/machine/virtualbox.conf        |  1 +
>  meta-isar/conf/machine/vmware.conf            |  1 +
>  .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
>  .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
>  .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 +++++++
>  .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
>  .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
>  ...oot_1.3.bb => expand-on-first-boot_1.4.bb} |  0
>  .../files/expand-last-partition.sh            | 31
> ++++++++++++++++--- .../files/expand-on-first-boot.service        |
> 1 + testsuite/cibuilder.py                        | 31
> ++++++++++++++++--- 12 files changed, 83 insertions(+), 9 deletions(-)
>  create mode 100644
> meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create
> mode 100644
> meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks rename
> meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb
> => expand-on-first-boot_1.4.bb} (100%)
> 

Re: [PATCH v7 5/6] meta-isar: install expand-on-first-boot in most images and add space

[Roberto A. Foglietta]

On Fri, 23 Dec 2022 at 09:55, Henning Schild <henning.schild@siemens.com> wrote:
>
> Here we take the most common and generic images and add some empty space
> at the end of then.
> This will enable better interactive use of our example images since
> people will have some space to install some more packages. While the
> space seems fixed it really is open end if the mass storage happens to be
> bigger, because we install expand-on-first-boot as well.

[...]

> +++ b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
> @@ -0,0 +1,5 @@
> +# some extra space we put in the end after the last partition
> +# that allows testing expand-on-first-boot while at the same time
> +# making our images more useful because they have space to play around
> +
> +part --source empty --no-table --ondisk sda --size 256M

This approach works only if the expand last partition is installed and
works but only after the first boot into a device or a virtual
machine.

Otherwise this space will remain unavailable. For example, if someone
chroot into wic image for applying some changes s/he wants to test
before integrating into a recipe.

An alternative to add extra space into the disk, is to add it directly
into the rootfs with these two last options:

part / --source rootfs [...] --extra-space 1G --overhead-factor 1.2

I developed wicshell.sh in order to automatize the operations of
chroot into the .wic image and also sharing a folder with the host
which is always useful when ssh/scp is not available.

https://github.com/robang74/isar-nvidia-debian/blob/main/wicshell.sh

Moreover, I saw that there are .conf files for producing vmdk and ova
files. IMHO, a more general and flexible approach is to create these
files directly from the .wic image, like in my project

wicinst vmdk:$filename.vmdk [disk_size_in_gigabytes]

wicinst ovaf:$filename.ova [disk_size_in_gigabytes]

https://github.com/robang74/isar-nvidia-debian/blob/main/wicinst.sh

I hope you are going to take in consideration this approach to
simplify the access of these functionalities which are documented into
my project README.md

Every suggestion and critic in order to improve this approach will be welcome.

Best regards, R-

Re: [PATCH v7 5/6] meta-isar: install expand-on-first-boot in most images and add space

[Roberto A. Foglietta]

On Fri, 23 Dec 2022 at 16:43, Roberto A. Foglietta
<roberto.foglietta@gmail.com> wrote:

> Moreover, I saw that there are .conf files for producing vmdk and ova
> files. IMHO, a more general and flexible approach is to create these
> files directly from the .wic image, like in my project
>
> wicinst vmdk:$filename.vmdk [disk_size_in_gigabytes]
>
> wicinst ovaf:$filename.ova [disk_size_in_gigabytes]

However, some automatic test systems might prefer the .ova or .vmdk as
the ultimate output of the building without the necessity to call a
secondary script. I have not in mind anyone but I cannot exclude that
they exist. So, the two approaches can exist together as complementary
ones. Or possibly, their .conf allows a more tunable output than a
mere conversion which is easier to adopt during the development,
instead. IMHO.

> Best regards, R-

Re: [PATCH v7 0/6] expand-on-first-boot btrfs and CI testing

[Anton Mikanovich]

23.12.2022 11:55, Henning Schild wrote:
> changes since v6:
>   - make sure CI only expects resize output when booting wic
>
> changes since v5:
>   - drop dead code around mktemp subshell
>   - revert cosmetic mktemp stuff back to v3
>   - remove "Suggested-by" tag to hopefully improve signal to noise ratio
>     in review process
>
> changes since v4:
>   - cosmetic fix in the way we call mktemp
>   - add a patch that makes sure the service is run only once, also on
>     error
>   - bump package version after changes are done
>
> changes since v3:
>   - rebased on "expand-on-first-boot: switch back away from systemd-growfs"
>   - implement btrfs resizing and CI testing it
>   - put the padding wks entry into an include
>
> changes since v2:
>   - add python namespace prefix for the bitbake get functions
>   - install only in images that use WKS_FILEs that have the "gap"
>
> changes since v1:
>   - complete rewrite to not focus only on the tests but rather the
>     feature for all examples
>
> This first was only about testing but meanwhile has a focus on enabling
> the expand feature in all example images anyhow, and later look for
> traces of that happening in CI test where we can.
> We install the application in any image and in the wic images add some
> scratch space at the end. That will enable improved interactive use of
> all those wic images, so makes sense apart from any testing.
> Later use the log reading feature of the qemu tests to look for traces
> of successful resizing for non ubuntu images. In ubuntu the
> console_loglevel is not verbose enough to find traces.

Applied to next, thanks.