[PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[henning.schild]

From: Henning Schild <henning.schild@siemens.com>

changes since v3:
 - rebased on "expand-on-first-boot: switch back away from systemd-growfs"
 - implement btrfs resizing and CI testing it
 - put the padding wks entry into an include

changes since v2:
 - add python namespace prefix for the bitbake get functions
 - install only in images that use WKS_FILEs that have the "gap"

changes since v1:
 - complete rewrite to not focus only on the tests but rather the
   feature for all examples

This first was only about testing but meanwhile has a focus on enabling
the expand feature in all example images anyhow, and later look for
traces of that happening in CI test where we can.
We install the application in any image and in the wic images add some
scratch space at the end. That will enable improved interactive use of
all those wic images, so makes sense apart from any testing.
Later use the log reading feature of the qemu tests to look for traces
of successful resizing for non ubuntu images. In ubuntu the
console_loglevel is not verbose enough to find traces.


Henning Schild (5):
  expand-on-first-boot: support resizing a btrfs
  meta-isar: introduce an example to use btrfs
  CI: improve cibuilder readability
  meta-isar: install expand-on-first-boot in most images and add space
  CI: expect a message about filesystem resize vom expand script

 meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
 meta-isar/conf/machine/virtualbox.conf        |  1 +
 meta-isar/conf/machine/vmware.conf            |  1 +
 .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
 .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
 .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
 .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
 .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
 .../files/expand-last-partition.sh            | 35 ++++++++++++++++---
 testsuite/cibuilder.py                        | 28 ++++++++++++---
 10 files changed, 83 insertions(+), 9 deletions(-)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks

-- 
2.37.4

[PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[henning.schild]

From: Henning Schild <henning.schild@siemens.com>

This adds support for resizing btrfs filesystems if they are in that
last partition. It also prepares for potentially other filesystems to
come in the future by introducing a switch-case.

The mounting logic is taken from the systemd-growfs patches we had to
revert again. Some filesystems need to online for resizing, but in order
to find the filesystem of a partition (without udev) mounting it and
letting the kernel detect seems a good idea.

Suggested-by: Roberto A. Foglietta <roberto.foglietta@gmail.com>
Suggested-by: Joe MacDonald <joe_macdonald@mentor.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 .../files/expand-last-partition.sh            | 35 ++++++++++++++++---
 1 file changed, 31 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 93eddda2a3b7..0bc686059718 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -62,8 +62,35 @@ if grep -q x-systemd.growfs /etc/fstab; then
 	exit 0
 fi
 
-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
+# some filesystems need to be mounted i.e. btrfs, but mounting also helps
+# detect the filesystem type without having to wait for udev
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")
+if [ ! -d "${MOUNT_POINT}" ]; then
+	echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
+	exit 1
+fi
+mount "${LAST_PART}" "${MOUNT_POINT}"
+
+ret=0
+# Determine the filesystem type and perform the appropriate resize function
+FS_TYPE=$(findmnt -fno FSTYPE "${MOUNT_POINT}" )
+case ${FS_TYPE} in
+ext*)
+	# Do not fail resize2fs if no mtab entry is found, e.g.,
+	# when using systemd mount units.
+	export EXT2FS_NO_MTAB_OK=1
+	resize2fs "${LAST_PART}"
+	;;
+btrfs)
+	btrfs filesystem resize max "${MOUNT_POINT}"
+	;;
+*)
+	echo "Unrecognized filesystem type ${FS_TYPE} - no resize performed"
+	ret=1
+	;;
+esac
 
-resize2fs "${LAST_PART}"
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"
+exit $ret
-- 
2.37.4

[PATCH v4 2/5] meta-isar: introduce an example to use btrfs

[henning.schild]

From: Henning Schild <henning.schild@siemens.com>

Switch debian bullseye over to btrfs rootfs and also to btrfs in last
partition to have a target where we test expand-on-first-boot for btrfs.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf                    | 5 +++++
 .../scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks     | 9 +++++++++
 2 files changed, 14 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index eca2628c4ae5..6b40cb50702e 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -13,6 +13,11 @@ IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 WKS_FILE_ubuntu-focal ?= "sdimage-efi-sd"
 IMAGER_INSTALL_remove_ubuntu-focal = "${GRUB_BOOTLOADER_INSTALL}"
 
+WKS_FILE_debian-bullseye ?= "sdimage-efi-btrfs"
+IMAGER_INSTALL_remove_debian-bullseye = "${GRUB_BOOTLOADER_INSTALL}"
+IMAGER_INSTALL_append_debian-bullseye = " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-progs"
+IMAGE_PREINSTALL_append_debian-bullseye = " btrfs-progs"
+
 IMAGE_INSTALL += "sshd-regen-keys"
 
 QEMU_ARCH ?= "x86_64"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
new file mode 100644
index 000000000000..014ecca61f0c
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
@@ -0,0 +1,9 @@
+# short-description: Create an EFI disk image without any swap and btrfs
+# long-description: Creates a partitioned EFI disk image without any swap that
+# the user can directly dd to boot media, where the last partition is btrfs.
+
+part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondisk sda --label efi --part-type EF00 --align 1024
+
+part / --source rootfs --ondisk sda --fstype btrfs --label platform --align 1024 --use-uuid --exclude-path boot/
+
+bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.37.4

[PATCH v4 3/5] CI: improve cibuilder readability

[henning.schild]

From: Henning Schild <henning.schild@siemens.com>

What was called "servive_prompt" is not a prompt nor a service. Document
what that is and rename the variable.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 testsuite/cibuilder.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index d5c83b222d50..07a9edc56d9c 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -229,7 +229,8 @@ class CIBuilder(Test):
         self.log.info('QEMU boot line: ' + str(cmdline))
 
         login_prompt = b'isar login:'
-        service_prompt = b'Just an example'
+        # the printk of recipes-kernel/example-module
+        module_output = b'Just an example'
 
         timeout = time.time() + int(time_to_wait)
 
@@ -263,7 +264,7 @@ class CIBuilder(Test):
         if os.path.exists(output_file) and os.path.getsize(output_file) > 0:
             with open(output_file, "rb") as f1:
                 data = f1.read()
-                if service_prompt in data and login_prompt in data:
+                if module_output in data and login_prompt in data:
                     return
                 else:
                     app_log.error(data.decode(errors='replace'))
-- 
2.37.4

[PATCH v4 4/5] meta-isar: install expand-on-first-boot in most images and add space

[henning.schild]

From: Henning Schild <henning.schild@siemens.com>

Here we take the most common and generic images and add some empty space
at the end of then.
This will enable better interactive use of our example images since
people will have some space to install some more packages. While the
space seems fixed it really is open end if the mass storage happens to be
bigger, because we install expand-on-first-boot as well.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf                       | 1 +
 meta-isar/conf/machine/virtualbox.conf                      | 1 +
 meta-isar/conf/machine/vmware.conf                          | 1 +
 meta-isar/conf/multiconfig/qemuarm-bookworm.conf            | 1 +
 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc | 5 +++++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks     | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks        | 2 ++
 8 files changed, 15 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index 6b40cb50702e..a1be5e5b113b 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -19,6 +19,7 @@ IMAGER_INSTALL_append_debian-bullseye = " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-pr
 IMAGE_PREINSTALL_append_debian-bullseye = " btrfs-progs"
 
 IMAGE_INSTALL += "sshd-regen-keys"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 QEMU_ARCH ?= "x86_64"
 QEMU_MACHINE ?= "q35"
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
index de3d04c3c4b4..8c3e9bacbf9b 100644
--- a/meta-isar/conf/machine/virtualbox.conf
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -10,6 +10,7 @@ KERNEL_NAME ?= "amd64"
 WKS_FILE ?= "sdimage-efi"
 
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 VMDK_SUBFORMAT = "monolithicSparse"
 IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
index fba639b19350..42503374742c 100644
--- a/meta-isar/conf/machine/vmware.conf
+++ b/meta-isar/conf/machine/vmware.conf
@@ -12,6 +12,7 @@ WKS_FILE ?= "sdimage-efi"
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 
 OVF_TEMPLATE_FILE ?= "vm-img-vmware.ovf.tmpl"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 VMDK_SUBFORMAT = "streamOptimized"
 IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/multiconfig/qemuarm-bookworm.conf b/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
index 126935b06fda..0b9ecb664545 100644
--- a/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
@@ -7,3 +7,4 @@ IMAGE_FSTYPES_append = " wic"
 WKS_FILE ?= "sdimage-efi-sd"
 
 IMAGER_INSTALL += "${SYSTEMD_BOOTLOADER_INSTALL}"
+IMAGE_INSTALL += "expand-on-first-boot"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
new file mode 100644
index 000000000000..701fba764176
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
@@ -0,0 +1,5 @@
+# some extra space we put in the end after the last partition
+# that allows testing expand-on-first-boot while at the same time
+# making our images more useful because they have space to play around
+
+part --source empty --no-table --ondisk sda --size 256M
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
index 014ecca61f0c..e22dba2f83f8 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondi
 
 part / --source rootfs --ondisk sda --fstype btrfs --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
index 754fbc46f8e4..7fe2953f5ff2 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondi
 
 part / --source rootfs --ondisk sda --fstype ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
index f3addbc7515d..80c3a5359b1f 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=grub-efi" --ondisk s
 
 part / --source rootfs --ondisk sda --fstype ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.37.4

[PATCH v4 5/5] CI: expect a message about filesystem resize vom expand script

[henning.schild]

From: Henning Schild <henning.schild@siemens.com>

We have added the expand-on-first-boot recipe and some space to grow to
our example images. So now any image using wic should "expand" on its
first boot. And the kernel actually leaves a message for us to read, at
least on debian but not on ubuntu.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 testsuite/cibuilder.py | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 07a9edc56d9c..8b67815442a8 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -231,6 +231,20 @@ class CIBuilder(Test):
         login_prompt = b'isar login:'
         # the printk of recipes-kernel/example-module
         module_output = b'Just an example'
+        resize_output = None
+
+        bb_output = start_vm.get_bitbake_env(arch, distro).decode()
+        wks_file = start_vm.get_bitbake_var(bb_output, 'WKS_FILE')
+        if wks_file:
+            bbdistro = start_vm.get_bitbake_var(bb_output, 'DISTRO')
+            # ubuntu is less verbose so we do not see the message
+            # /etc/sysctl.d/10-console-messages.conf
+            if bbdistro and "ubuntu" not in bbdistro:
+                if "sdimage-efi-sd" in wks_file:
+                    # output we see when expand-on-first-boot runs on ext4
+                    resize_output = b'resized filesystem to'
+                if "sdimage-efi-btrfs" in wks_file:
+                    resize_output = b': resize device '
 
         timeout = time.time() + int(time_to_wait)
 
@@ -265,8 +279,11 @@ class CIBuilder(Test):
             with open(output_file, "rb") as f1:
                 data = f1.read()
                 if module_output in data and login_prompt in data:
-                    return
-                else:
-                    app_log.error(data.decode(errors='replace'))
+                    if resize_output:
+                        if resize_output in data:
+                            return
+                    else:
+                        return
+                app_log.error(data.decode(errors='replace'))
 
         self.fail('Log ' + output_file)
-- 
2.37.4

Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[Henning Schild]

Hi all,

i decided to merge the re-introduction of btrfs with the CI series i
had going on. Because that is all so important yet seemingly fragile
that we really do not want any big change without CI.

this is based on
"expand-on-first-boot: switch back away from systemd-growfs"
which should be merged because it breaks several machines like rpis,
with the udev race we found lately

It is tested manually in qemu, especially debian-bullseye (btrfs
candidate) and on that formerly affected rpi.
In addition i pushed it here to trigger CI runs at ilbers already:
https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci

I get the feeling we really need a way to log in to the images running
in CI and run test code there. Only waiting for the login prompt and
hoping for bits in the kernel log is not powerful enough do perform
real tests. That is something to improve at some point.

regards,
Henning

Am Tue, 13 Dec 2022 11:15:04 +0100
schrieb henning.schild@siemens.com:

> From: Henning Schild <henning.schild@siemens.com>
> 
> changes since v3:
>  - rebased on "expand-on-first-boot: switch back away from
> systemd-growfs"
>  - implement btrfs resizing and CI testing it
>  - put the padding wks entry into an include
> 
> changes since v2:
>  - add python namespace prefix for the bitbake get functions
>  - install only in images that use WKS_FILEs that have the "gap"
> 
> changes since v1:
>  - complete rewrite to not focus only on the tests but rather the
>    feature for all examples
> 
> This first was only about testing but meanwhile has a focus on
> enabling the expand feature in all example images anyhow, and later
> look for traces of that happening in CI test where we can.
> We install the application in any image and in the wic images add some
> scratch space at the end. That will enable improved interactive use of
> all those wic images, so makes sense apart from any testing.
> Later use the log reading feature of the qemu tests to look for traces
> of successful resizing for non ubuntu images. In ubuntu the
> console_loglevel is not verbose enough to find traces.
> 
> 
> Henning Schild (5):
>   expand-on-first-boot: support resizing a btrfs
>   meta-isar: introduce an example to use btrfs
>   CI: improve cibuilder readability
>   meta-isar: install expand-on-first-boot in most images and add space
>   CI: expect a message about filesystem resize vom expand script
> 
>  meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
>  meta-isar/conf/machine/virtualbox.conf        |  1 +
>  meta-isar/conf/machine/vmware.conf            |  1 +
>  .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
>  .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
>  .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
>  .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
>  .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
>  .../files/expand-last-partition.sh            | 35
> ++++++++++++++++--- testsuite/cibuilder.py                        |
> 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9 deletions(-)
>  create mode 100644
> meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create
> mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
> 

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
>
> +MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")

this uses /tmp which could not be available

debraf@debraf:~$ sudo find / -name expand-last-partition.sh
/usr/share/expand-on-first-boot/expand-last-partition.sh
debraf@debraf:~$ mktemp -d -p "" "expand-last-partition.sh.XXXXXXXXXX"
/tmp/expand-last-partition.sh.K9OhJ5KxIF

my suggestion is to use /dev/shm which usually is configured into the
kernel or at least try also /dev/shm if the use of /tmp fails.

== FELIX ==

One problem remains: Where to mount.
If we mount on /mnt this will not work on an RO rootfs out of the box.
We currently have this issue in CIP core.

We could mount below /tmp which should be writable, but I don't know
which side effects this as the tmpfs itself will be mounted / created
by systemd.

== ==

Best regards, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
>
> From: Henning Schild <henning.schild@siemens.com>
>
> This adds support for resizing btrfs filesystems if they are in that
> last partition. It also prepares for potentially other filesystems to
> come in the future by introducing a switch-case.
>
> The mounting logic is taken from the systemd-growfs patches we had to
> revert again. Some filesystems need to online for resizing, but in order
> to find the filesystem of a partition (without udev) mounting it and
> letting the kernel detect seems a good idea.
>

It is possible to avoid the udevadm settle constraints only because
the boot device [1] is the same as the rootfs device. The boot device
should be mapped by the kernel at boot time, so it would be available
but not necessarily present in /dev partition if this has no static
links or devfs mounted at the time of mount. If udevd is the demon
that we choose to populate the /dev then it would be better to wait
until it settles down. The stats about boot time on different
platforms will show that only hardware slow to detect has a larger
boot time.

[1] in some embedded devices the boot device is an internal and small
device dedicated to the boot while the rootfs is on a separate
plug-able device.

I have no clue how this would matter in the case of ISAR.

Best regards, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Henning Schild]

Am Tue, 13 Dec 2022 17:45:23 +0100
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
> >
> > +MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")  
> 
> this uses /tmp which could not be available

The remark from Felix triggered the switch from '-p "/mnt"' to '-p ""',
one can set TMPDIR if that does not work.

A valid remark, while yours seems made up just to keep trolling around
in the whole discussion. Stop it please.

Henning
 
> debraf@debraf:~$ sudo find / -name expand-last-partition.sh
> /usr/share/expand-on-first-boot/expand-last-partition.sh
> debraf@debraf:~$ mktemp -d -p "" "expand-last-partition.sh.XXXXXXXXXX"
> /tmp/expand-last-partition.sh.K9OhJ5KxIF
> 
> my suggestion is to use /dev/shm which usually is configured into the
> kernel or at least try also /dev/shm if the use of /tmp fails.
> 
> == FELIX ==
> 
> One problem remains: Where to mount.
> If we mount on /mnt this will not work on an RO rootfs out of the box.
> We currently have this issue in CIP core.
> 
> We could mount below /tmp which should be writable, but I don't know
> which side effects this as the tmpfs itself will be mounted / created
> by systemd.
> 
> == ==
> 
> Best regards, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Tue, 13 Dec 2022 at 18:18, Henning Schild <henning.schild@siemens.com> wrote:
>
> Am Tue, 13 Dec 2022 17:45:23 +0100
> schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
>
> > On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
> > >
> > > +MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")
> >
> > this uses /tmp which could not be available
>
> The remark from Felix triggered the switch from '-p "/mnt"' to '-p ""',
> one can set TMPDIR if that does not work.
>
> A valid remark, while yours seems made up just to keep trolling around
> in the whole discussion. Stop it please.

Please Hanning, tell to it to my ubuntu - she has some issue to
understand your slang

roberto:~$ export TMPDIR=/dev/shm
roberto:~$ mktemp -d -p "" ciao.XXXXX
/dev/shm/ciao.X3wlT

Ah, oops - in this following way it working, instead

roberto:~$ mktemp -d -p "$TMPDIR" ciao.XXXXX
/dev/shm/ciao.jQFYr

Possibly you did not catch the Felix suggestion? LOL

Ciao, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Henning Schild]

Am Tue, 13 Dec 2022 21:44:01 +0100
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> On Tue, 13 Dec 2022 at 18:18, Henning Schild
> <henning.schild@siemens.com> wrote:
> >
> > Am Tue, 13 Dec 2022 17:45:23 +0100
> > schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
> >  
> > > On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
> > >  
> > > >
> > > > +MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")  
> > >
> > > this uses /tmp which could not be available  
> >
> > The remark from Felix triggered the switch from '-p "/mnt"' to '-p
> > ""', one can set TMPDIR if that does not work.
> >
> > A valid remark, while yours seems made up just to keep trolling
> > around in the whole discussion. Stop it please.  
> 
> Please Hanning, tell to it to my ubuntu - she has some issue to
> understand your slang

My name is "Henning" and yours is not Joe!

> roberto:~$ export TMPDIR=/dev/shm
> roberto:~$ mktemp -d -p "" ciao.XXXXX
> /dev/shm/ciao.X3wlT
> 
> Ah, oops - in this following way it working, instead
> 
> roberto:~$ mktemp -d -p "$TMPDIR" ciao.XXXXX
> /dev/shm/ciao.jQFYr

Not sure what i see here but it looks like both temporary directories
have been created under TMPDIR with -p "" and with -p "$TMPDIR".

I like the "ciao" since it seems to suggest you are willing to leave
it.

Henning

> Possibly you did not catch the Felix suggestion? LOL
> 
> Ciao, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Tue, 13 Dec 2022 at 22:06, Henning Schild <henning.schild@siemens.com> wrote:
>
> Am Tue, 13 Dec 2022 21:44:01 +0100
> schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
>
> > On Tue, 13 Dec 2022 at 18:18, Henning Schild
> > <henning.schild@siemens.com> wrote:
> > >
> > > Am Tue, 13 Dec 2022 17:45:23 +0100
> > > schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
> > >
> > > > On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
> > > >
> > > > >
> > > > > +MOUNT_POINT=$(mktemp -d -p "" "$(basename "$0").XXXXXXXXXX")
> > > >
> > > > this uses /tmp which could not be available
> > >
> > > The remark from Felix triggered the switch from '-p "/mnt"' to '-p
> > > ""', one can set TMPDIR if that does not work.
> > >
> > > A valid remark, while yours seems made up just to keep trolling
> > > around in the whole discussion. Stop it please.
> >
> > Please Hanning, tell to it to my ubuntu - she has some issue to
> > understand your slang
>
> My name is "Henning" and yours is not Joe!
>
> > roberto:~$ export TMPDIR=/dev/shm
> > roberto:~$ mktemp -d -p "" ciao.XXXXX
> > /dev/shm/ciao.X3wlT
> >
> > Ah, oops - in this following way it working, instead
> >
> > roberto:~$ mktemp -d -p "$TMPDIR" ciao.XXXXX
> > /dev/shm/ciao.jQFYr
>
> Not sure what i see here but it looks like both temporary directories
> have been created under TMPDIR with -p "" and with -p "$TMPDIR".

What Felix tried to tell you was to use a variable like in my patch
but using TMPDIR as the name of the variable. When that variable is
not defined then "" is the default value. Now, what happens to mktemp
when it is called with -p ""? Nothing bad, it simply performs without
-p. Why was Felix pointing this out? Probably because in my patch tmp
variable should be set if null but using mktemp this is not necessary
anymore.

>
> I like the "ciao" since it seems to suggest you are willing to leave
> it.
>

I have the light sensation that after two and more months needed to
solve an issue that we solved in 2 days after my intervention in this
list... ciao is for you.

> Henning
>
> > Possibly you did not catch the Felix suggestion? LOL
> >
> > Ciao, R-
>

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Henning Schild]

Am Tue, 13 Dec 2022 18:01:49 +0100
schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com> wrote:
> >
> > From: Henning Schild <henning.schild@siemens.com>
> >
> > This adds support for resizing btrfs filesystems if they are in that
> > last partition. It also prepares for potentially other filesystems
> > to come in the future by introducing a switch-case.
> >
> > The mounting logic is taken from the systemd-growfs patches we had
> > to revert again. Some filesystems need to online for resizing, but
> > in order to find the filesystem of a partition (without udev)
> > mounting it and letting the kernel detect seems a good idea.
> >  
> 
> It is possible to avoid the udevadm settle constraints only because
> the boot device [1] is the same as the rootfs device. The boot device
> should be mapped by the kernel at boot time, so it would be available
> but not necessarily present in /dev partition if this has no static
> links or devfs mounted at the time of mount. If udevd is the demon
> that we choose to populate the /dev then it would be better to wait
> until it settles down. The stats about boot time on different
> platforms will show that only hardware slow to detect has a larger
> boot time.
> 
> [1] in some embedded devices the boot device is an internal and small
> device dedicated to the boot while the rootfs is on a separate
> plug-able device.
> 
> I have no clue how this would matter in the case of ISAR.

The whole thing is about only one device and its last partition, not
more! And we clearly expect boot and root to reside on the same
mass-storage. Or any other last partition (my series was also manually
tested against another /data partition to be grown).

Thanks for approving that we do not need to "udevadm settle" in that
case!

For any more advanced scenario people can always run whatever they
want, possibly systemd-repart. Isar is all about a single image for one
mass storage device, it can be used for more but such code will have to
live in layers.

With btrfs we could have one filesystem with multiple
devices/partitions and the code i (and you and Joe) proposed here will
likely not work. Let us keep things simple in the core while allowing
layers to do whatever they please.

expand-on-first-boot is for the "normal rpi case" where a stand-alone
image gets flashed to one device of which we only know the final size
when we boot it up. Not just rpi ... it could also be your qemuamd64
image flashed to an ssd/hdd in any PC. 

ciao,
Henning

> Best regards, R-
> 

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Tue, 13 Dec 2022 at 22:22, Henning Schild <henning.schild@siemens.com> wrote:
>
> Am Tue, 13 Dec 2022 18:01:49 +0100
> schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:

> > [1] in some embedded devices the boot device is an internal and small
> > device dedicated to the boot while the rootfs is on a separate
> > plug-able device.
> >
> > I have no clue how this would matter in the case of ISAR.
>
> The whole thing is about only one device and its last partition, not
> more! And we clearly expect boot and root to reside on the same
> mass-storage. Or any other last partition (my series was also manually
> tested against another /data partition to be grown).
>
> Thanks for approving that we do not need to "udevadm settle" in that
> case!

Thank you for letting us know that the main problem was even easier
than supposed to be. Trying to help you or give you an assist is
clearly a boomerang. I will take it into consideration for the future.

By the way, with my virtualbox machine (basic-os image) the kernel
time indicates 4.8s with btrfs resize at boot completion and udevadm
settle takes so little that it has no meaning to be accounted for.

Using /usr/bin/time -o/dev/shm/udevadm-settle.time udevadm settle, the
result is here below shown:

debraf@debraf:~$ cat /dev/shm/udevadm-settle.time
0.00user 0.00system 0:00.50elapsed 1%CPU (0avgtext+0avgdata 4196maxresident)k
0inputs+0outputs (0major+248minor)pagefaults 0swaps

Best regards, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Moessbauer, Felix]

On Tue, 2022-12-13 at 22:20 +0100, Roberto A. Foglietta wrote:
> On Tue, 13 Dec 2022 at 22:06, Henning Schild <
> henning.schild@siemens.com> wrote:
> > 
> > Am Tue, 13 Dec 2022 21:44:01 +0100
> > schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
> > 
> > > On Tue, 13 Dec 2022 at 18:18, Henning Schild
> > > <henning.schild@siemens.com> wrote:
> > > > 
> > > > Am Tue, 13 Dec 2022 17:45:23 +0100
> > > > schrieb "Roberto A. Foglietta" <roberto.foglietta@gmail.com>:
> > > > 
> > > > > On Tue, 13 Dec 2022 at 11:15, <henning.schild@siemens.com>
> > > > > wrote:
> > > > > 
> > > > > > 
> > > > > > +MOUNT_POINT=$(mktemp -d -p "" "$(basename
> > > > > > "$0").XXXXXXXXXX")
> > > > > 
> > > > > this uses /tmp which could not be available
> > > > 
> > > > The remark from Felix triggered the switch from '-p "/mnt"' to
> > > > '-p
> > > > ""', one can set TMPDIR if that does not work.
> > > > 
> > > > A valid remark, while yours seems made up just to keep trolling
> > > > around in the whole discussion. Stop it please.
> > > 
> > > Please Hanning, tell to it to my ubuntu - she has some issue to
> > > understand your slang
> > 
> > My name is "Henning" and yours is not Joe!
> > 
> > > roberto:~$ export TMPDIR=/dev/shm
> > > roberto:~$ mktemp -d -p "" ciao.XXXXX
> > > /dev/shm/ciao.X3wlT
> > > 
> > > Ah, oops - in this following way it working, instead
> > > 
> > > roberto:~$ mktemp -d -p "$TMPDIR" ciao.XXXXX
> > > /dev/shm/ciao.jQFYr
> > 
> > Not sure what i see here but it looks like both temporary
> > directories
> > have been created under TMPDIR with -p "" and with -p "$TMPDIR".
> 
> What Felix tried to tell you was to use a variable like in my patch
> but using TMPDIR as the name of the variable. When that variable is
> not defined then "" is the default value. Now, what happens to mktemp
> when it is called with -p ""? Nothing bad, it simply performs without
> -p. Why was Felix pointing this out? Probably because in my patch tmp
> variable should be set if null but using mktemp this is not necessary
> anymore.

No! Please also stop guessing what I could have meant and wait at least
a day for me to answer (instead of trying to convince others of your
interpretation of my opinion).

The way Henning implemented it is exactly how I meant it.

As this whole thing runs as a systemd service, a user can easily set
TMPDIR via a drop-in file.

Felix

> 
> > 
> > I like the "ciao" since it seems to suggest you are willing to
> > leave
> > it.
> > 
> 
> I have the light sensation that after two and more months needed to
> solve an issue that we solved in 2 days after my intervention in this
> list... ciao is for you.
> 
> > Henning
> > 
> > > Possibly you did not catch the Felix suggestion? LOL
> > > 
> > > Ciao, R-
> > 

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Wed, 14 Dec 2022 at 04:33, Moessbauer, Felix
<felix.moessbauer@siemens.com> wrote:

> > What Felix tried to tell you was to use a variable like in my patch
> > but using TMPDIR as the name of the variable. When that variable is
> > not defined then "" is the default value. Now, what happens to mktemp
> > when it is called with -p ""? Nothing bad, it simply performs without
> > -p. Why was Felix pointing this out? Probably because in my patch tmp
> > variable should be set if null but using mktemp this is not necessary
> > anymore.
>
> No! Please also stop guessing what I could have meant and wait at least
> a day for me to answer (instead of trying to convince others of your
> interpretation of my opinion).
>
> The way Henning implemented it is exactly how I meant it.
>
> As this whole thing runs as a systemd service, a user can easily set
> TMPDIR via a drop-in file.
>

Felix, thanks for the clarification, the behaviour is aligned with the
manual but expliciting the variable would have been easier to read
without checking the manal

       -p DIR, --tmpdir[=DIR]
              interpret TEMPLATE relative to DIR; if DIR is not
specified, use $TMPDIR if set, else /tmp.  With this option, TEMPLATE
must not be an absolute name; unlike with  -t,  TEMPLATE  may  contain
              slashes, but mktemp creates only the final component

This is the output of the command without, with and with specified
TMPDIR variable

roberto:~/robang74/isar-nvidia-debian$ echo "x$TMPDIR"
x
roberto:~/robang74/isar-nvidia-debian$ mktemp -d -p "" ciao.XXX
/tmp/ciao.wLK
roberto:~/robang74/isar-nvidia-debian$ export TMPDIR=/dev/shm
roberto:~/robang74/isar-nvidia-debian$ mktemp -d -p "" ciao.XXX
/dev/shm/ciao.fYb
roberto:~/robang74/isar-nvidia-debian$ mktemp -d  ciao.XXX
ciao.Lwp
roberto:~/robang74/isar-nvidia-debian$ mktemp -d -p "$TMPDIR" ciao.XXX
/dev/shm/ciao.gw1

Cheers, R-

Re: [PATCH v4 1/5] expand-on-first-boot: support resizing a btrfs

[Roberto A. Foglietta]

On Wed, 14 Dec 2022 at 07:50, Roberto A. Foglietta
<roberto.foglietta@gmail.com> wrote:
>
> >
> > The way Henning implemented it is exactly how I meant it.
>

..and moreover my suggestion to him was to make a second try with
/dev/shm before quitting with error

roberto:~/robang74/isar-nvidia-debian$ export TMPDIR="/dev/null"
roberto:~/robang74/isar-nvidia-debian$ tmpdir=$(mktemp -d -p "$TMPDIR"
ciao.XXX || mktemp -d -p "/dev/shm" ciao.XXX)
mktemp: failed to create directory via template ‘/dev/null/ciao.XXX’:
Not a directory
roberto:~/robang74/isar-nvidia-debian$ echo $tmpdir
/dev/shm/ciao.NWo

More in general

- maintainability: make code that are easier to read
- resilience: provide a reasonable alternative to failure

Best regards, R-

Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[Uladzimir Bely]

In the email from Tuesday, 13 December 2022 13:25:26 +03 user Henning Schild 
wrote:
> Hi all,
> 
> i decided to merge the re-introduction of btrfs with the CI series i
> had going on. Because that is all so important yet seemingly fragile
> that we really do not want any big change without CI.
> 
> this is based on
> "expand-on-first-boot: switch back away from systemd-growfs"
> which should be merged because it breaks several machines like rpis,
> with the udev race we found lately
> 
> It is tested manually in qemu, especially debian-bullseye (btrfs
> candidate) and on that formerly affected rpi.
> In addition i pushed it here to trigger CI runs at ilbers already:
> https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci
> 
> I get the feeling we really need a way to log in to the images running
> in CI and run test code there. Only waiting for the login prompt and
> hoping for bits in the kernel log is not powerful enough do perform
> real tests. That is something to improve at some point.
> 

Hello Henning.

We are currently working on the downstream that requires such kind of access 
to the machine running under qemu. We want to implement the essential parts of 
functionality in Isar in the nearest future:
- remove scripts/start_vm in favor of testsuite/start_vm.py (or, at least, 
scripts/start_vm will be just a wrapper)
- allow custom image names (not only hardcoded 'isar-image-base') to be tested
- support checking the image for service status (at least)
- support any custom commands execution (at most)

Actually, the latest thing should also cover previous "service status 
checking". Also, when implemented, we will be able to completely remove boot 
log parsing that is used currently.

The main idea of the upcoming implementation is executing custom commands over 
SSH in "test" thread while the machine is kept running in "main" thread.

> regards,
> Henning
> 
> Am Tue, 13 Dec 2022 11:15:04 +0100
> 
> schrieb henning.schild@siemens.com:
> > From: Henning Schild <henning.schild@siemens.com>
> > 
> > changes since v3:
> >  - rebased on "expand-on-first-boot: switch back away from
> > 
> > systemd-growfs"
> > 
> >  - implement btrfs resizing and CI testing it
> >  - put the padding wks entry into an include
> > 
> > changes since v2:
> >  - add python namespace prefix for the bitbake get functions
> >  - install only in images that use WKS_FILEs that have the "gap"
> > 
> > changes since v1:
> >  - complete rewrite to not focus only on the tests but rather the
> >  
> >    feature for all examples
> > 
> > This first was only about testing but meanwhile has a focus on
> > enabling the expand feature in all example images anyhow, and later
> > look for traces of that happening in CI test where we can.
> > We install the application in any image and in the wic images add some
> > scratch space at the end. That will enable improved interactive use of
> > all those wic images, so makes sense apart from any testing.
> > Later use the log reading feature of the qemu tests to look for traces
> > of successful resizing for non ubuntu images. In ubuntu the
> > console_loglevel is not verbose enough to find traces.
> > 
> > Henning Schild (5):
> >   expand-on-first-boot: support resizing a btrfs
> >   meta-isar: introduce an example to use btrfs
> >   CI: improve cibuilder readability
> >   meta-isar: install expand-on-first-boot in most images and add space
> >   CI: expect a message about filesystem resize vom expand script
> >  
> >  meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
> >  meta-isar/conf/machine/virtualbox.conf        |  1 +
> >  meta-isar/conf/machine/vmware.conf            |  1 +
> >  .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
> >  .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
> >  .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
> >  .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
> >  .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
> >  .../files/expand-last-partition.sh            | 35
> > 
> > ++++++++++++++++--- testsuite/cibuilder.py                        |
> > 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9 deletions(-)
> > 
> >  create mode 100644
> > 
> > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create
> > mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks


-- 
Uladzimir Bely

Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[Henning Schild]

Am Tue, 20 Dec 2022 12:37:24 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:

> In the email from Tuesday, 13 December 2022 13:25:26 +03 user Henning
> Schild wrote:
> > Hi all,
> > 
> > i decided to merge the re-introduction of btrfs with the CI series i
> > had going on. Because that is all so important yet seemingly fragile
> > that we really do not want any big change without CI.
> > 
> > this is based on
> > "expand-on-first-boot: switch back away from systemd-growfs"
> > which should be merged because it breaks several machines like rpis,
> > with the udev race we found lately
> > 
> > It is tested manually in qemu, especially debian-bullseye (btrfs
> > candidate) and on that formerly affected rpi.
> > In addition i pushed it here to trigger CI runs at ilbers already:
> > https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci
> > 
> > I get the feeling we really need a way to log in to the images
> > running in CI and run test code there. Only waiting for the login
> > prompt and hoping for bits in the kernel log is not powerful enough
> > do perform real tests. That is something to improve at some point.
> >   
> 
> Hello Henning.
> 
> We are currently working on the downstream that requires such kind of
> access to the machine running under qemu. We want to implement the
> essential parts of functionality in Isar in the nearest future:
> - remove scripts/start_vm in favor of testsuite/start_vm.py (or, at
> least, scripts/start_vm will be just a wrapper)
> - allow custom image names (not only hardcoded 'isar-image-base') to
> be tested
> - support checking the image for service status (at least)
> - support any custom commands execution (at most)
> 
> Actually, the latest thing should also cover previous "service status 
> checking". Also, when implemented, we will be able to completely
> remove boot log parsing that is used currently.
> 
> The main idea of the upcoming implementation is executing custom
> commands over SSH in "test" thread while the machine is kept running
> in "main" thread.

For a truly generic way i strongly suggest looking into lava or tbot2.
We use lava in many downstream projects.

For the mainly CI topics that might be overkill, maybe OEs scripts
might be the better option. Not sure how they do that but for some of
the fstab changes we got merged there lately we had to adopt test code
that was running in the context of the newly assembled rootfs.

Henning

> > regards,
> > Henning
> > 
> > Am Tue, 13 Dec 2022 11:15:04 +0100
> > 
> > schrieb henning.schild@siemens.com:  
> > > From: Henning Schild <henning.schild@siemens.com>
> > > 
> > > changes since v3:
> > >  - rebased on "expand-on-first-boot: switch back away from
> > > 
> > > systemd-growfs"
> > > 
> > >  - implement btrfs resizing and CI testing it
> > >  - put the padding wks entry into an include
> > > 
> > > changes since v2:
> > >  - add python namespace prefix for the bitbake get functions
> > >  - install only in images that use WKS_FILEs that have the "gap"
> > > 
> > > changes since v1:
> > >  - complete rewrite to not focus only on the tests but rather the
> > >  
> > >    feature for all examples
> > > 
> > > This first was only about testing but meanwhile has a focus on
> > > enabling the expand feature in all example images anyhow, and
> > > later look for traces of that happening in CI test where we can.
> > > We install the application in any image and in the wic images add
> > > some scratch space at the end. That will enable improved
> > > interactive use of all those wic images, so makes sense apart
> > > from any testing. Later use the log reading feature of the qemu
> > > tests to look for traces of successful resizing for non ubuntu
> > > images. In ubuntu the console_loglevel is not verbose enough to
> > > find traces.
> > > 
> > > Henning Schild (5):
> > >   expand-on-first-boot: support resizing a btrfs
> > >   meta-isar: introduce an example to use btrfs
> > >   CI: improve cibuilder readability
> > >   meta-isar: install expand-on-first-boot in most images and add
> > > space CI: expect a message about filesystem resize vom expand
> > > script 
> > >  meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
> > >  meta-isar/conf/machine/virtualbox.conf        |  1 +
> > >  meta-isar/conf/machine/vmware.conf            |  1 +
> > >  .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
> > >  .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
> > >  .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
> > >  .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
> > >  .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
> > >  .../files/expand-last-partition.sh            | 35
> > > 
> > > ++++++++++++++++--- testsuite/cibuilder.py
> > > | 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9
> > > deletions(-)
> > > 
> > >  create mode 100644
> > > 
> > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create
> > > mode 100644
> > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks  
> 
> 

Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[Uladzimir Bely]

In the email from Tuesday, 20 December 2022 18:08:49 +03 user Henning Schild 
wrote:
> Am Tue, 20 Dec 2022 12:37:24 +0300
> 
> schrieb Uladzimir Bely <ubely@ilbers.de>:
> > In the email from Tuesday, 13 December 2022 13:25:26 +03 user Henning
> > 
> > Schild wrote:
> > > Hi all,
> > > 
> > > i decided to merge the re-introduction of btrfs with the CI series i
> > > had going on. Because that is all so important yet seemingly fragile
> > > that we really do not want any big change without CI.
> > > 
> > > this is based on
> > > "expand-on-first-boot: switch back away from systemd-growfs"
> > > which should be merged because it breaks several machines like rpis,
> > > with the udev race we found lately
> > > 
> > > It is tested manually in qemu, especially debian-bullseye (btrfs
> > > candidate) and on that formerly affected rpi.
> > > In addition i pushed it here to trigger CI runs at ilbers already:
> > > https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci
> > > 
> > > I get the feeling we really need a way to log in to the images
> > > running in CI and run test code there. Only waiting for the login
> > > prompt and hoping for bits in the kernel log is not powerful enough
> > > do perform real tests. That is something to improve at some point.
> > 
> > Hello Henning.
> > 
> > We are currently working on the downstream that requires such kind of
> > access to the machine running under qemu. We want to implement the
> > essential parts of functionality in Isar in the nearest future:
> > - remove scripts/start_vm in favor of testsuite/start_vm.py (or, at
> > least, scripts/start_vm will be just a wrapper)
> > - allow custom image names (not only hardcoded 'isar-image-base') to
> > be tested
> > - support checking the image for service status (at least)
> > - support any custom commands execution (at most)
> > 
> > Actually, the latest thing should also cover previous "service status
> > checking". Also, when implemented, we will be able to completely
> > remove boot log parsing that is used currently.
> > 
> > The main idea of the upcoming implementation is executing custom
> > commands over SSH in "test" thread while the machine is kept running
> > in "main" thread.
> 
> For a truly generic way i strongly suggest looking into lava or tbot2.
> We use lava in many downstream projects.
> 

I've looked at some of them and found that lava seems to be used in `xenomai-
images`. But I didn't find anything using tbot2. Could you point at some 
example projects (if they are not closed-source)?

And saying "tbot2" did you mean https://tbot.tools/ or something else? At 
first glance it looks something promising and able to be integrated to our 
python-based testsuite. While lava looks more complicated. 

> For the mainly CI topics that might be overkill, maybe OEs scripts
> might be the better option. Not sure how they do that but for some of
> the fstab changes we got merged there lately we had to adopt test code
> that was running in the context of the newly assembled rootfs.
> 
> Henning
> 
> > > regards,
> > > Henning
> > > 
> > > Am Tue, 13 Dec 2022 11:15:04 +0100
> > > 
> > > schrieb henning.schild@siemens.com:
> > > > From: Henning Schild <henning.schild@siemens.com>
> > > > 
> > > > changes since v3:
> > > >  - rebased on "expand-on-first-boot: switch back away from
> > > > 
> > > > systemd-growfs"
> > > > 
> > > >  - implement btrfs resizing and CI testing it
> > > >  - put the padding wks entry into an include
> > > > 
> > > > changes since v2:
> > > >  - add python namespace prefix for the bitbake get functions
> > > >  - install only in images that use WKS_FILEs that have the "gap"
> > > > 
> > > > changes since v1:
> > > >  - complete rewrite to not focus only on the tests but rather the
> > > >  
> > > >    feature for all examples
> > > > 
> > > > This first was only about testing but meanwhile has a focus on
> > > > enabling the expand feature in all example images anyhow, and
> > > > later look for traces of that happening in CI test where we can.
> > > > We install the application in any image and in the wic images add
> > > > some scratch space at the end. That will enable improved
> > > > interactive use of all those wic images, so makes sense apart
> > > > from any testing. Later use the log reading feature of the qemu
> > > > tests to look for traces of successful resizing for non ubuntu
> > > > images. In ubuntu the console_loglevel is not verbose enough to
> > > > find traces.
> > > > 
> > > > Henning Schild (5):
> > > >   expand-on-first-boot: support resizing a btrfs
> > > >   meta-isar: introduce an example to use btrfs
> > > >   CI: improve cibuilder readability
> > > >   meta-isar: install expand-on-first-boot in most images and add
> > > > 
> > > > space CI: expect a message about filesystem resize vom expand
> > > > script
> > > > 
> > > >  meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
> > > >  meta-isar/conf/machine/virtualbox.conf        |  1 +
> > > >  meta-isar/conf/machine/vmware.conf            |  1 +
> > > >  .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
> > > >  .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
> > > >  .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
> > > >  .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
> > > >  .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
> > > >  .../files/expand-last-partition.sh            | 35
> > > > 
> > > > ++++++++++++++++--- testsuite/cibuilder.py
> > > > 
> > > > | 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9
> > > > 
> > > > deletions(-)
> > > > 
> > > >  create mode 100644
> > > > 
> > > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create
> > > > mode 100644
> > > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks


-- 
Uladzimir Bely

Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing

[Henning Schild]

Am Wed, 21 Dec 2022 13:25:29 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:

> In the email from Tuesday, 20 December 2022 18:08:49 +03 user Henning
> Schild wrote:
> > Am Tue, 20 Dec 2022 12:37:24 +0300
> > 
> > schrieb Uladzimir Bely <ubely@ilbers.de>:  
> > > In the email from Tuesday, 13 December 2022 13:25:26 +03 user
> > > Henning
> > > 
> > > Schild wrote:  
> > > > Hi all,
> > > > 
> > > > i decided to merge the re-introduction of btrfs with the CI
> > > > series i had going on. Because that is all so important yet
> > > > seemingly fragile that we really do not want any big change
> > > > without CI.
> > > > 
> > > > this is based on
> > > > "expand-on-first-boot: switch back away from systemd-growfs"
> > > > which should be merged because it breaks several machines like
> > > > rpis, with the udev race we found lately
> > > > 
> > > > It is tested manually in qemu, especially debian-bullseye (btrfs
> > > > candidate) and on that formerly affected rpi.
> > > > In addition i pushed it here to trigger CI runs at ilbers
> > > > already:
> > > > https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci
> > > > 
> > > > I get the feeling we really need a way to log in to the images
> > > > running in CI and run test code there. Only waiting for the
> > > > login prompt and hoping for bits in the kernel log is not
> > > > powerful enough do perform real tests. That is something to
> > > > improve at some point.  
> > > 
> > > Hello Henning.
> > > 
> > > We are currently working on the downstream that requires such
> > > kind of access to the machine running under qemu. We want to
> > > implement the essential parts of functionality in Isar in the
> > > nearest future:
> > > - remove scripts/start_vm in favor of testsuite/start_vm.py (or,
> > > at least, scripts/start_vm will be just a wrapper)
> > > - allow custom image names (not only hardcoded 'isar-image-base')
> > > to be tested
> > > - support checking the image for service status (at least)
> > > - support any custom commands execution (at most)
> > > 
> > > Actually, the latest thing should also cover previous "service
> > > status checking". Also, when implemented, we will be able to
> > > completely remove boot log parsing that is used currently.
> > > 
> > > The main idea of the upcoming implementation is executing custom
> > > commands over SSH in "test" thread while the machine is kept
> > > running in "main" thread.  
> > 
> > For a truly generic way i strongly suggest looking into lava or
> > tbot2. We use lava in many downstream projects.
> >   
> 
> I've looked at some of them and found that lava seems to be used in
> `xenomai- images`. But I didn't find anything using tbot2. Could you
> point at some example projects (if they are not closed-source)?

AFAIK lava is the dominant player in that space, it is also behind
kernel-ci. I just wanted to mention tbot as an alternative i know, but
i do not know users. I called it tbot2 because a very early version of
its old implementation is actually used at Siemens in a project we ran
like this forever and never cared to switch to lava.

Its history is similar to the one of Isar. Once custom made for some
project at Siemens and later it became more abstract and OSS.

> And saying "tbot2" did you mean https://tbot.tools/ or something
> else? At first glance it looks something promising and able to be
> integrated to our python-based testsuite. While lava looks more
> complicated. 

Yes that is the one. I did add Harald maybe he can share some insights
on maturity and users or say a few words on how it compares to lava.

But again, i think in isar CI we might not need such complex things. A
way to start a bunch of qemus might be good enough, only we want to
gain a way to execute code inside them ... via serial or ssh.
I would first look what OE does.

When it comes to testing on Hardware, lava or maybe tbot are what you
want. But as long as it is only qemu ... one might not want either of
the two just yet.

Henning

> > For the mainly CI topics that might be overkill, maybe OEs scripts
> > might be the better option. Not sure how they do that but for some
> > of the fstab changes we got merged there lately we had to adopt
> > test code that was running in the context of the newly assembled
> > rootfs.
> > 
> > Henning
> >   
> > > > regards,
> > > > Henning
> > > > 
> > > > Am Tue, 13 Dec 2022 11:15:04 +0100
> > > > 
> > > > schrieb henning.schild@siemens.com:  
> > > > > From: Henning Schild <henning.schild@siemens.com>
> > > > > 
> > > > > changes since v3:
> > > > >  - rebased on "expand-on-first-boot: switch back away from
> > > > > 
> > > > > systemd-growfs"
> > > > > 
> > > > >  - implement btrfs resizing and CI testing it
> > > > >  - put the padding wks entry into an include
> > > > > 
> > > > > changes since v2:
> > > > >  - add python namespace prefix for the bitbake get functions
> > > > >  - install only in images that use WKS_FILEs that have the
> > > > > "gap"
> > > > > 
> > > > > changes since v1:
> > > > >  - complete rewrite to not focus only on the tests but rather
> > > > > the 
> > > > >    feature for all examples
> > > > > 
> > > > > This first was only about testing but meanwhile has a focus on
> > > > > enabling the expand feature in all example images anyhow, and
> > > > > later look for traces of that happening in CI test where we
> > > > > can. We install the application in any image and in the wic
> > > > > images add some scratch space at the end. That will enable
> > > > > improved interactive use of all those wic images, so makes
> > > > > sense apart from any testing. Later use the log reading
> > > > > feature of the qemu tests to look for traces of successful
> > > > > resizing for non ubuntu images. In ubuntu the
> > > > > console_loglevel is not verbose enough to find traces.
> > > > > 
> > > > > Henning Schild (5):
> > > > >   expand-on-first-boot: support resizing a btrfs
> > > > >   meta-isar: introduce an example to use btrfs
> > > > >   CI: improve cibuilder readability
> > > > >   meta-isar: install expand-on-first-boot in most images and
> > > > > add
> > > > > 
> > > > > space CI: expect a message about filesystem resize vom expand
> > > > > script
> > > > > 
> > > > >  meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
> > > > >  meta-isar/conf/machine/virtualbox.conf        |  1 +
> > > > >  meta-isar/conf/machine/vmware.conf            |  1 +
> > > > >  .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
> > > > >  .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
> > > > >  .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
> > > > >  .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
> > > > >  .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
> > > > >  .../files/expand-last-partition.sh            | 35
> > > > > 
> > > > > ++++++++++++++++--- testsuite/cibuilder.py
> > > > > 
> > > > > | 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9
> > > > > 
> > > > > deletions(-)
> > > > > 
> > > > >  create mode 100644
> > > > > 
> > > > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
> > > > > create mode 100644
> > > > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks  
> 
>