public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
* [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing
@ 2022-12-15 14:27 henning.schild
  2022-12-15 14:27 ` [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure henning.schild
                   ` (5 more replies)
  0 siblings, 6 replies; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild

From: Henning Schild <henning.schild@siemens.com>

changes since v4:
 - cosmetic fix in the way we call mktemp
 - add a patch that makes sure the service is run only once, also on
   error
 - bump package version after changes are done

changes since v3:
 - rebased on "expand-on-first-boot: switch back away from systemd-growfs"
 - implement btrfs resizing and CI testing it
 - put the padding wks entry into an include

changes since v2:
 - add python namespace prefix for the bitbake get functions
 - install only in images that use WKS_FILEs that have the "gap"

changes since v1:
 - complete rewrite to not focus only on the tests but rather the
   feature for all examples

This first was only about testing but meanwhile has a focus on enabling
the expand feature in all example images anyhow, and later look for
traces of that happening in CI test where we can.
We install the application in any image and in the wic images add some
scratch space at the end. That will enable improved interactive use of
all those wic images, so makes sense apart from any testing.
Later use the log reading feature of the qemu tests to look for traces
of successful resizing for non ubuntu images. In ubuntu the
console_loglevel is not verbose enough to find traces.

Henning Schild (6):
  expand-on-first-boot: really only do that once, especially on failure
  expand-on-first-boot: support resizing a btrfs
  meta-isar: introduce an example to use btrfs
  CI: improve cibuilder readability
  meta-isar: install expand-on-first-boot in most images and add space
  CI: expect a message about filesystem resize vom expand script

 meta-isar/conf/machine/qemuamd64.conf         |  6 ++++
 meta-isar/conf/machine/virtualbox.conf        |  1 +
 meta-isar/conf/machine/vmware.conf            |  1 +
 .../conf/multiconfig/qemuarm-bookworm.conf    |  1 +
 .../lib/wic/canned-wks/expand-padding.wks.inc |  5 +++
 .../lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 11 ++++++
 .../lib/wic/canned-wks/sdimage-efi-sd.wks     |  2 ++
 .../lib/wic/canned-wks/sdimage-efi.wks        |  2 ++
 ...oot_1.3.bb => expand-on-first-boot_1.4.bb} |  0
 .../files/expand-last-partition.sh            | 35 ++++++++++++++++---
 .../files/expand-on-first-boot.service        |  1 +
 testsuite/cibuilder.py                        | 28 ++++++++++++---
 12 files changed, 84 insertions(+), 9 deletions(-)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
 rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%)

-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure
  2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
@ 2022-12-15 14:27 ` henning.schild
  2022-12-15 15:32   ` Roberto A. Foglietta
  2022-12-15 14:27 ` [PATCH v5 2/6] expand-on-first-boot: support resizing a btrfs henning.schild
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild

From: Henning Schild <henning.schild@siemens.com>

We do hope the expand will work but we also know very well that it can
fail. The script is written in a way that currently only supports one
try. And not a retry if it failed in an earlier boot.
So make sure to disable the service also on failure.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 .../expand-on-first-boot/files/expand-on-first-boot.service      | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service b/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service
index fda50016e9fe..90c92a396b27 100644
--- a/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service
@@ -15,6 +15,7 @@ ConditionPathIsReadWrite=/etc
 Type=oneshot
 ExecStart=/usr/share/expand-on-first-boot/expand-last-partition.sh
 ExecStartPost=-/bin/systemctl disable expand-on-first-boot.service
+ExecStopPost=-/bin/systemctl disable expand-on-first-boot.service
 
 [Install]
 WantedBy=sysinit.target
-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v5 2/6] expand-on-first-boot: support resizing a btrfs
  2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
  2022-12-15 14:27 ` [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure henning.schild
@ 2022-12-15 14:27 ` henning.schild
  2022-12-15 17:48   ` Roberto A. Foglietta
  2022-12-15 14:27 ` [PATCH v5 3/6] meta-isar: introduce an example to use btrfs henning.schild
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild, Roberto A . Foglietta, Joe MacDonald

From: Henning Schild <henning.schild@siemens.com>

This adds support for resizing btrfs filesystems if they are in that
last partition. It also prepares for potentially other filesystems to
come in the future by introducing a switch-case.

The mounting logic is taken from the systemd-growfs patches we had to
revert again. Some filesystems need to online for resizing, but in order
to find the filesystem of a partition (without udev) mounting it and
letting the kernel detect seems a good idea.

Suggested-by: Roberto A. Foglietta <roberto.foglietta@gmail.com>
Suggested-by: Joe MacDonald <joe_macdonald@mentor.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 ...oot_1.3.bb => expand-on-first-boot_1.4.bb} |  0
 .../files/expand-last-partition.sh            | 35 ++++++++++++++++---
 2 files changed, 31 insertions(+), 4 deletions(-)
 rename meta/recipes-support/expand-on-first-boot/{expand-on-first-boot_1.3.bb => expand-on-first-boot_1.4.bb} (100%)

diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.3.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.4.bb
similarity index 100%
rename from meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.3.bb
rename to meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.4.bb
diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 93eddda2a3b7..753b8c33399c 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -62,8 +62,35 @@ if grep -q x-systemd.growfs /etc/fstab; then
 	exit 0
 fi
 
-# Do not fail resize2fs if no mtab entry is found, e.g.,
-# when using systemd mount units.
-export EXT2FS_NO_MTAB_OK=1
+# some filesystems need to be mounted i.e. btrfs, but mounting also helps
+# detect the filesystem type without having to wait for udev
+# mount $LAST_PART out of tree, so we won't conflict with other mounts
+MOUNT_POINT=$(mktemp -d --tmpdir "$(basename "$0").XXXXXXXXXX")
+if [ ! -d "${MOUNT_POINT}" ]; then
+	echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
+	exit 1
+fi
+mount "${LAST_PART}" "${MOUNT_POINT}"
+
+ret=0
+# Determine the filesystem type and perform the appropriate resize function
+FS_TYPE=$(findmnt -fno FSTYPE "${MOUNT_POINT}" )
+case ${FS_TYPE} in
+ext*)
+	# Do not fail resize2fs if no mtab entry is found, e.g.,
+	# when using systemd mount units.
+	export EXT2FS_NO_MTAB_OK=1
+	resize2fs "${LAST_PART}"
+	;;
+btrfs)
+	btrfs filesystem resize max "${MOUNT_POINT}"
+	;;
+*)
+	echo "Unrecognized filesystem type ${FS_TYPE} - no resize performed"
+	ret=1
+	;;
+esac
 
-resize2fs "${LAST_PART}"
+umount "${MOUNT_POINT}"
+rmdir "${MOUNT_POINT}"
+exit $ret
-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v5 3/6] meta-isar: introduce an example to use btrfs
  2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
  2022-12-15 14:27 ` [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure henning.schild
  2022-12-15 14:27 ` [PATCH v5 2/6] expand-on-first-boot: support resizing a btrfs henning.schild
@ 2022-12-15 14:27 ` henning.schild
  2022-12-15 14:27 ` [PATCH v5 4/6] CI: improve cibuilder readability henning.schild
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild

From: Henning Schild <henning.schild@siemens.com>

Switch debian bullseye over to btrfs rootfs and also to btrfs in last
partition to have a target where we test expand-on-first-boot for btrfs.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf                    | 5 +++++
 .../scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks     | 9 +++++++++
 2 files changed, 14 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index eca2628c4ae5..6b40cb50702e 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -13,6 +13,11 @@ IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 WKS_FILE_ubuntu-focal ?= "sdimage-efi-sd"
 IMAGER_INSTALL_remove_ubuntu-focal = "${GRUB_BOOTLOADER_INSTALL}"
 
+WKS_FILE_debian-bullseye ?= "sdimage-efi-btrfs"
+IMAGER_INSTALL_remove_debian-bullseye = "${GRUB_BOOTLOADER_INSTALL}"
+IMAGER_INSTALL_append_debian-bullseye = " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-progs"
+IMAGE_PREINSTALL_append_debian-bullseye = " btrfs-progs"
+
 IMAGE_INSTALL += "sshd-regen-keys"
 
 QEMU_ARCH ?= "x86_64"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
new file mode 100644
index 000000000000..014ecca61f0c
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
@@ -0,0 +1,9 @@
+# short-description: Create an EFI disk image without any swap and btrfs
+# long-description: Creates a partitioned EFI disk image without any swap that
+# the user can directly dd to boot media, where the last partition is btrfs.
+
+part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondisk sda --label efi --part-type EF00 --align 1024
+
+part / --source rootfs --ondisk sda --fstype btrfs --label platform --align 1024 --use-uuid --exclude-path boot/
+
+bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v5 4/6] CI: improve cibuilder readability
  2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
                   ` (2 preceding siblings ...)
  2022-12-15 14:27 ` [PATCH v5 3/6] meta-isar: introduce an example to use btrfs henning.schild
@ 2022-12-15 14:27 ` henning.schild
  2022-12-15 14:27 ` [PATCH v5 5/6] meta-isar: install expand-on-first-boot in most images and add space henning.schild
  2022-12-15 14:27 ` [PATCH v5 6/6] CI: expect a message about filesystem resize vom expand script henning.schild
  5 siblings, 0 replies; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild

From: Henning Schild <henning.schild@siemens.com>

What was called "servive_prompt" is not a prompt nor a service. Document
what that is and rename the variable.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 testsuite/cibuilder.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index d5c83b222d50..07a9edc56d9c 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -229,7 +229,8 @@ class CIBuilder(Test):
         self.log.info('QEMU boot line: ' + str(cmdline))
 
         login_prompt = b'isar login:'
-        service_prompt = b'Just an example'
+        # the printk of recipes-kernel/example-module
+        module_output = b'Just an example'
 
         timeout = time.time() + int(time_to_wait)
 
@@ -263,7 +264,7 @@ class CIBuilder(Test):
         if os.path.exists(output_file) and os.path.getsize(output_file) > 0:
             with open(output_file, "rb") as f1:
                 data = f1.read()
-                if service_prompt in data and login_prompt in data:
+                if module_output in data and login_prompt in data:
                     return
                 else:
                     app_log.error(data.decode(errors='replace'))
-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v5 5/6] meta-isar: install expand-on-first-boot in most images and add space
  2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
                   ` (3 preceding siblings ...)
  2022-12-15 14:27 ` [PATCH v5 4/6] CI: improve cibuilder readability henning.schild
@ 2022-12-15 14:27 ` henning.schild
  2022-12-15 14:27 ` [PATCH v5 6/6] CI: expect a message about filesystem resize vom expand script henning.schild
  5 siblings, 0 replies; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild

From: Henning Schild <henning.schild@siemens.com>

Here we take the most common and generic images and add some empty space
at the end of then.
This will enable better interactive use of our example images since
people will have some space to install some more packages. While the
space seems fixed it really is open end if the mass storage happens to be
bigger, because we install expand-on-first-boot as well.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/machine/qemuamd64.conf                       | 1 +
 meta-isar/conf/machine/virtualbox.conf                      | 1 +
 meta-isar/conf/machine/vmware.conf                          | 1 +
 meta-isar/conf/multiconfig/qemuarm-bookworm.conf            | 1 +
 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc | 5 +++++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks  | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks     | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks        | 2 ++
 8 files changed, 15 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc

diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index 6b40cb50702e..a1be5e5b113b 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -19,6 +19,7 @@ IMAGER_INSTALL_append_debian-bullseye = " ${SYSTEMD_BOOTLOADER_INSTALL} btrfs-pr
 IMAGE_PREINSTALL_append_debian-bullseye = " btrfs-progs"
 
 IMAGE_INSTALL += "sshd-regen-keys"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 QEMU_ARCH ?= "x86_64"
 QEMU_MACHINE ?= "q35"
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
index de3d04c3c4b4..8c3e9bacbf9b 100644
--- a/meta-isar/conf/machine/virtualbox.conf
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -10,6 +10,7 @@ KERNEL_NAME ?= "amd64"
 WKS_FILE ?= "sdimage-efi"
 
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 VMDK_SUBFORMAT = "monolithicSparse"
 IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
index fba639b19350..42503374742c 100644
--- a/meta-isar/conf/machine/vmware.conf
+++ b/meta-isar/conf/machine/vmware.conf
@@ -12,6 +12,7 @@ WKS_FILE ?= "sdimage-efi"
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 
 OVF_TEMPLATE_FILE ?= "vm-img-vmware.ovf.tmpl"
+IMAGE_INSTALL += "expand-on-first-boot"
 
 VMDK_SUBFORMAT = "streamOptimized"
 IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/multiconfig/qemuarm-bookworm.conf b/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
index 126935b06fda..0b9ecb664545 100644
--- a/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
+++ b/meta-isar/conf/multiconfig/qemuarm-bookworm.conf
@@ -7,3 +7,4 @@ IMAGE_FSTYPES_append = " wic"
 WKS_FILE ?= "sdimage-efi-sd"
 
 IMAGER_INSTALL += "${SYSTEMD_BOOTLOADER_INSTALL}"
+IMAGE_INSTALL += "expand-on-first-boot"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
new file mode 100644
index 000000000000..701fba764176
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc
@@ -0,0 +1,5 @@
+# some extra space we put in the end after the last partition
+# that allows testing expand-on-first-boot while at the same time
+# making our images more useful because they have space to play around
+
+part --source empty --no-table --ondisk sda --size 256M
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
index 014ecca61f0c..e22dba2f83f8 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondi
 
 part / --source rootfs --ondisk sda --fstype btrfs --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
index 754fbc46f8e4..7fe2953f5ff2 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-sd.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=systemd-boot" --ondi
 
 part / --source rootfs --ondisk sda --fstype ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
index f3addbc7515d..80c3a5359b1f 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
@@ -6,4 +6,6 @@ part /boot --source bootimg-efi-isar --sourceparams "loader=grub-efi" --ondisk s
 
 part / --source rootfs --ondisk sda --fstype ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid --exclude-path boot/
 
+include expand-padding.wks.inc
+
 bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v5 6/6] CI: expect a message about filesystem resize vom expand script
  2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
                   ` (4 preceding siblings ...)
  2022-12-15 14:27 ` [PATCH v5 5/6] meta-isar: install expand-on-first-boot in most images and add space henning.schild
@ 2022-12-15 14:27 ` henning.schild
  5 siblings, 0 replies; 9+ messages in thread
From: henning.schild @ 2022-12-15 14:27 UTC (permalink / raw)
  To: isar-users; +Cc: Henning Schild

From: Henning Schild <henning.schild@siemens.com>

We have added the expand-on-first-boot recipe and some space to grow to
our example images. So now any image using wic should "expand" on its
first boot. And the kernel actually leaves a message for us to read, at
least on debian but not on ubuntu.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 testsuite/cibuilder.py | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 07a9edc56d9c..8b67815442a8 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -231,6 +231,20 @@ class CIBuilder(Test):
         login_prompt = b'isar login:'
         # the printk of recipes-kernel/example-module
         module_output = b'Just an example'
+        resize_output = None
+
+        bb_output = start_vm.get_bitbake_env(arch, distro).decode()
+        wks_file = start_vm.get_bitbake_var(bb_output, 'WKS_FILE')
+        if wks_file:
+            bbdistro = start_vm.get_bitbake_var(bb_output, 'DISTRO')
+            # ubuntu is less verbose so we do not see the message
+            # /etc/sysctl.d/10-console-messages.conf
+            if bbdistro and "ubuntu" not in bbdistro:
+                if "sdimage-efi-sd" in wks_file:
+                    # output we see when expand-on-first-boot runs on ext4
+                    resize_output = b'resized filesystem to'
+                if "sdimage-efi-btrfs" in wks_file:
+                    resize_output = b': resize device '
 
         timeout = time.time() + int(time_to_wait)
 
@@ -265,8 +279,11 @@ class CIBuilder(Test):
             with open(output_file, "rb") as f1:
                 data = f1.read()
                 if module_output in data and login_prompt in data:
-                    return
-                else:
-                    app_log.error(data.decode(errors='replace'))
+                    if resize_output:
+                        if resize_output in data:
+                            return
+                    else:
+                        return
+                app_log.error(data.decode(errors='replace'))
 
         self.fail('Log ' + output_file)
-- 
2.37.4


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure
  2022-12-15 14:27 ` [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure henning.schild
@ 2022-12-15 15:32   ` Roberto A. Foglietta
  0 siblings, 0 replies; 9+ messages in thread
From: Roberto A. Foglietta @ 2022-12-15 15:32 UTC (permalink / raw)
  To: isar-users; +Cc: henning.schild, Joe MacDonald, Moessbauer, Felix

On Thu, 15 Dec 2022 at 15:28, <henning.schild@siemens.com> wrote:
>
> From: Henning Schild <henning.schild@siemens.com>
>
> We do hope the expand will work but we also know very well that it can
> fail. The script is written in a way that currently only supports one
> try. And not a retry if it failed in an earlier boot.
> So make sure to disable the service also on failure.

Disabling the expansion of last partition service at the first failure:

- decreases the chance that the human operator will notice the failure
in the boot logs and thus create a barrier to investigate that failure
and/or report it promptly.

- it is not compatible with the case of an unsupervised embedded
system that should not be able to go in service unless it is provided
with the full storage space.

- does not solve any real problem but just masquerading any related
issue and limits the options above.

My suggestion is to not apply this patch. I did a test and without
this patch the service continues to be active unless it completes
successfully.

I hope this helps, R-

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH v5 2/6] expand-on-first-boot: support resizing a btrfs
  2022-12-15 14:27 ` [PATCH v5 2/6] expand-on-first-boot: support resizing a btrfs henning.schild
@ 2022-12-15 17:48   ` Roberto A. Foglietta
  0 siblings, 0 replies; 9+ messages in thread
From: Roberto A. Foglietta @ 2022-12-15 17:48 UTC (permalink / raw)
  To: henning.schild; +Cc: isar-users, Joe MacDonald

On Thu, 15 Dec 2022 at 15:28, <henning.schild@siemens.com> wrote:

> +MOUNT_POINT=$(mktemp -d --tmpdir "$(basename "$0").XXXXXXXXXX")

Good choice to use the long option --tmpdir but it is still less
readable than -p "$TMPDIR"

> +if [ ! -d "${MOUNT_POINT}" ]; then
> +       echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2
> +       exit 1
> +fi

This is useless because set -e detected the failure also in child shells.

Just try these two commands:

roberto$ set -e
roberto$ test=$(false)

The terminal / shell will be closed immediately after pressing ENTER

> +mount "${LAST_PART}" "${MOUNT_POINT}"

If mktemp fails then also the mount will fail so whatever the shell
takes in account or not the subshell errors it is useless explicitly
check the mount point existence

> +
> +ret=0
> +# Determine the filesystem type and perform the appropriate resize function
> +FS_TYPE=$(findmnt -fno FSTYPE "${MOUNT_POINT}" )
> +case ${FS_TYPE} in
> +ext*)

Using ext*) seems legit but what about the introduction of extmyfs? A
completely different filesystem from ext[234]? I suggest ext[234]),
instead.

> +       # Do not fail resize2fs if no mtab entry is found, e.g.,
> +       # when using systemd mount units.
> +       export EXT2FS_NO_MTAB_OK=1
> +       resize2fs "${LAST_PART}"
> +       ;;
> +btrfs)
> +       btrfs filesystem resize max "${MOUNT_POINT}"Using ext*) seems legit but what about the introduction of extmyfs? A completely different filesystem from ext[234]? I suggest ext[234]), instead.
> +       ;;
> +*)
> +       echo "Unrecognized filesystem type ${FS_TYPE} - no resize performed"
> +       ret=1

Looking at your other patches this script has been added by default in
many templates. So, why consider a failure the execution of something
that the user might not even choose to use but just accepted by
default? Would it not be better to have a simple warning and nothing
else? Moreover, if the failure of this script does not deactivate its
service [as I hope] this script will run at every boot without any
reason. Adding an exit 0 here will also remove the use of ret variable
simplifying the code.

> +       ;;
> +esac
>
> -resize2fs "${LAST_PART}"
> +umount "${MOUNT_POINT}"
> +rmdir "${MOUNT_POINT}"
> +exit $ret
> --
> 2.37.4
>

Best regards, R-

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-12-15 17:48 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-15 14:27 [PATCH v5 0/6] expand-on-first-boot btrfs and CI testing henning.schild
2022-12-15 14:27 ` [PATCH v5 1/6] expand-on-first-boot: really only do that once, especially on failure henning.schild
2022-12-15 15:32   ` Roberto A. Foglietta
2022-12-15 14:27 ` [PATCH v5 2/6] expand-on-first-boot: support resizing a btrfs henning.schild
2022-12-15 17:48   ` Roberto A. Foglietta
2022-12-15 14:27 ` [PATCH v5 3/6] meta-isar: introduce an example to use btrfs henning.schild
2022-12-15 14:27 ` [PATCH v5 4/6] CI: improve cibuilder readability henning.schild
2022-12-15 14:27 ` [PATCH v5 5/6] meta-isar: install expand-on-first-boot in most images and add space henning.schild
2022-12-15 14:27 ` [PATCH v5 6/6] CI: expect a message about filesystem resize vom expand script henning.schild

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox