From b41bef462289422e9fedc8d87706e11f267b2c37 Mon Sep 17 00:00:00 2001 From: "Roberto A. Foglietta" Date: Mon, 12 Dec 2022 21:14:26 +0100 Subject: [PATCH] The expand last partition rework and generalisation The script has been reowrked to simplify its code: - the last partition free space is not computed anymore because useless - on exit a function take care of some final tasks and logs in /dev/kmsg and to extend its functionalities supporting new filesystems: - filesystem type is managed by a switch-case block of code - temporary directory to mount is created relying on TMPDIR - unsupervised systems have the option to reboot because error - if temporary directory creation fails 2nd try with /dev/shm - additional ext2, ext3 and btrfs filesystems supported - udev support: udevadm settle is the first task to run - unpartitioned volumes are resized as well (tested) This patch has been created by a collection of 7 patches to be applied on ilbers ISAR, branch next, commit d26660b724b034b602f3889f55a23cd9be2e87bd Author: Roberto A. Foglietta This code has been developed taking inspiration by suggestions and code from: Felix Moessbauer Tobias Schmidl Henning Schild Signed-off-by: Roberto A. Foglietta = 1/7 == In expand last partition wait for udev have finished ================= bugfix, expand last partition, wait for udev completion On some hardware the udev did not create the links to the disk partitions while this script is working thus it fails. This patch fixes those cases. v2: as underlined by Tobias this script requires systemd-udevd.service and it can safely run only after that service has been started otherwise it might fail. This dependency will reorder the boot procedure but not slow down it unless the hardware is slow to get mapped by udev but in that case resize the last partition would have failed. Thus, no regression is expected. = 2/7 == In expand last partition script btrfs support added ================== Improvement, expand-on-first-boot: support for btrfs added v2: the mount point for btrfs filesystem to resize is under /dev/shm by default but a variable defined in the running environment can change the temporary directory path. The /dev/shm has been chosen because in some systems /tmp can be on a RO root filesystem and /tmp not yet mounted with tmpfs. This requires the system to have /dev/shm configured into the kernel. v3: the previous version was using a bashism but the shebang is /bin/sh this means that the script will fail in a customized system in which /bin/sh do not link to bash or bash is not available. v4: Felix suggested using mktemp for the temporary directory in which to mount the btrfs filesystem to resize. The default path /tmp could be changed using the variable TMPDIR to set in the running environment. v5: typo fixes into the patch description v6: mktemp does a reasonable second try with -p /dev/shm before failing = 3/7 == If expand last partition fails, unsupervised systems reboot ========== Improvement for unsupervised embedded devices which need to reboot if expanding last partition fails because an under-provided system should not even complete the boot in the field in order to avoid functioning failures at unexpected future time. This patch do not affect the previous behaviour because requires that /etc/unsupervised file exists. v2: if there is no partition table exit 0 instead of exit 1 = 4/7 == In expand last partition checking the size is useless ================ code maintenance, superfluous operations commented, kept for the future Tests shown that resize in full a ext4 or btrfs partition/filesystem does not hurt and does not report a failure, so nothing changes without the size check = 5/7 == The expand last partition might support full volumes also, to test == Improvement, expansion of unpartitioned volumes supported also but to test DONE, 14.12.2022: this change should be tested with a single volume system preliminary tests shown that ext4 and btrfs entire volumes cam be resized = 6/7 == In expand last partition trap exitnlog EXIT logs with simpler code === Improvement, the script log at exit with the status of completion and its full path in this way - in case of failure - the user will find in dmesg the error and also the script full path to debug. It uses /dev/kmsg to log. Code simplified. = 7/7 == In expand last partition few improvements and a bugfix =============== Some improvements and a bugfix in expanding last partition. Improvements: filesystems ext2 and ext3 also supported, warning in dmesg in case of unsupported filesystem, the variable tmpdir is used with double quotes even if unneeded but be consistent with the original script notation. Bugfix: even if btrfs resize fails, umount the last partition and return error. --- .../expand-on-first-boot_1.2.bb | 2 +- .../files/expand-last-partition.sh | 134 ++++++++++-------- .../files/expand-on-first-boot.service | 2 + 3 files changed, 77 insertions(+), 61 deletions(-) diff --git a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb index 0996000..fe7b72b 100644 --- a/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb +++ b/meta/recipes-support/expand-on-first-boot/expand-on-first-boot_1.2.bb @@ -10,7 +10,7 @@ inherit dpkg-raw DESCRIPTION = "This service grows the last partition to the full medium during first boot" MAINTAINER = "isar-users " -DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux" +DEBIAN_DEPENDS = "systemd, sed, grep, coreutils, mount, e2fsprogs, fdisk (>=2.29.2-3) | util-linux (<2.29.2-3), util-linux, btrfs-progs" SRC_URI = " \ file://expand-on-first-boot.service \ diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh index 57055cc..5cd6fc3 100755 --- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh +++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh @@ -7,8 +7,25 @@ # # SPDX-License-Identifier: MIT +exitnlog() { + ec=$? + set +e + if [ "$ec" != "0" ]; then + echo "ERROR: $0 failed" + else + echo "$0 succeded" + fi >/dev/kmsg + if [ "$ec" != "0" -a -e /etc/unsupervised ]; then + reboot + fi +} +export -f exitnlog +trap exitnlog EXIT + set -e +udevadm settle + ROOT_DEV="$(findmnt / -o source -n)" ROOT_DEV_NAME=${ROOT_DEV##*/} ROOT_DEV_SLAVE=$(find /sys/block/"${ROOT_DEV_NAME}"/slaves -mindepth 1 -print -quit 2>/dev/null || true) @@ -16,70 +33,67 @@ if [ -n "${ROOT_DEV_SLAVE}" ]; then ROOT_DEV=/dev/${ROOT_DEV_SLAVE##*/} fi -BOOT_DEV="$(echo "${ROOT_DEV}" | sed 's/p\?[0-9]*$//')" -if [ "${ROOT_DEV}" = "${BOOT_DEV}" ]; then - echo "Boot device equals root device - no partitioning found" >&2 - exit 1 -fi - -# this value is in blocks. Normally a block has 512 bytes. -BUFFER_SIZE=32768 -BOOT_DEV_NAME=${BOOT_DEV##*/} -DISK_SIZE="$(cat /sys/class/block/"${BOOT_DEV_NAME}"/size)" -ALL_PARTS_SIZE=0 -for PARTITION in /sys/class/block/"${BOOT_DEV_NAME}"/"${BOOT_DEV_NAME}"*; do - PART_SIZE=$(cat "${PARTITION}"/size) - ALL_PARTS_SIZE=$((ALL_PARTS_SIZE + PART_SIZE)) -done - -MINIMAL_SIZE=$((ALL_PARTS_SIZE + BUFFER_SIZE)) -if [ "$DISK_SIZE" -lt "$MINIMAL_SIZE" ]; then - echo "Disk is practically already full, doing nothing." >&2 - exit 0 -fi +# full resizing of ext4 and btrfs does not fail nor hurt but supporting more +# filesystems in future might change this condition, so commenting this code +useless_for_now() { + # this value is in blocks. Normally a block has 512 bytes. + BUFFER_SIZE=32768 + BOOT_DEV_NAME=${BOOT_DEV##*/} + DISK_SIZE="$(cat /sys/class/block/"${BOOT_DEV_NAME}"/size)" + ALL_PARTS_SIZE=0 + for PARTITION in /sys/class/block/"${BOOT_DEV_NAME}"/"${BOOT_DEV_NAME}"*; do + PART_SIZE=$(cat "${PARTITION}"/size) + ALL_PARTS_SIZE=$((ALL_PARTS_SIZE + PART_SIZE)) + done -LAST_PART="$(sfdisk -d "${BOOT_DEV}" 2>/dev/null | tail -1 | cut -d ' ' -f 1)" + MINIMAL_SIZE=$((ALL_PARTS_SIZE + BUFFER_SIZE)) + if [ "$DISK_SIZE" -lt "$MINIMAL_SIZE" ]; then + echo "Disk is practically already full, doing nothing." >&2 + exit 0 + fi +} -# Transform the partition table as follows: -# -# - Remove any 'last-lba' header so sfdisk uses the entire available space. -# - If this partition table is MBR and an extended partition container (EBR) -# exists, we assume this needs to be expanded as well; remove its size -# field so sfdisk expands it. -# - For the previously fetched last partition, also remove the size field so -# sfdisk expands it. -sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ - grep -v last-lba | \ - sed 's|^\(.*, \)size=[^,]*, \(type=[f5]\)$|\1\2|' | \ - sed 's|^\('"${LAST_PART}"' .*, \)size=[^,]*, |\1|' | \ - sfdisk --force "${BOOT_DEV}" - -# Inform the kernel about the partitioning change -partx -u "${LAST_PART}" - -# this is for debian stretch or systemd < 236 -if [ ! -x /lib/systemd/systemd-growfs ]; then - # Do not fail resize2fs if no mtab entry is found, e.g., - # when using systemd mount units. - export EXT2FS_NO_MTAB_OK=1 +# TODO, 14.12.2022: this change should be tested with a single volume system +# preliminary tests shown that ext4 and btrfs entire volumes cam be resized +BOOT_DEV="$(echo "${ROOT_DEV}" | sed 's/p\?[0-9]*$//')" +if [ "${ROOT_DEV}" = "${BOOT_DEV}" ]; then + LAST_PART="${BOOT_DEV}" +else + LAST_PART="$(sfdisk -d "${BOOT_DEV}" 2>/dev/null | tail -1 | cut -d ' ' -f 1)" - resize2fs "${LAST_PART}" - exit 0 -fi + # Transform the partition table as follows: + # + # - Remove any 'last-lba' header so sfdisk uses the entire available space. + # - If this partition table is MBR and an extended partition container (EBR) + # exists, we assume this needs to be expanded as well; remove its size + # field so sfdisk expands it. + # - For the previously fetched last partition, also remove the size field so + # sfdisk expands it. + sfdisk -d "${BOOT_DEV}" 2>/dev/null | \ + grep -v last-lba | \ + sed 's|^\(.*, \)size=[^,]*, \(type=[f5]\)$|\1\2|' | \ + sed 's|^\('"${LAST_PART}"' .*, \)size=[^,]*, |\1|' | \ + sfdisk --force "${BOOT_DEV}" -if grep -q x-systemd.growfs /etc/fstab; then - echo "Found x-systemd.growfs option in /etc/fstab, won't call it explicitly." >&2 - exit 0 + # Inform the kernel about the partitioning change + partx -u "${LAST_PART}" fi -# mount $LAST_PART out of tree, so we won't conflict with other mounts -MOUNT_POINT=$(mktemp -d -p /mnt "$(basename "$0").XXXXXXXXXX") -if [ ! -d "${MOUNT_POINT}" ]; then - echo "Cannot create temporary mount point ${MOUNT_POINT}." >&2 - exit 1 -fi +# Do not fail resize2fs if no mtab entry is found, e.g., +# when using systemd mount units. +export EXT2FS_NO_MTAB_OK=1 -mount "${LAST_PART}" "${MOUNT_POINT}" -/lib/systemd/systemd-growfs "${MOUNT_POINT}" -umount "${MOUNT_POINT}" -rmdir "${MOUNT_POINT}" +case $(lsblk -fno FSTYPE "${LAST_PART}") in + ext[234]) resize2fs "${LAST_PART}" + ;; + btrfs) err=0 + tmpdir=$(mktemp -d -p "$TMPDIR" btrfs.XXXX || \ + mktemp -d -p "/dev/shm" btrfs.XXXX) + mount "${LAST_PART}" "$tmpdir" + btrfs filesystem resize max "$tmpdir" || err=1 + umount "$tmpdir" && rmdir "$tmpdir" + exit $err + ;; + *) echo "WARNING: $0 unsupported filesystem" >/dev/kmsg || true + ;; +esac diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service b/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service index fda5001..2c120c4 100644 --- a/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service +++ b/meta/recipes-support/expand-on-first-boot/files/expand-on-first-boot.service @@ -7,7 +7,9 @@ Description=Expand last partition DefaultDependencies=no Conflicts=shutdown.target +Requires=systemd-udevd.service After=systemd-remount-fs.service +After=systemd-udevd.service Before=local-fs-pre.target shutdown.target ConditionPathIsReadWrite=/etc -- 2.34.1