WARNING: expand-on-first boot might shrink your partition on Debian bookworm

WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Florian Bezdeka]

Hi all,

here comes another Debian bookworm related problem. This time related to
expand-on-first-boot, which destroys my image during the fist boot.

The linux-util packages has been updated and includes some changes
related to partition alignment. See [1]. (Thanks Henning for pointing me
here.) It might happen now that the last partition is shrinked instead
of expanded.

This is especially relevant for setups where the last partition is the
root partition and where no more space is available (e.g. booting up a
wic image with qemu where disk size = image size)

The original partition table looked like this:

label: gpt
label-id: 8CC4BCD4-F7B9-45F1-A066-DFB4068D6DFC
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 4554194
sector-size: 512

After expand-on-first-boot it looks like that:
(note the last-lba field)

label: gpt
label-id: 5305C98F-17AD-4B41-9388-7DC319741C9D
device: talker-debian-bookworm-qemuamd64.wic
unit: sectors
first-lba: 34
last-lba: 4554320
sector-size: 512

Booting up again stops in the initrd with the following error:

(initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block count
550586 exceeds size of device (550400 blocks)

resize2fs called from expand-last-partition.sh already throws a warning
resize2fs: New size smaller than minimum (550586)

Ideas welcome...
Taking over even more...

Best regards,
Florian


[1]
https://github.com/util-linux/util-linux/commit/921c7da55ec78350e4067b3fd6b7de6f299106ee

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Schmidl, Tobias]

Hi Florian,

Am Donnerstag, dem 07.07.2022 um 09:48 +0200 schrieb Florian Bezdeka:
> Hi all,
> 
> here comes another Debian bookworm related problem. This time related to
> expand-on-first-boot, which destroys my image during the fist boot.
> 

will look into it.

Kind regards,

Tobias

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Henning Schild]

Am Thu, 7 Jul 2022 09:48:22 +0200
schrieb Florian Bezdeka <florian.bezdeka@siemens.com>:

> Hi all,
> 
> here comes another Debian bookworm related problem. This time related
> to expand-on-first-boot, which destroys my image during the fist boot.
> 
> The linux-util packages has been updated and includes some changes
> related to partition alignment. See [1]. (Thanks Henning for pointing
> me here.) It might happen now that the last partition is shrinked
> instead of expanded.
> 
> This is especially relevant for setups where the last partition is the
> root partition and where no more space is available (e.g. booting up a
> wic image with qemu where disk size = image size)
> 
> The original partition table looked like this:
> 
> label: gpt
> label-id: 8CC4BCD4-F7B9-45F1-A066-DFB4068D6DFC
> device: /dev/sda
> unit: sectors
> first-lba: 34
> last-lba: 4554194
> sector-size: 512
> 
> After expand-on-first-boot it looks like that:
> (note the last-lba field)
> 
> label: gpt
> label-id: 5305C98F-17AD-4B41-9388-7DC319741C9D
> device: talker-debian-bookworm-qemuamd64.wic
> unit: sectors
> first-lba: 34
> last-lba: 4554320
> sector-size: 512
> 
> Booting up again stops in the initrd with the following error:
> 
> (initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block count
> 550586 exceeds size of device (550400 blocks)
> 
> resize2fs called from expand-last-partition.sh already throws a
> warning resize2fs: New size smaller than minimum (550586)
> 
> Ideas welcome...
> Taking over even more...

I think one idea as a quick hack in an affected layer could be to add
more EXTRA_SPACE, but i did not try that. With more space a slight
shrinking might just work. Or put a large fixes size in your wks ... or
play with alignment there to avoid shrinking.

And the good solution would probably to detect that the disk is already
"close to full" and simply step out early.

if /sys/class/block/sda/size - sum(/sys/class/block/sda/sda*/size) < 32M
echo "Nothing to do"; exit 0

Where 32M is made up, one should look up what the maximum shrink could
be. I think we could even say 4M. Or we could use percentage ... if sum
>= 0.95*size

Henning

> Best regards,
> Florian
> 
> 
> [1]
> https://github.com/util-linux/util-linux/commit/921c7da55ec78350e4067b3fd6b7de6f299106ee
> 

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Bezdeka, Florian]

On Thu, 2022-07-07 at 10:54 +0200, Henning Schild wrote:
> Am Thu, 7 Jul 2022 09:48:22 +0200
> schrieb Florian Bezdeka <florian.bezdeka@siemens.com>:
> 
> > Hi all,
> > 
> > here comes another Debian bookworm related problem. This time related
> > to expand-on-first-boot, which destroys my image during the fist boot.
> > 
> > The linux-util packages has been updated and includes some changes
> > related to partition alignment. See [1]. (Thanks Henning for pointing
> > me here.) It might happen now that the last partition is shrinked
> > instead of expanded.
> > 
> > This is especially relevant for setups where the last partition is the
> > root partition and where no more space is available (e.g. booting up a
> > wic image with qemu where disk size = image size)
> > 
> > The original partition table looked like this:
> > 
> > label: gpt
> > label-id: 8CC4BCD4-F7B9-45F1-A066-DFB4068D6DFC
> > device: /dev/sda
> > unit: sectors
> > first-lba: 34
> > last-lba: 4554194
> > sector-size: 512
> > 
> > After expand-on-first-boot it looks like that:
> > (note the last-lba field)
> > 
> > label: gpt
> > label-id: 5305C98F-17AD-4B41-9388-7DC319741C9D
> > device: talker-debian-bookworm-qemuamd64.wic
> > unit: sectors
> > first-lba: 34
> > last-lba: 4554320
> > sector-size: 512
> > 
> > Booting up again stops in the initrd with the following error:
> > 
> > (initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block count
> > 550586 exceeds size of device (550400 blocks)
> > 
> > resize2fs called from expand-last-partition.sh already throws a
> > warning resize2fs: New size smaller than minimum (550586)
> > 
> > Ideas welcome...
> > Taking over even more...
> 
> I think one idea as a quick hack in an affected layer could be to add
> more EXTRA_SPACE, but i did not try that. With more space a slight
> shrinking might just work. Or put a large fixes size in your wks ... or
> play with alignment there to avoid shrinking.

I played around with that as well, the problem is that you have to find
a matching value for EXTRA_SPACE to avoid the alignment. Once found it
may break again with one more package added to the image.

> 
> And the good solution would probably to detect that the disk is already
> "close to full" and simply step out early.
> 
> if /sys/class/block/sda/size - sum(/sys/class/block/sda/sda*/size) < 32M
> echo "Nothing to do"; exit 0

I like the idea but keep in mind that such sysfs or procfs entries
might not exist in all cases. There might be custom kernel
configurations out there with such features disabled. Might not be a
real world problem, but if so we would break such setups. So far we
only had dependencies to user space components.

> 
> Where 32M is made up, one should look up what the maximum shrink could
> be. I think we could even say 4M. Or we could use percentage ... if sum
> > = 0.95*size
> 
> Henning
> 
> > Best regards,
> > Florian
> > 
> > 
> > [1]
> > https://github.com/util-linux/util-linux/commit/921c7da55ec78350e4067b3fd6b7de6f299106ee
> > 
> 

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Henning Schild]

Am Thu, 7 Jul 2022 12:51:55 +0200
schrieb "Bezdeka, Florian (T CED SES-DE)" <florian.bezdeka@siemens.com>:

> On Thu, 2022-07-07 at 10:54 +0200, Henning Schild wrote:
> > Am Thu, 7 Jul 2022 09:48:22 +0200
> > schrieb Florian Bezdeka <florian.bezdeka@siemens.com>:
> >   
> > > Hi all,
> > > 
> > > here comes another Debian bookworm related problem. This time
> > > related to expand-on-first-boot, which destroys my image during
> > > the fist boot.
> > > 
> > > The linux-util packages has been updated and includes some changes
> > > related to partition alignment. See [1]. (Thanks Henning for
> > > pointing me here.) It might happen now that the last partition is
> > > shrinked instead of expanded.
> > > 
> > > This is especially relevant for setups where the last partition
> > > is the root partition and where no more space is available (e.g.
> > > booting up a wic image with qemu where disk size = image size)
> > > 
> > > The original partition table looked like this:
> > > 
> > > label: gpt
> > > label-id: 8CC4BCD4-F7B9-45F1-A066-DFB4068D6DFC
> > > device: /dev/sda
> > > unit: sectors
> > > first-lba: 34
> > > last-lba: 4554194
> > > sector-size: 512
> > > 
> > > After expand-on-first-boot it looks like that:
> > > (note the last-lba field)
> > > 
> > > label: gpt
> > > label-id: 5305C98F-17AD-4B41-9388-7DC319741C9D
> > > device: talker-debian-bookworm-qemuamd64.wic
> > > unit: sectors
> > > first-lba: 34
> > > last-lba: 4554320
> > > sector-size: 512
> > > 
> > > Booting up again stops in the initrd with the following error:
> > > 
> > > (initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block
> > > count 550586 exceeds size of device (550400 blocks)
> > > 
> > > resize2fs called from expand-last-partition.sh already throws a
> > > warning resize2fs: New size smaller than minimum (550586)
> > > 
> > > Ideas welcome...
> > > Taking over even more...  
> > 
> > I think one idea as a quick hack in an affected layer could be to
> > add more EXTRA_SPACE, but i did not try that. With more space a
> > slight shrinking might just work. Or put a large fixes size in your
> > wks ... or play with alignment there to avoid shrinking.  
> 
> I played around with that as well, the problem is that you have to
> find a matching value for EXTRA_SPACE to avoid the alignment. Once
> found it may break again with one more package added to the image.

Ok, i was assuming a minor shrinking of both partition and filesystem
can work as long as the filesystem has space left to shrink.

> > 
> > And the good solution would probably to detect that the disk is
> > already "close to full" and simply step out early.
> > 
> > if /sys/class/block/sda/size - sum(/sys/class/block/sda/sda*/size)
> > < 32M echo "Nothing to do"; exit 0  
> 
> I like the idea but keep in mind that such sysfs or procfs entries
> might not exist in all cases. There might be custom kernel
> configurations out there with such features disabled. Might not be a
> real world problem, but if so we would break such setups. So far we
> only had dependencies to user space components.

No problem! The script is set -e and we will fail without breaking
anything, unfortunately also without doing its job. If people want the
feature they can then dig out what they might be missing in their
configs. That is the price to pay when using weird configs ;).

Henning

> > 
> > Where 32M is made up, one should look up what the maximum shrink
> > could be. I think we could even say 4M. Or we could use percentage
> > ... if sum  
> > > = 0.95*size  
> > 
> > Henning
> >   
> > > Best regards,
> > > Florian
> > > 
> > > 
> > > [1]
> > > https://github.com/util-linux/util-linux/commit/921c7da55ec78350e4067b3fd6b7de6f299106ee
> > >   
> >   
> 

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Henning Schild]

Am Thu, 7 Jul 2022 09:48:22 +0200
schrieb Florian Bezdeka <florian.bezdeka@siemens.com>:

> Hi all,
> 
> here comes another Debian bookworm related problem. This time related
> to expand-on-first-boot, which destroys my image during the fist boot.
> 
> The linux-util packages has been updated and includes some changes
> related to partition alignment. See [1]. (Thanks Henning for pointing
> me here.) It might happen now that the last partition is shrinked
> instead of expanded.
> 
> This is especially relevant for setups where the last partition is the
> root partition and where no more space is available (e.g. booting up a
> wic image with qemu where disk size = image size)
> 
> The original partition table looked like this:
> 
> label: gpt
> label-id: 8CC4BCD4-F7B9-45F1-A066-DFB4068D6DFC
> device: /dev/sda
> unit: sectors
> first-lba: 34
> last-lba: 4554194
> sector-size: 512
> 
> After expand-on-first-boot it looks like that:
> (note the last-lba field)
> 
> label: gpt
> label-id: 5305C98F-17AD-4B41-9388-7DC319741C9D
> device: talker-debian-bookworm-qemuamd64.wic
> unit: sectors
> first-lba: 34
> last-lba: 4554320
> sector-size: 512
> 
> Booting up again stops in the initrd with the following error:
> 
> (initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block count
> 550586 exceeds size of device (550400 blocks)
> 
> resize2fs called from expand-last-partition.sh already throws a
> warning resize2fs: New size smaller than minimum (550586)

btw systemd-growfs would run into the same problem, fail to shrink ...
brick

Henning

> Ideas welcome...
> Taking over even more...
> 
> Best regards,
> Florian
> 
> 
> [1]
> https://github.com/util-linux/util-linux/commit/921c7da55ec78350e4067b3fd6b7de6f299106ee
> 

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Schmidl, Tobias]

Hi all,

Am Donnerstag, dem 07.07.2022 um 10:59 +0200 schrieb Henning Schild:
> 
> btw systemd-growfs would run into the same problem, fail to shrink ...
> brick
> 

Well, it doesn't *brick*, but the script nevertheless fails.

Kind regards,

Tobias

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Henning Schild]

Am Thu, 7 Jul 2022 11:30:15 +0200
schrieb "Schmidl, Tobias (T CED SES-DE)" <tobiasschmidl@siemens.com>:

> Hi all,
> 
> Am Donnerstag, dem 07.07.2022 um 10:59 +0200 schrieb Henning Schild:
> > 
> > btw systemd-growfs would run into the same problem, fail to shrink
> > ... brick
> >   
> 
> Well, it doesn't *brick*, but the script nevertheless fails.

If you reboot you will end up in a rescue shell. Partition being
smaller than filesystem inside. ... mount fails.

Henning

> Kind regards,
> 
> Tobias
> 

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Schmidl, Tobias]

Am Donnerstag, dem 07.07.2022 um 09:48 +0200 schrieb Florian Bezdeka:
> Hi all,
> 
> here comes another Debian bookworm related problem. This time related to
> expand-on-first-boot, which destroys my image during the fist boot.
> 
> The linux-util packages has been updated and includes some changes
> related to partition alignment. See [1]. (Thanks Henning for pointing me
> here.) It might happen now that the last partition is shrinked instead
> of expanded.
> 
> […]
> 
> Booting up again stops in the initrd with the following error:
> 
> (initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block count
> 550586 exceeds size of device (550400 blocks)
> 
> resize2fs called from expand-last-partition.sh already throws a warning
> resize2fs: New size smaller than minimum (550586)
> 
> 


I might have a solution:


diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
index 31f1ae3..5da9f7a 100755
--- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
+++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
@@ -22,6 +22,23 @@ if [ "${ROOT_DEV}" = "${BOOT_DEV}" ]; then
        exit 1
 fi
 
+# this value is purely speculative
+BUFFER_SIZE=16384
+BOOT_DEV_NAME=${BOOT_DEV##*/}
+DISK_SIZE="$(cat /sys/class/block/"${BOOT_DEV_NAME}"/size)"
+ALL_PARTS_SIZE=0
+for PARTITION in /sys/class/block/"${BOOT_DEV_NAME}"[1-9]*; do
+       PART_SIZE=$(cat "${PARTITION}"/size)
+       ALL_PARTS_SIZE=$((ALL_PARTS_SIZE + PART_SIZE))
+done
+echo "Disk ${BOOT_DEV}: ${DISK_SIZE}, all partitions combined: ${ALL_PARTS_SIZE}" >&2
+
+MINIMAL_SIZE=$((ALL_PARTS_SIZE + BUFFER_SIZE))
+if [ "$DISK_SIZE" -lt "$MINIMAL_SIZE" ]; then
+       echo "Disk is not big enough, won't resize. Current size: $DISK_SIZE, minimal size for resizing: $MINIMAL_SIZE" >&2
+       #exit 0
+fi
+
 LAST_PART="$(sfdisk -d "${BOOT_DEV}" 2>/dev/null | tail -1 | cut -d ' ' -f 1)"
 
 # Transform the partition table as follows:


The problem is - the numbers don't really add up. We see a difference of
128 blocks - that IMHO isn't reflected anywhere in the code.

I would even opt for bigger numbers, maybe 32k blocks. If the difference
in size is so low, the benefit of resizing is marginal.

Kind regards,

Tobias

Re: WARNING: expand-on-first boot might shrink your partition on Debian bookworm

[Jan Kiszka]

On 08.07.22 08:35, Schmidl, Tobias (T CED SES-DE) wrote:
> Am Donnerstag, dem 07.07.2022 um 09:48 +0200 schrieb Florian Bezdeka:
>> Hi all,
>>
>> here comes another Debian bookworm related problem. This time related to
>> expand-on-first-boot, which destroys my image during the fist boot.
>>
>> The linux-util packages has been updated and includes some changes
>> related to partition alignment. See [1]. (Thanks Henning for pointing me
>> here.) It might happen now that the last partition is shrinked instead
>> of expanded.
>>
>> […]
>>
>> Booting up again stops in the initrd with the following error:
>>
>> (initramfs) [    1.978569] EXT4-fs (sda2): bad geometry: block count
>> 550586 exceeds size of device (550400 blocks)
>>
>> resize2fs called from expand-last-partition.sh already throws a warning
>> resize2fs: New size smaller than minimum (550586)
>>
>>
> 
> 
> I might have a solution:
> 
> 
> diff --git a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> index 31f1ae3..5da9f7a 100755
> --- a/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> +++ b/meta/recipes-support/expand-on-first-boot/files/expand-last-partition.sh
> @@ -22,6 +22,23 @@ if [ "${ROOT_DEV}" = "${BOOT_DEV}" ]; then
>         exit 1
>  fi
> 
> +# this value is purely speculative
> +BUFFER_SIZE=16384
> +BOOT_DEV_NAME=${BOOT_DEV##*/}
> +DISK_SIZE="$(cat /sys/class/block/"${BOOT_DEV_NAME}"/size)"
> +ALL_PARTS_SIZE=0
> +for PARTITION in /sys/class/block/"${BOOT_DEV_NAME}"[1-9]*; do
> +       PART_SIZE=$(cat "${PARTITION}"/size)
> +       ALL_PARTS_SIZE=$((ALL_PARTS_SIZE + PART_SIZE))
> +done
> +echo "Disk ${BOOT_DEV}: ${DISK_SIZE}, all partitions combined: ${ALL_PARTS_SIZE}" >&2
> +
> +MINIMAL_SIZE=$((ALL_PARTS_SIZE + BUFFER_SIZE))
> +if [ "$DISK_SIZE" -lt "$MINIMAL_SIZE" ]; then
> +       echo "Disk is not big enough, won't resize. Current size: $DISK_SIZE, minimal size for resizing: $MINIMAL_SIZE" >&2
> +       #exit 0
> +fi
> +
>  LAST_PART="$(sfdisk -d "${BOOT_DEV}" 2>/dev/null | tail -1 | cut -d ' ' -f 1)"
> 
>  # Transform the partition table as follows:
> 
> 
> The problem is - the numbers don't really add up. We see a difference of
> 128 blocks - that IMHO isn't reflected anywhere in the code.
> 
> I would even opt for bigger numbers, maybe 32k blocks. If the difference
> in size is so low, the benefit of resizing is marginal.
> 

Yeah, everything below some MB is not worth trying to change the size.
It rather usual a signal that something was forgotten, either to remove
this service from the image or resize it after creation if it is used
for a VM or an otherwise emulated storage (USB gadget).

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux