Re: [PATCH 03/17] meta-isar: deploy-image: Change reboot logic

[Uladzimir Bely <ubely@ilbers.de>]

On Tue, 2024-07-02 at 15:38 +0200, 'Jan Kiszka' via isar-users wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> Pull the reboot out of the script. This allows for cleaner
> integration
> with different calling environment, may they be a systemd unit, an
> initramfs script or simply a shell for testing purposes.
> 
> And if the script exits with an error, wait a minute before rebooting
> the system, rather than just trying to re-execute it. This permits to
> inspect potential error as well.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  .../recipes-installer/deploy-image/files/deploy-image-wic.sh    | 2
> +-
>  .../recipes-installer/deploy-image/files/install.override.conf  | 2
> +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/meta-isar/recipes-installer/deploy-image/files/deploy-
> image-wic.sh b/meta-isar/recipes-installer/deploy-image/files/deploy-
> image-wic.sh
> index 8043aff1..12c1eea2 100644
> --- a/meta-isar/recipes-installer/deploy-image/files/deploy-image-
> wic.sh
> +++ b/meta-isar/recipes-installer/deploy-image/files/deploy-image-
> wic.sh
> @@ -105,4 +105,4 @@ fi
>  umount "$installdata"
>  sync
>  dialog --title "Reboot" --msgbox "Installation is successful. System
> will be rebooted. Please remove the USB stick." 7 60
> -reboot
> +exit 0
> diff --git a/meta-isar/recipes-installer/deploy-
> image/files/install.override.conf b/meta-isar/recipes-
> installer/deploy-image/files/install.override.conf
> index 73874caa..357d8662 100644
> --- a/meta-isar/recipes-installer/deploy-
> image/files/install.override.conf
> +++ b/meta-isar/recipes-installer/deploy-
> image/files/install.override.conf
> @@ -1,5 +1,5 @@
>  [Service]
>  ExecStart=
> -ExecStart=/usr/bin/deploy-image-wic.sh
> +ExecStart=/bin/sh -c "deploy-image-wic.sh || (echo 'Rebooting in 60
> s'; sleep 60); reboot"

Hello Jan.

This change seems to cause a sort of race issue.

Few days ago "[PATCH v3 0/2] Cover installer image with tests" was
merged. Tests were OK (and "next" is still OK) on two jenkins and one
gitlab instances. But on the third jenkins instance, they were failing.

Boot log:

```
Got config:
  installer_unattended=true
  installer_image_uri=/install/isar-image-ci-debian-bookworm-
qemuamd64.wic.zst
  installer_target_dev=/dev/sda
  installer_target_overwrite=OVERWRITE
bmaptool: ERROR: An error occurred, here is the traceback:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/bmaptools/CLI.py", line 116, in
open_block_device
    descriptor = os.open(path, os.O_WRONLY | os.O_EXCL)

bmaptool: ERROR: cannot open block device '/dev/sda' in exclusive mode:
[Errno 16] Device or resource busy: '/dev/sda'

Rebooting in 60 s
```

But either machine is not booted, or there are some "ext4" errors
happen.

I couldn't understand few things:
 - why this happens (unable to get exclusive access to block device)$
 - why device is written even if there is such error in logs$
 - why sometimes we don't see this error in logs$
 - why this doesn't happen when "unattended mode" is off.

The reason is that we have these two "getty" instances overriden by
custom one having the race. While "serial-getty@ttyS0.service.d" does
the job, "getty@tty1.service.d" produces this error visible in logs and
reboot machine in 60 seconds.

In some cases 60sec is enough for first service to finish writing to
the disc, but on slower machines we result in incomplete copy and
broken system.

Before this patch, when "reboot" calls were in the script, "failed"
instance simply exited and didn't try to reboot the board.

As a quick fix, I would reverted this patch.

Also, I think this behaviour may impact on bmap-tools "psplash" patches
currently we have on maillist. I would expect that user won't see the
progress bar on screen because it's somewhere on serial console.


>  StandardInput=tty
>  StandardOutput=tty
> -- 
> 2.43.0
> 

-- 
Best regards,
Uladzimir.



-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/3fd6bdc491628229686855d3278b8d5684643108.camel%40ilbers.de.