Re: [PATCH v4 4/5] ci: Add test cases for container fetching and loading

[Uladzimir Bely <ubely@ilbers.de>]

On Tue, 2024-07-30 at 23:45 +0200, Jan Kiszka wrote:
> On 30.07.24 09:44, Uladzimir Bely wrote:
> > On Fri, 2024-07-19 at 18:38 +0200, 'Jan Kiszka' via isar-users
> > wrote:
> > > From: Jan Kiszka <jan.kiszka@siemens.com>
> > > 
> > > This plugs the two example recipes for loading container images
> > > into
> > > VM-based testing. The test consists of running 'true' in the
> > > installed
> > > alpine images.
> > > 
> > > Rather than enabling the ci user to do password-less sudo, this
> > > uses
> > > su
> > > with the piped-in password. Another trick needed is to poll for
> > > the
> > > images because loading is performed asynchronously.
> > > 
> > > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> > > ---
> > >  .../recipes-core/images/isar-image-ci.bb      |  2 ++
> > >  testsuite/citest.py                           | 21
> > > +++++++++++++++++++
> > >  2 files changed, 23 insertions(+)
> > > 
> > > diff --git a/meta-test/recipes-core/images/isar-image-ci.bb
> > > b/meta-
> > > test/recipes-core/images/isar-image-ci.bb
> > > index e5d51e6e..9133da74 100644
> > > --- a/meta-test/recipes-core/images/isar-image-ci.bb
> > > +++ b/meta-test/recipes-core/images/isar-image-ci.bb
> > > @@ -16,6 +16,7 @@ IMAGE_INSTALL += "sshd-regen-keys"
> > >  
> > >  # qemuamd64-bookworm
> > >  WKS_FILE:qemuamd64:debian-bookworm ?= "multipart-efi.wks"
> > > +IMAGE_INSTALL:append:qemuamd64:debian-bookworm = " prebuilt-
> > > docker-
> > > img prebuilt-podman-img"
> > >  
> > >  # qemuamd64-bullseye
> > >  IMAGE_FSTYPES:append:qemuamd64:debian-bullseye ?= " cpio.gz
> > > tar.gz"
> > > @@ -51,3 +52,4 @@ IMAGER_INSTALL:append:qemuarm:debian-bookworm
> > > ?= "
> > > ${SYSTEMD_BOOTLOADER_INSTALL}
> > >  # qemuarm64-bookworm
> > >  IMAGE_FSTYPES:append:qemuarm64:debian-bookworm ?= " wic.xz"
> > >  IMAGER_INSTALL:append:qemuarm64:debian-bookworm ?= "
> > > ${GRUB_BOOTLOADER_INSTALL}"
> > > +IMAGE_INSTALL:append:qemuarm64:debian-bookworm = " prebuilt-
> > > docker-
> > > img prebuilt-podman-img"
> > > diff --git a/testsuite/citest.py b/testsuite/citest.py
> > > index 8dd907d0..539c9440 100755
> > > --- a/testsuite/citest.py
> > > +++ b/testsuite/citest.py
> > > @@ -522,3 +522,24 @@ class VmBootTestFull(CIBaseTest):
> > >          self.init()
> > >          self.vm_start('mipsel','bookworm', image='isar-image-
> > > ci',
> > >                        script='test_kernel_module.sh
> > > example_module')
> > > +
> > > +
> > > +    def test_amd64_bookworm_prebuilt_containers(self):
> > > +        self.init()
> > > +        self.vm_start('amd64', 'bookworm', image='isar-image-
> > > ci',
> > > +                      cmd='echo root | su -c \'' \
> > > +                          'PATH=\$PATH:/usr/sbin;' \
> > > +                          'for n in \$(seq 30); do docker images
> > > |
> > > grep -q alpine && break; sleep 10; done;' \
> > > +                          'docker run --rm
> > > quay.io/libpod/alpine:3.10.2 true && ' \
> > > +                          'for n in \$(seq 30); do podman images
> > > |
> > > grep -q alpine && break; sleep 10; done;' \
> > > +                          'podman run --rm
> > > quay.io/libpod/alpine:latest true\'')
> > > +
> > > +    def test_arm64_bookworm_prebuilt_containers(self):
> > > +        self.init()
> > > +        self.vm_start('arm64', 'bookworm', image='isar-image-
> > > ci',
> > > +                      cmd='echo root | su -c \'' \
> > > +                          'PATH=\$PATH:/usr/sbin;' \
> > > +                          'for n in \$(seq 30); do docker images
> > > |
> > > grep -q alpine && break; sleep 10; done;' \
> > > +                          'docker run --rm
> > > quay.io/libpod/alpine:3.10.2 true && ' \
> > > +                          'for n in \$(seq 30); do podman images
> > > |
> > > grep -q alpine && break; sleep 10; done;' \
> > > +                          'podman run --rm
> > > quay.io/libpod/alpine:latest true\'')
> > 
> > This appears to fail in CI:
> > 
> > [stdlog] 2024-07-30 07:29:37,721 avocado.app ERROR| qemu-system-
> > aarch64: -drive
> > file=/workspace/build/isar_ub_devel/49/build/tmp/deploy/images/qemu
> > arm6
> > 4/isar-image-ci-debian-bookworm-
> > qemuarm64.ext4,if=none,format=raw,id=hd0: Could not open
> > '/workspace/build/isar_ub_devel/49/build/tmp/deploy/images/qemuarm6
> > 4/is
> > ar-image-ci-debian-bookworm-qemuarm64.ext4': No such file or
> > directory
> > 
> > Should be either changed the bullseye here (not yet checked if it
> > works). Or, we need to build arm64-bullseye image in build tests
> > before.
> 
> bullseye is legacy.
> 

Hello.

I rebased the patchset on top of "[PATCH] testsuite: Build bookworm
arm64 image in no-cross test". So, now this test doesn't fail this way.

But there were other issues related to the backslashed '\$'. I couldn't
catch this until applied one more useful patch: "[PATCH] testsuite:
Save ssh command stdout/stderr in case of failure" that allowed to see
syntax error here.

Also, flake8 appeared to recognize this as "W605 invalid escape
sequence '\$'".

I've just sent a replacement of the patch that fixes the issue and
python style here.

You could prepare pathset v5, or I could merge it as is after testing.

> Unfortunately, the testsuite lacks a dependency model. Here,
> VmBootTestFull depends on CrossTest and NoCrossTest - apparently this
> was not the case so far.
> 
> I'm still wondering why we are using such an unhandy and non-standard
> test framework if not even such basic things are expressed - or are
> even
> expressible?
> 
> Jan
> 

-- 
Best regards,
Uladzimir.



-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/ec827283af69b6564ae5915ada6e9def88f7ec6c.camel%40ilbers.de.