[PATCH 0/3] Add run tests for bookworm images

[PATCH 0/3] Add run tests for bookworm images

[Uladzimir Bely]

There are some bookworm images that we build in CI, but don't have
run tests for them.

This patchset does some test rearrangement in the testsuite and
adds run tests for existing bookworm images.

Uladzimir Bely (3):
  Small rework of groups of tests
  Separate fast and full groups of run tests.
  Add run tests for bookworm images

 testsuite/citest.py | 44 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 40 insertions(+), 4 deletions(-)

-- 
2.20.1

[PATCH 1/3] Small rework of groups of tests

[Uladzimir Bely]

This is mostly cosmetic change that just aligns some tests with
test groups they belong:
 - move mc:qemuarm64-bookworm:isar-image-base under proper group;
 - change confusing test group name under ContainerImageTest.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 testsuite/citest.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 448e866e..9335b04b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -72,7 +72,6 @@ class CrossTest(CIBaseTest):
             'mc:qemuarm-buster:isar-image-base',
             'mc:qemuarm-bullseye:isar-image-base',
             'mc:qemuarm64-bullseye:isar-image-base',
-            'mc:qemuarm64-bookworm:isar-image-base',
             'mc:de0-nano-soc-bullseye:isar-image-base',
             'mc:stm32mp15x-buster:isar-image-base'
                   ]
@@ -104,7 +103,8 @@ class CrossTest(CIBaseTest):
 
     def test_cross_bookworm(self):
         targets = [
-            'mc:qemuarm-bookworm:isar-image-base'
+            'mc:qemuarm-bookworm:isar-image-base',
+            'mc:qemuarm64-bookworm:isar-image-base'
                   ]
 
         self.init()
@@ -255,7 +255,7 @@ class ContainerImageTest(CIBaseTest):
     :avocado: tags=containerbuild,fast,full,container
     """
     @skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
-    def test_nocross(self):
+    def test_container_image(self):
         targets = [
             'mc:container-amd64-stretch:isar-image-base',
             'mc:container-amd64-buster:isar-image-base',
-- 
2.20.1

[PATCH 2/3] Separate fast and full groups of run tests.

[Uladzimir Bely]

Before, "full" CI build executed both "full" and "fast" subsets
of run tests. So, it was impossible to add run test for a target
that existed only in "fast" group.

Now, run tests can be individually added for both "fast" and "full"
subsets and they won't depend on each other.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 testsuite/citest.py | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 9335b04b..b161c967 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -299,7 +299,7 @@ class VmBootTestFast(CIBaseTest):
     """
     Test QEMU image start (fast)
 
-    :avocado: tags=startvm,fast,full
+    :avocado: tags=startvm,fast
     """
     def test_arm_bullseye(self):
         self.init()
@@ -324,6 +324,22 @@ class VmBootTestFull(CIBaseTest):
 
     :avocado: tags=startvm,full
     """
+    def test_arm_bullseye(self):
+        self.init()
+        self.vm_start('arm','bullseye')
+
+    def test_arm_buster(self):
+        self.init()
+        self.vm_start('arm','buster')
+
+    def test_arm64_bullseye(self):
+        self.init()
+        self.vm_start('arm64','bullseye')
+
+    def test_amd64_bullseye(self):
+        self.init()
+        self.vm_start('amd64','bullseye')
+
     def test_i386_stretch(self):
         self.init()
         self.vm_start('i386','stretch')
-- 
2.20.1

[PATCH 3/3] Add run tests for bookworm images

[Uladzimir Bely]

We build several bookworm-based images, but don't test if they
are run by qemu.

Here we simply add run tests for them and it costs almost nothing
(few seconds needed to run an image until login prompt reached).

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 testsuite/citest.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index b161c967..2ee5ba98 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -317,6 +317,10 @@ class VmBootTestFast(CIBaseTest):
         self.init()
         self.vm_start('amd64','bullseye')
 
+    def test_arm_bookworm(self):
+        self.init()
+        self.vm_start('arm','bookworm')
+
 class VmBootTestFull(CIBaseTest):
 
     """
@@ -358,3 +362,19 @@ class VmBootTestFull(CIBaseTest):
     def test_amd64_focal(self):
         self.init()
         self.vm_start('amd64','focal')
+
+    def test_amd64_bookworm(self):
+        self.init()
+        self.vm_start('amd','bookworm')
+
+    def test_arm_bookworm(self):
+        self.init()
+        self.vm_start('arm','bookworm')
+
+    def test_i386_bookworm(self):
+        self.init()
+        self.vm_start('i386','bookworm')
+
+    def test_mipsel_bookworm(self):
+        self.init()
+        self.vm_start('mipsel','bookworm')
-- 
2.20.1

Re: [PATCH 0/3] Add run tests for bookworm images

[Henning Schild]

Looks good, but i guess it will have to wait for a v2 of

"Set default locale in /etc/locale.conf"

Henning

Am Fri, 21 Oct 2022 12:59:28 +0200
schrieb Uladzimir Bely <ubely@ilbers.de>:

> There are some bookworm images that we build in CI, but don't have
> run tests for them.
> 
> This patchset does some test rearrangement in the testsuite and
> adds run tests for existing bookworm images.
> 
> Uladzimir Bely (3):
>   Small rework of groups of tests
>   Separate fast and full groups of run tests.
>   Add run tests for bookworm images
> 
>  testsuite/citest.py | 44 ++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 40 insertions(+), 4 deletions(-)
>