* [PATCH 00/23] Additional CI improvements
@ 2024-03-25 15:55 Anton Mikanovich
2024-03-25 15:55 ` [PATCH 01/23] CI: Introduce Standard CI for cross building Anton Mikanovich
` (22 more replies)
0 siblings, 23 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
This patchset accumulates testsuite fixes and improvements in the
following main ways:
- Migration of full testsuite to cross building
- Target lists update
- Speedup overall CI execution
- Improve testsuite naming
- Make output log lines shorter
- Improve startvm errors handling
- Increase startvm test coverage
- Add an ability to check all the possible targets
- Improve Avocado tags usage
- Code cleanup
The main change here is a new Standard testsuite to be placed between
fast and old full testsuites.
Anton Mikanovich (23):
CI: Introduce Standard CI for cross building
CI: Rename cross to fast
CI: Rename nocross to full
CI: Remove test keyword from class names
CI: Rename base test cases
CI: Remove cleanup before nocross build
CI: Update default target to bookworm
CI: Remove duplicated targets
CI: Run startvm for all qemu targets
CI: Fix errors reporting during startvm
CI: Remove logging hack for job.log
CI: Do not lost errors on failed processes
CI: Remove container package list hack from testsuite
CI: Minimize debsrc_cache checking
CI: Minimize kselftest checking
CI: Do not include testsuites into each other
CI: Mark some single testcases with tags
CI: Execute startvm testcases after building
CI: Fix missing endline in generated config
CI: Remove absolute path to citest.py
CI: Create avocado.conf inside build_dir
CI: Set also cache dir for Avocado
CI: Add make world test case
.gitignore | 2 +-
meta-isar/conf/local.conf.sample | 4 +
meta-test/conf/local.conf.sample | 3 +
scripts/ci_build.sh | 24 +-
testsuite/cibuilder.py | 86 +++--
testsuite/citest.py | 608 ++++++++++++++++++++++++-------
testsuite/data/targets.yml | 118 ++++++
testsuite/utils/targets_gen.py | 27 ++
8 files changed, 692 insertions(+), 180 deletions(-)
create mode 100644 testsuite/data/targets.yml
create mode 100755 testsuite/utils/targets_gen.py
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 01/23] CI: Introduce Standard CI for cross building
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 02/23] CI: Rename cross to fast Anton Mikanovich
` (21 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Add new testsuite called Standard as successor of Full CI.
It will build the same targets as Full but using cross compile.
There are also few targets left noncross for minimal checking.
>From now we should switch using:
$ ci_build.sh -T full
to:
$ ci_build.sh -T standard
Old full testsuite is left for compatibility and keeping coverage only.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 109 ++++++++++++++++++++++++++++++++++++++++----
1 file changed, 100 insertions(+), 9 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 7e24c498..92499781 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -22,7 +22,7 @@ class DevTest(CIBaseTest):
"""
Developer's test
- :avocado: tags=dev,fast,full
+ :avocado: tags=dev,fast,standard,full
"""
def test_dev(self):
targets = [
@@ -76,7 +76,7 @@ class ReproTest(CIBaseTest):
"""
Test cached base repository
- :avocado: tags=repro,full
+ :avocado: tags=repro,standard,full
"""
def test_repro_signed(self):
targets = [
@@ -108,7 +108,7 @@ class CcacheTest(CIBaseTest):
"""
Test rebuild speed improve with ccache
- :avocado: tags=ccache,full
+ :avocado: tags=ccache,standard,full
"""
def test_ccache_rebuild(self):
targets = ['mc:qemuamd64-bullseye:hello-isar']
@@ -120,7 +120,7 @@ class CrossTest(CIBaseTest):
"""
Start cross build for the defined set of configurations
- :avocado: tags=cross,fast,full
+ :avocado: tags=cross,fast,standard,full
"""
def test_cross(self):
targets = [
@@ -153,7 +153,7 @@ class WicTest(CIBaseTest):
"""
Test creation of wic images
- :avocado: tags=wic,full
+ :avocado: tags=wic,standard,full
"""
def test_wic_nodeploy_partitions(self):
targets = ['mc:qemuarm64-bookworm:isar-image-ci']
@@ -171,6 +171,97 @@ class WicTest(CIBaseTest):
self.perform_wic_partition_test(targets,
wic_deploy_parts=True, debsrc_cache=True, compat_arch=False)
+class StandardTest(CIBaseTest):
+
+ """
+ Start standard build for the defined set of configurations
+
+ :avocado: tags=standardbase,standard
+ """
+ def test_standard_cross(self):
+ targets = [
+ 'mc:qemuarm-buster:isar-image-ci',
+ 'mc:qemuarm-bullseye:isar-image-base',
+ 'mc:qemuarm64-bullseye:isar-image-ci',
+ 'mc:qemui386-buster:isar-image-base',
+ 'mc:qemui386-bullseye:isar-image-base',
+ 'mc:qemuamd64-buster:isar-image-ci',
+ 'mc:qemuamd64-bullseye:isar-initramfs',
+ 'mc:qemumipsel-bullseye:isar-image-base',
+ 'mc:imx6-sabrelite-bullseye:isar-image-base',
+ 'mc:phyboard-mira-bullseye:isar-image-base',
+ 'mc:hikey-bullseye:isar-image-base',
+ 'mc:virtualbox-bullseye:isar-image-base',
+ 'mc:virtualbox-bookworm:isar-image-base',
+ 'mc:bananapi-bullseye:isar-image-base',
+ 'mc:bananapi-bookworm:isar-image-base',
+ 'mc:nanopi-neo-bullseye:isar-image-base',
+ 'mc:nanopi-neo-bookworm:isar-image-base',
+ 'mc:stm32mp15x-bullseye:isar-image-base',
+ 'mc:qemuamd64-focal:isar-image-ci',
+ 'mc:qemuamd64-bookworm:isar-image-ci',
+ 'mc:qemui386-bookworm:isar-image-base',
+ 'mc:qemumipsel-bookworm:isar-image-ci',
+ 'mc:hikey-bookworm:isar-image-base',
+ 'mc:de0-nano-soc-bookworm:isar-image-base',
+ ]
+
+ self.init()
+ self.perform_build_test(targets, debsrc_cache=True)
+
+ def test_standard_nocross(self):
+ targets = [
+ 'mc:qemumipsel-buster:isar-image-base',
+ 'mc:qemuarm-bookworm:isar-image-ci',
+ ]
+
+ self.init()
+ self.perform_build_test(targets, cross=False)
+
+ def test_standard_rpi(self):
+ targets = [
+ 'mc:rpi-arm-bullseye:isar-image-base',
+ 'mc:rpi-arm-v7-bullseye:isar-image-base',
+ 'mc:rpi-arm-v7l-bullseye:isar-image-base',
+ 'mc:rpi-arm64-v8-bullseye:isar-image-base',
+ 'mc:rpi-arm-bookworm:isar-image-base',
+ 'mc:rpi-arm-v7-bookworm:isar-image-base',
+ 'mc:rpi-arm-v7l-bookworm:isar-image-base',
+ 'mc:rpi-arm64-v8-bookworm:isar-image-base',
+ ]
+
+ self.init()
+ try:
+ self.perform_build_test(targets, debsrc_cache=True)
+ except:
+ self.cancel('KFAIL')
+
+ def test_standard_trixie(self):
+ targets = [
+ 'mc:qemuamd64-trixie:isar-image-base',
+ 'mc:qemuarm64-trixie:isar-image-base',
+ 'mc:qemuarm-trixie:isar-image-base',
+ ]
+
+ self.init()
+ try:
+ self.perform_build_test(targets)
+ except:
+ self.cancel('KFAIL')
+
+ def test_standard_sid(self):
+ targets = [
+ 'mc:qemuriscv64-sid:isar-image-base',
+ 'mc:sifive-fu540-sid:isar-image-base',
+ 'mc:starfive-visionfive2-sid:isar-image-base',
+ ]
+
+ self.init()
+ try:
+ self.perform_build_test(targets)
+ except:
+ self.cancel('KFAIL')
+
class NoCrossTest(CIBaseTest):
"""
@@ -263,7 +354,7 @@ class ContainerImageTest(CIBaseTest):
"""
Test containerized images creation
- :avocado: tags=containerbuild,fast,full,container
+ :avocado: tags=containerbuild,fast,standard,full,container
"""
@skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
def test_container_image(self):
@@ -281,7 +372,7 @@ class ContainerSdkTest(CIBaseTest):
"""
Test SDK container image creation
- :avocado: tags=containersdk,fast,full,container
+ :avocado: tags=containersdk,fast,standard,full,container
"""
@skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
def test_container_sdk(self):
@@ -295,7 +386,7 @@ class SstateTest(CIBaseTest):
"""
Test builds with artifacts taken from sstate cache
- :avocado: tags=sstate,full
+ :avocado: tags=sstate,standard,full
"""
def test_sstate_populate(self):
@@ -406,7 +497,7 @@ class VmBootTestFull(CIBaseTest):
"""
Test QEMU image start (full)
- :avocado: tags=startvm,full
+ :avocado: tags=startvm,standard,full
"""
def test_arm_bullseye(self):
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 02/23] CI: Rename cross to fast
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
2024-03-25 15:55 ` [PATCH 01/23] CI: Introduce Standard CI for cross building Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 03/23] CI: Rename nocross to full Anton Mikanovich
` (20 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Because now both fast and standard are using cross building.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 92499781..7c80b910 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -115,14 +115,14 @@ class CcacheTest(CIBaseTest):
self.init()
self.perform_ccache_test(targets)
-class CrossTest(CIBaseTest):
+class FastTest(CIBaseTest):
"""
- Start cross build for the defined set of configurations
+ Start fast build for the defined set of configurations
- :avocado: tags=cross,fast,standard,full
+ :avocado: tags=fastbase,fast,standard,full
"""
- def test_cross(self):
+ def test_fast(self):
targets = [
'mc:qemuarm-buster:isar-image-ci',
'mc:qemuarm-bullseye:isar-image-ci',
@@ -137,7 +137,7 @@ class CrossTest(CIBaseTest):
self.init()
self.perform_build_test(targets, debsrc_cache=True)
- def test_cross_rpi(self):
+ def test_fast_rpi(self):
targets = [
'mc:rpi-arm-v7-bullseye:isar-image-base',
]
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 03/23] CI: Rename nocross to full
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
2024-03-25 15:55 ` [PATCH 01/23] CI: Introduce Standard CI for cross building Anton Mikanovich
2024-03-25 15:55 ` [PATCH 02/23] CI: Rename cross to fast Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 04/23] CI: Remove test keyword from class names Anton Mikanovich
` (19 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
As we have more then just two testsuites there is no direct connection
between testsuite and cross/nocross modes. So get rid of old naming.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 7c80b910..575aaded 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -262,14 +262,14 @@ class StandardTest(CIBaseTest):
except:
self.cancel('KFAIL')
-class NoCrossTest(CIBaseTest):
+class FullTest(CIBaseTest):
"""
- Start non-cross build for the defined set of configurations
+ Start full build for the defined set of configurations
- :avocado: tags=nocross,full
+ :avocado: tags=fullbase,full
"""
- def test_nocross(self):
+ def test_full_nocross(self):
targets = [
'mc:qemuarm-buster:isar-image-ci',
'mc:qemuarm-bullseye:isar-image-base',
@@ -302,10 +302,10 @@ class NoCrossTest(CIBaseTest):
self.init()
# Cleanup after cross build
- self.move_in_build_dir('tmp', 'tmp_before_nocross')
+ self.move_in_build_dir('tmp', 'tmp_before_full_nocross')
self.perform_build_test(targets, cross=False, debsrc_cache=True)
- def test_nocross_rpi(self):
+ def test_full_rpi(self):
targets = [
'mc:rpi-arm-bullseye:isar-image-base',
'mc:rpi-arm-v7-bullseye:isar-image-base',
@@ -323,7 +323,7 @@ class NoCrossTest(CIBaseTest):
except:
self.cancel('KFAIL')
- def test_nocross_trixie(self):
+ def test_full_trixie(self):
targets = [
'mc:qemuamd64-trixie:isar-image-base',
'mc:qemuarm64-trixie:isar-image-base',
@@ -336,7 +336,7 @@ class NoCrossTest(CIBaseTest):
except:
self.cancel('KFAIL')
- def test_nocross_sid(self):
+ def test_full_sid(self):
targets = [
'mc:qemuriscv64-sid:isar-image-base',
'mc:sifive-fu540-sid:isar-image-base',
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 04/23] CI: Remove test keyword from class names
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (2 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 03/23] CI: Rename nocross to full Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 05/23] CI: Rename base test cases Anton Mikanovich
` (18 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
There are no non-test classes in citest.py.
This will make avocado log output little shorter.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 28 ++++++++++++++--------------
1 file changed, 14 insertions(+), 14 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 575aaded..ce6e107d 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -17,7 +17,7 @@ try:
except path.CmdNotFoundError:
SKOPEO_AVAILABLE = False
-class DevTest(CIBaseTest):
+class Dev(CIBaseTest):
"""
Developer's test
@@ -71,7 +71,7 @@ class DevTest(CIBaseTest):
self.init()
self.vm_start('arm', 'bullseye', skip_modulecheck=True)
-class ReproTest(CIBaseTest):
+class Repro(CIBaseTest):
"""
Test cached base repository
@@ -103,7 +103,7 @@ class ReproTest(CIBaseTest):
finally:
self.move_in_build_dir('tmp', 'tmp_repro_unsigned')
-class CcacheTest(CIBaseTest):
+class Ccache(CIBaseTest):
"""
Test rebuild speed improve with ccache
@@ -115,7 +115,7 @@ class CcacheTest(CIBaseTest):
self.init()
self.perform_ccache_test(targets)
-class FastTest(CIBaseTest):
+class Fast(CIBaseTest):
"""
Start fast build for the defined set of configurations
@@ -148,7 +148,7 @@ class FastTest(CIBaseTest):
except:
self.cancel('KFAIL')
-class WicTest(CIBaseTest):
+class Wic(CIBaseTest):
"""
Test creation of wic images
@@ -171,7 +171,7 @@ class WicTest(CIBaseTest):
self.perform_wic_partition_test(targets,
wic_deploy_parts=True, debsrc_cache=True, compat_arch=False)
-class StandardTest(CIBaseTest):
+class Standard(CIBaseTest):
"""
Start standard build for the defined set of configurations
@@ -262,7 +262,7 @@ class StandardTest(CIBaseTest):
except:
self.cancel('KFAIL')
-class FullTest(CIBaseTest):
+class Full(CIBaseTest):
"""
Start full build for the defined set of configurations
@@ -349,7 +349,7 @@ class FullTest(CIBaseTest):
except:
self.cancel('KFAIL')
-class ContainerImageTest(CIBaseTest):
+class ContainerImage(CIBaseTest):
"""
Test containerized images creation
@@ -367,7 +367,7 @@ class ContainerImageTest(CIBaseTest):
self.init()
self.perform_build_test(targets, container=True)
-class ContainerSdkTest(CIBaseTest):
+class ContainerSdk(CIBaseTest):
"""
Test SDK container image creation
@@ -381,7 +381,7 @@ class ContainerSdkTest(CIBaseTest):
self.init()
self.perform_build_test(targets, bitbake_cmd='do_populate_sdk', container=True)
-class SstateTest(CIBaseTest):
+class Sstate(CIBaseTest):
"""
Test builds with artifacts taken from sstate cache
@@ -401,7 +401,7 @@ class SstateTest(CIBaseTest):
self.init('build-sstate')
self.perform_sstate_test(image_target, package_target)
-class SingleTest(CIBaseTest):
+class Single(CIBaseTest):
"""
Single test for selected target
@@ -423,7 +423,7 @@ class SingleTest(CIBaseTest):
self.vm_start(machine.removeprefix('qemu'), distro)
-class SourceTest(CIBaseTest):
+class Source(CIBaseTest):
"""
Source contents test
@@ -439,7 +439,7 @@ class SourceTest(CIBaseTest):
self.init()
self.perform_source_test(targets)
-class VmBootTestFast(CIBaseTest):
+class VmBootFast(CIBaseTest):
"""
Test QEMU image start (fast)
@@ -492,7 +492,7 @@ class VmBootTestFast(CIBaseTest):
script='test_systemd_unit.sh getty.target 10')
-class VmBootTestFull(CIBaseTest):
+class VmBootFull(CIBaseTest):
"""
Test QEMU image start (full)
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 05/23] CI: Rename base test cases
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (3 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 04/23] CI: Remove test keyword from class names Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 06/23] CI: Remove cleanup before nocross build Anton Mikanovich
` (17 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Single test case can be called by the command like:
$ avocado run ../testsuite/citest.py:DevTest.test_dev
But this will actually call all test_dev* test cases.
So remove names which can be a substring of other ones.
After that running test_dev_min* will not cover all test_dev* cases.
This will allow user to call any single test case.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index ce6e107d..780a2882 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -24,7 +24,7 @@ class Dev(CIBaseTest):
:avocado: tags=dev,fast,standard,full
"""
- def test_dev(self):
+ def test_dev_min(self):
targets = [
'mc:qemuamd64-bullseye:isar-image-ci',
'mc:qemuarm-bullseye:isar-image-base',
@@ -122,7 +122,7 @@ class Fast(CIBaseTest):
:avocado: tags=fastbase,fast,standard,full
"""
- def test_fast(self):
+ def test_fast_min(self):
targets = [
'mc:qemuarm-buster:isar-image-ci',
'mc:qemuarm-bullseye:isar-image-ci',
@@ -394,7 +394,7 @@ class Sstate(CIBaseTest):
self.perform_sstate_populate(image_target)
- def test_sstate(self):
+ def test_sstate_reuse(self):
image_target = 'mc:qemuamd64-bullseye:isar-image-base'
package_target = 'mc:qemuamd64-bullseye:hello'
@@ -447,7 +447,7 @@ class VmBootFast(CIBaseTest):
:avocado: tags=startvm,fast
"""
- def test_arm_bullseye(self):
+ def test_arm_bullseye_base(self):
self.init()
self.vm_start('arm','bullseye', image='isar-image-ci', keep=True)
@@ -462,7 +462,7 @@ class VmBootFast(CIBaseTest):
script='test_systemd_unit.sh getty.target 10')
- def test_arm_buster(self):
+ def test_arm_buster_base(self):
self.init()
self.vm_start('arm','buster', image='isar-image-ci', keep=True)
@@ -477,7 +477,7 @@ class VmBootFast(CIBaseTest):
script='test_kernel_module.sh example_module')
- def test_arm_bookworm(self):
+ def test_arm_bookworm_base(self):
self.init()
self.vm_start('arm','bookworm', image='isar-image-ci', keep=True)
@@ -505,7 +505,7 @@ class VmBootFull(CIBaseTest):
self.vm_start('arm','bullseye')
- def test_arm_buster(self):
+ def test_arm_buster_base(self):
self.init()
self.vm_start('arm','buster', image='isar-image-ci', keep=True)
@@ -520,7 +520,7 @@ class VmBootFull(CIBaseTest):
script='test_systemd_unit.sh getty.target 10')
- def test_arm64_bullseye(self):
+ def test_arm64_bullseye_base(self):
self.init()
self.vm_start('arm64','bullseye', image='isar-image-ci', keep=True)
@@ -548,7 +548,7 @@ class VmBootFull(CIBaseTest):
self.vm_start('amd64', 'buster', True, image='isar-image-ci')
- def test_amd64_focal(self):
+ def test_amd64_focal_base(self):
self.init()
self.vm_start('amd64','focal', image='isar-image-ci', keep=True)
@@ -578,7 +578,7 @@ class VmBootFull(CIBaseTest):
self.vm_start('i386','bookworm')
- def test_mipsel_bookworm(self):
+ def test_mipsel_bookworm_base(self):
self.init()
self.vm_start('mipsel','bookworm', image='isar-image-ci', keep=True)
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 06/23] CI: Remove cleanup before nocross build
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (4 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 05/23] CI: Rename base test cases Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 07/23] CI: Update default target to bookworm Anton Mikanovich
` (16 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
If nocross building will not work after switching from cross mode it
should be considered as a bug.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 2 --
1 file changed, 2 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 780a2882..d95b60aa 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -301,8 +301,6 @@ class Full(CIBaseTest):
]
self.init()
- # Cleanup after cross build
- self.move_in_build_dir('tmp', 'tmp_before_full_nocross')
self.perform_build_test(targets, cross=False, debsrc_cache=True)
def test_full_rpi(self):
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 07/23] CI: Update default target to bookworm
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (5 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 06/23] CI: Remove cleanup before nocross build Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 08/23] CI: Remove duplicated targets Anton Mikanovich
` (15 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
As bookworm is now stable it should be checked by default.
Target qemuamd64-bullseye is moved to fast CI to keep coverage.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 52 +++++++++++++++++++++++++--------------------
1 file changed, 29 insertions(+), 23 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index d95b60aa..040202c4 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -26,10 +26,10 @@ class Dev(CIBaseTest):
"""
def test_dev_min(self):
targets = [
- 'mc:qemuamd64-bullseye:isar-image-ci',
- 'mc:qemuarm-bullseye:isar-image-base',
- 'mc:qemuarm-bullseye:isar-image-base:do_populate_sdk',
- 'mc:qemuarm64-bullseye:isar-image-base',
+ 'mc:qemuamd64-bookworm:isar-image-ci',
+ 'mc:qemuarm-bookworm:isar-image-base',
+ 'mc:qemuarm-bookworm:isar-image-base:do_populate_sdk',
+ 'mc:qemuarm64-bookworm:isar-image-base',
]
self.init()
@@ -37,8 +37,8 @@ class Dev(CIBaseTest):
def test_dev_apps(self):
targets = [
- 'mc:qemuamd64-bullseye:isar-image-ci',
- 'mc:qemuarm64-bullseye:isar-image-base',
+ 'mc:qemuamd64-bookworm:isar-image-ci',
+ 'mc:qemuarm64-bookworm:isar-image-base',
]
self.init()
@@ -55,21 +55,21 @@ class Dev(CIBaseTest):
file.write('do_fetch:append() {\n\n}')
try:
- self.perform_build_test('mc:qemuamd64-bullseye:isar-image-ci')
+ self.perform_build_test('mc:qemuamd64-bookworm:isar-image-ci')
finally:
self.restorefile(dpkgbase_file)
- def test_dev_run_amd64_bullseye(self):
+ def test_dev_run_amd64_bookworm(self):
self.init()
- self.vm_start('amd64', 'bullseye', image='isar-image-ci')
+ self.vm_start('amd64', 'bookworm', image='isar-image-ci')
- def test_dev_run_arm64_bullseye(self):
+ def test_dev_run_arm64_bookworm(self):
self.init()
- self.vm_start('arm64', 'bullseye')
+ self.vm_start('arm64', 'bookworm')
- def test_dev_run_arm_bullseye(self):
+ def test_dev_run_arm_bookworm(self):
self.init()
- self.vm_start('arm', 'bullseye', skip_modulecheck=True)
+ self.vm_start('arm', 'bookworm', skip_modulecheck=True)
class Repro(CIBaseTest):
@@ -82,7 +82,7 @@ class Repro(CIBaseTest):
targets = [
'mc:rpi-arm-v7-bullseye:isar-image-base',
'mc:rpi-arm64-v8-bullseye:isar-image-base',
- 'mc:qemuarm64-bullseye:isar-image-base',
+ 'mc:qemuarm64-bookworm:isar-image-base',
]
self.init()
@@ -93,8 +93,8 @@ class Repro(CIBaseTest):
def test_repro_unsigned(self):
targets = [
- 'mc:qemuamd64-bullseye:isar-image-base',
- 'mc:qemuarm-bullseye:isar-image-base',
+ 'mc:qemuamd64-bookworm:isar-image-base',
+ 'mc:qemuarm-bookworm:isar-image-base',
]
self.init()
@@ -111,7 +111,7 @@ class Ccache(CIBaseTest):
:avocado: tags=ccache,standard,full
"""
def test_ccache_rebuild(self):
- targets = ['mc:qemuamd64-bullseye:hello-isar']
+ targets = ['mc:qemuamd64-bookworm:hello-isar']
self.init()
self.perform_ccache_test(targets)
@@ -124,6 +124,7 @@ class Fast(CIBaseTest):
"""
def test_fast_min(self):
targets = [
+ 'mc:qemuamd64-bullseye:isar-image-ci',
'mc:qemuarm-buster:isar-image-ci',
'mc:qemuarm-bullseye:isar-image-ci',
'mc:de0-nano-soc-bullseye:isar-image-base',
@@ -374,7 +375,7 @@ class ContainerSdk(CIBaseTest):
"""
@skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
def test_container_sdk(self):
- targets = ['mc:container-amd64-bullseye:isar-image-base']
+ targets = ['mc:container-amd64-bookworm:isar-image-base']
self.init()
self.perform_build_test(targets, bitbake_cmd='do_populate_sdk', container=True)
@@ -388,13 +389,13 @@ class Sstate(CIBaseTest):
"""
def test_sstate_populate(self):
- image_target = 'mc:qemuamd64-bullseye:isar-image-base'
+ image_target = 'mc:qemuamd64-bookworm:isar-image-base'
self.perform_sstate_populate(image_target)
def test_sstate_reuse(self):
- image_target = 'mc:qemuamd64-bullseye:isar-image-base'
- package_target = 'mc:qemuamd64-bullseye:hello'
+ image_target = 'mc:qemuamd64-bookworm:isar-image-base'
+ package_target = 'mc:qemuamd64-bookworm:hello'
self.init('build-sstate')
self.perform_sstate_test(image_target, package_target)
@@ -409,7 +410,7 @@ class Single(CIBaseTest):
def test_single_build(self):
self.init()
machine = self.params.get('machine', default='qemuamd64')
- distro = self.params.get('distro', default='bullseye')
+ distro = self.params.get('distro', default='bookworm')
image = self.params.get('image', default='isar-image-base')
self.perform_build_test('mc:%s-%s:%s' % (machine, distro, image))
@@ -417,7 +418,7 @@ class Single(CIBaseTest):
def test_single_run(self):
self.init()
machine = self.params.get('machine', default='qemuamd64')
- distro = self.params.get('distro', default='bullseye')
+ distro = self.params.get('distro', default='bookworm')
self.vm_start(machine.removeprefix('qemu'), distro)
@@ -445,6 +446,11 @@ class VmBootFast(CIBaseTest):
:avocado: tags=startvm,fast
"""
+ def test_amd64_bullseye(self):
+ self.init()
+ self.vm_start('amd64', 'bullseye', image='isar-image-ci')
+
+
def test_arm_bullseye_base(self):
self.init()
self.vm_start('arm','bullseye', image='isar-image-ci', keep=True)
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 08/23] CI: Remove duplicated targets
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (6 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 07/23] CI: Update default target to bookworm Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 09/23] CI: Run startvm for all qemu targets Anton Mikanovich
` (14 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
After changing targets and cross modes some target building got
duplicated. Optimise target lists for faster CI execution.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 37 ++++---------------------------------
1 file changed, 4 insertions(+), 33 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 040202c4..091526db 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -27,9 +27,8 @@ class Dev(CIBaseTest):
def test_dev_min(self):
targets = [
'mc:qemuamd64-bookworm:isar-image-ci',
- 'mc:qemuarm-bookworm:isar-image-base',
- 'mc:qemuarm-bookworm:isar-image-base:do_populate_sdk',
- 'mc:qemuarm64-bookworm:isar-image-base',
+ 'mc:qemuarm64-bookworm:isar-image-ci',
+ 'mc:qemuarm64-bookworm:isar-image-base:do_populate_sdk',
]
self.init()
@@ -38,7 +37,7 @@ class Dev(CIBaseTest):
def test_dev_apps(self):
targets = [
'mc:qemuamd64-bookworm:isar-image-ci',
- 'mc:qemuarm64-bookworm:isar-image-base',
+ 'mc:qemuarm64-bookworm:isar-image-ci',
]
self.init()
@@ -65,11 +64,7 @@ class Dev(CIBaseTest):
def test_dev_run_arm64_bookworm(self):
self.init()
- self.vm_start('arm64', 'bookworm')
-
- def test_dev_run_arm_bookworm(self):
- self.init()
- self.vm_start('arm', 'bookworm', skip_modulecheck=True)
+ self.vm_start('arm64', 'bookworm', image='isar-image-ci')
class Repro(CIBaseTest):
@@ -130,7 +125,6 @@ class Fast(CIBaseTest):
'mc:de0-nano-soc-bullseye:isar-image-base',
'mc:stm32mp15x-bullseye:isar-image-base',
'mc:qemuarm-bookworm:isar-image-ci',
- 'mc:qemuarm64-bookworm:isar-image-ci',
'mc:qemuarm64-focal:isar-image-base',
'mc:nanopi-neo-efi-bookworm:isar-image-base',
]
@@ -181,8 +175,6 @@ class Standard(CIBaseTest):
"""
def test_standard_cross(self):
targets = [
- 'mc:qemuarm-buster:isar-image-ci',
- 'mc:qemuarm-bullseye:isar-image-base',
'mc:qemuarm64-bullseye:isar-image-ci',
'mc:qemui386-buster:isar-image-base',
'mc:qemui386-bullseye:isar-image-base',
@@ -198,7 +190,6 @@ class Standard(CIBaseTest):
'mc:bananapi-bookworm:isar-image-base',
'mc:nanopi-neo-bullseye:isar-image-base',
'mc:nanopi-neo-bookworm:isar-image-base',
- 'mc:stm32mp15x-bullseye:isar-image-base',
'mc:qemuamd64-focal:isar-image-ci',
'mc:qemuamd64-bookworm:isar-image-ci',
'mc:qemui386-bookworm:isar-image-base',
@@ -504,26 +495,6 @@ class VmBootFull(CIBaseTest):
:avocado: tags=startvm,standard,full
"""
- def test_arm_bullseye(self):
- self.init()
- self.vm_start('arm','bullseye')
-
-
- def test_arm_buster_base(self):
- self.init()
- self.vm_start('arm','buster', image='isar-image-ci', keep=True)
-
- def test_arm_buster_example_module(self):
- self.init()
- self.vm_start('arm','buster', image='isar-image-ci',
- cmd='lsmod | grep example_module', keep=True)
-
- def test_arm_buster_getty_target(self):
- self.init()
- self.vm_start('arm','buster', image='isar-image-ci',
- script='test_systemd_unit.sh getty.target 10')
-
-
def test_arm64_bullseye_base(self):
self.init()
self.vm_start('arm64','bullseye', image='isar-image-ci', keep=True)
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 09/23] CI: Run startvm for all qemu targets
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (7 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 08/23] CI: Remove duplicated targets Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 10/23] CI: Fix errors reporting during startvm Anton Mikanovich
` (13 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Startvm tests are now fast enough to run them for every targets built.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 091526db..cef39319 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -487,6 +487,11 @@ class VmBootFast(CIBaseTest):
script='test_systemd_unit.sh getty.target 10')
+ def test_arm64_focal(self):
+ self.init()
+ self.vm_start('arm64', 'focal', image='isar-image-base')
+
+
class VmBootFull(CIBaseTest):
"""
@@ -515,6 +520,11 @@ class VmBootFull(CIBaseTest):
self.vm_start('i386','buster')
+ def test_i386_bullseye(self):
+ self.init()
+ self.vm_start('i386', 'bullseye')
+
+
def test_amd64_buster(self):
self.init()
# test efi boot
@@ -523,6 +533,16 @@ class VmBootFull(CIBaseTest):
self.vm_start('amd64', 'buster', True, image='isar-image-ci')
+ def test_mipsel_bullseye(self):
+ self.init()
+ self.vm_start('mipsel', 'bullseye')
+
+
+ def test_mipsel_buster(self):
+ self.init()
+ self.vm_start('mipsel', 'buster')
+
+
def test_amd64_focal_base(self):
self.init()
self.vm_start('amd64','focal', image='isar-image-ci', keep=True)
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 10/23] CI: Fix errors reporting during startvm
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (8 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 09/23] CI: Run startvm for all qemu targets Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 11/23] CI: Remove logging hack for job.log Anton Mikanovich
` (12 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Test case failure should be triggered by self.fail() API.
It helps to prepare much more correct log output.
So rebuild error handling of startvm related functions.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 35 +++++++++++++++++++++++------------
1 file changed, 23 insertions(+), 12 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index fa30c2f5..d282f8ee 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -475,7 +475,7 @@ BBPATH .= ":${LAYERDIR}"\
p1 = subprocess.Popen('exec ' + ' '.join(cmdline), shell=True,
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
universal_newlines=True)
- self.log.info("Started VM with pid %s" % (p1.pid))
+ self.log.info("Starting VM with pid %s" % (p1.pid))
return p1, cmdline, boot_log
@@ -506,8 +506,10 @@ BBPATH .= ":${LAYERDIR}"\
if fd == p1.stderr.fileno():
app_log.error(p1.stderr.readline().rstrip())
- self.log.error("Didn't get login prompt")
- return 1
+ rc = 1
+ if time.time() > timeout:
+ rc = 2
+ return rc
def vm_parse_output(self, boot_log, multiconfig, skip_modulecheck):
@@ -539,10 +541,8 @@ BBPATH .= ":${LAYERDIR}"\
if (module_output in data or skip_modulecheck):
if resize_output and not resize_output in data:
rc = 1
- self.log.error("No resize output while expected")
else:
rc = 2
- self.log.error("No example module output while expected")
return rc
@@ -553,13 +553,16 @@ BBPATH .= ":${LAYERDIR}"\
def vm_turn_off(self, vm):
- pid = self.vm_dict[vm][0]
- os.kill(pid, signal.SIGKILL)
+ try:
+ pid = self.vm_dict[vm][0]
+ os.kill(pid, signal.SIGKILL)
- del(self.vm_dict[vm])
- self.vm_dump_dict(vm)
+ del(self.vm_dict[vm])
+ self.vm_dump_dict(vm)
- self.log.info("Stopped VM with pid %s" % (pid))
+ self.log.info("Stopped VM with pid %s" % (pid))
+ except ProcessLookupError:
+ self.log.error("Can't stop VM %s" % vm)
def vm_start(self, arch='amd64', distro='buster',
@@ -611,7 +614,10 @@ BBPATH .= ":${LAYERDIR}"\
rc = self.vm_wait_boot(p1, timeout)
if rc != 0:
self.vm_turn_off(vm)
- self.fail('Failed to boot qemu machine')
+ if rc == 2:
+ self.fail("Didn't get login prompt")
+ else:
+ self.fail('Failed to boot qemu machine')
if cmd is not None or script is not None:
self.ssh_user='ci'
@@ -638,7 +644,12 @@ BBPATH .= ":${LAYERDIR}"\
if rc != 0:
if not keep:
self.vm_turn_off(vm)
- self.fail('Failed to parse output')
+ if rc == 1:
+ self.fail("No resize output while expected")
+ elif rc == 2:
+ self.fail("No example module output while expected")
+ else:
+ self.fail('Failed to parse output')
if not keep:
self.vm_turn_off(vm)
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 11/23] CI: Remove logging hack for job.log
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (9 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 10/23] CI: Fix errors reporting during startvm Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 12/23] CI: Do not lost errors on failed processes Anton Mikanovich
` (11 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
As job.log no more store all the output of executed test cases we don't
need to inject errors in it. And full.log which contains full output
already has errors in it.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 18 ++----------------
1 file changed, 2 insertions(+), 16 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index d282f8ee..ec2c0044 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -1,6 +1,5 @@
#!/usr/bin/env python3
-import logging
import os
import pickle
import re
@@ -29,23 +28,10 @@ DEF_VM_TO_SEC = 600
isar_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
backup_prefix = '.ci-backup'
-app_log = logging.getLogger("avocado.app")
-
class CanBeFinished(Exception):
pass
class CIBuilder(Test):
- def setUp(self):
- super(CIBuilder, self).setUp()
- job_log = os.path.join(os.path.dirname(self.logdir), '..', 'job.log')
- self._file_handler = logging.FileHandler(filename=job_log)
- self._file_handler.setLevel(logging.ERROR)
- fmt = ('%(asctime)s %(module)-16.16s L%(lineno)-.4d %('
- 'levelname)-5.5s| %(message)s')
- formatter = logging.Formatter(fmt=fmt)
- self._file_handler.setFormatter(formatter)
- app_log.addHandler(self._file_handler)
-
def init(self, build_dir='build', isar_dir=isar_root):
# initialize build_dir and setup environment
# needs to run once (per test case)
@@ -219,7 +205,7 @@ class CIBuilder(Test):
if fd == p1.stdout.fileno():
self.log.info(p1.stdout.readline().rstrip())
if fd == p1.stderr.fileno():
- app_log.error(p1.stderr.readline().rstrip())
+ self.log.error(p1.stderr.readline().rstrip())
p1.wait()
if p1.returncode:
self.fail('Bitbake failed')
@@ -504,7 +490,7 @@ BBPATH .= ":${LAYERDIR}"\
self.log.info('Got login prompt')
return 0
if fd == p1.stderr.fileno():
- app_log.error(p1.stderr.readline().rstrip())
+ self.log.error(p1.stderr.readline().rstrip())
rc = 1
if time.time() > timeout:
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 12/23] CI: Do not lost errors on failed processes
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (10 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 11/23] CI: Remove logging hack for job.log Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 13/23] CI: Remove container package list hack from testsuite Anton Mikanovich
` (10 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
If subprocess failed before parsing its output we should try to get
error output at least once. Otherwise it will be lost.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index ec2c0044..a4658ea7 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -197,7 +197,7 @@ class CIBuilder(Test):
poller = select.poll()
poller.register(p1.stdout, select.POLLIN)
poller.register(p1.stderr, select.POLLIN)
- while p1.poll() is None:
+ while True:
events = poller.poll(1000)
for fd, event in events:
if event != select.POLLIN:
@@ -206,6 +206,8 @@ class CIBuilder(Test):
self.log.info(p1.stdout.readline().rstrip())
if fd == p1.stderr.fileno():
self.log.error(p1.stderr.readline().rstrip())
+ if p1.poll() is not None:
+ break
p1.wait()
if p1.returncode:
self.fail('Bitbake failed')
@@ -477,7 +479,7 @@ BBPATH .= ":${LAYERDIR}"\
databuf = bytearray(b'')
databuf_size = 1024 * 2 + len(login_prompt)
- while time.time() < timeout and p1.poll() is None:
+ while time.time() < timeout:
events = poller.poll(1000 * (timeout - time.time()))
for fd, event in events:
if event != select.POLLIN:
@@ -491,6 +493,8 @@ BBPATH .= ":${LAYERDIR}"\
return 0
if fd == p1.stderr.fileno():
self.log.error(p1.stderr.readline().rstrip())
+ if p1.poll() is not None:
+ break
rc = 1
if time.time() > timeout:
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 13/23] CI: Remove container package list hack from testsuite
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (11 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 12/23] CI: Do not lost errors on failed processes Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 14/23] CI: Minimize debsrc_cache checking Anton Mikanovich
` (9 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
As IMAGE_INSTALL list is set in local.conf removing items should be
also done in local.conf.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta-isar/conf/local.conf.sample | 4 ++++
meta-test/conf/local.conf.sample | 3 +++
testsuite/cibuilder.py | 1 -
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 17455015..5baefa73 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -156,6 +156,10 @@ IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsc
IMAGE_INSTALL:remove:qemuamd64-sb = "example-module-${KERNEL_NAME}"
IMAGE_INSTALL:append:qemuamd64-sb = " example-module-signed-${KERNEL_NAME}"
+#
+# Remove not-needed packages for containerized images.
+IMAGE_INSTALL:remove:container-amd64 = "example-module-${KERNEL_NAME} enable-fsck"
+
#
# Enable cross-compilation support
ISAR_CROSS_COMPILE ?= "0"
diff --git a/meta-test/conf/local.conf.sample b/meta-test/conf/local.conf.sample
index f692f533..029e0767 100644
--- a/meta-test/conf/local.conf.sample
+++ b/meta-test/conf/local.conf.sample
@@ -34,6 +34,9 @@ IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsc
IMAGE_INSTALL:remove:qemuamd64-sb = "example-module-${KERNEL_NAME}"
IMAGE_INSTALL:append:qemuamd64-sb = " example-module-signed-${KERNEL_NAME}"
+# Remove not-needed packages for containerized images.
+IMAGE_INSTALL:remove:container-amd64 = "example-module-${KERNEL_NAME} enable-fsck"
+
# Users and groups
USERS += "root"
USER_root[password] ??= "$6$rounds=10000$RXeWrnFmkY$DtuS/OmsAS2cCEDo0BF5qQsizIrq6jPgXnwv3PHqREJeKd1sXdHX/ayQtuQWVDHe0KIO0/sVH8dvQm1KthF0d/"
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index a4658ea7..0f26c906 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -134,7 +134,6 @@ class CIBuilder(Test):
f.write('BB_NO_NETWORK = "1"\n')
if container:
f.write('SDK_FORMATS = "docker-archive"\n')
- f.write('IMAGE_INSTALL:remove = "example-module-${KERNEL_NAME} enable-fsck"\n')
if gpg_pub_key:
f.write('BASE_REPO_KEY="file://' + gpg_pub_key + '"\n')
if wic_deploy_parts:
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 14/23] CI: Minimize debsrc_cache checking
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (12 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 13/23] CI: Remove container package list hack from testsuite Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 15/23] CI: Minimize kselftest checking Anton Mikanovich
` (8 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Enable debsrc_cache for such a big set of targets is not needed for
checking this functionality. Speedup testsuites by enabling
debsrc_cache partially.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 26 ++++++++++++++++++++------
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index cef39319..b3fd8ac2 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -124,8 +124,15 @@ class Fast(CIBaseTest):
'mc:qemuarm-bullseye:isar-image-ci',
'mc:de0-nano-soc-bullseye:isar-image-base',
'mc:stm32mp15x-bullseye:isar-image-base',
- 'mc:qemuarm-bookworm:isar-image-ci',
'mc:qemuarm64-focal:isar-image-base',
+ ]
+
+ self.init()
+ self.perform_build_test(targets)
+
+ def test_fast_debsrc(self):
+ targets = [
+ 'mc:qemuarm-bookworm:isar-image-ci',
'mc:nanopi-neo-efi-bookworm:isar-image-base',
]
@@ -139,7 +146,7 @@ class Fast(CIBaseTest):
self.init()
try:
- self.perform_build_test(targets, debsrc_cache=True)
+ self.perform_build_test(targets)
except:
self.cancel('KFAIL')
@@ -156,7 +163,7 @@ class Wic(CIBaseTest):
self.init()
self.move_in_build_dir('tmp', 'tmp_before_wic')
self.perform_wic_partition_test(targets,
- wic_deploy_parts=False, debsrc_cache=True, compat_arch=False)
+ wic_deploy_parts=False, compat_arch=False)
def test_wic_deploy_partitions(self):
targets = ['mc:qemuarm64-bookworm:isar-image-ci']
@@ -175,7 +182,6 @@ class Standard(CIBaseTest):
"""
def test_standard_cross(self):
targets = [
- 'mc:qemuarm64-bullseye:isar-image-ci',
'mc:qemui386-buster:isar-image-base',
'mc:qemui386-bullseye:isar-image-base',
'mc:qemuamd64-buster:isar-image-ci',
@@ -183,7 +189,6 @@ class Standard(CIBaseTest):
'mc:qemumipsel-bullseye:isar-image-base',
'mc:imx6-sabrelite-bullseye:isar-image-base',
'mc:phyboard-mira-bullseye:isar-image-base',
- 'mc:hikey-bullseye:isar-image-base',
'mc:virtualbox-bullseye:isar-image-base',
'mc:virtualbox-bookworm:isar-image-base',
'mc:bananapi-bullseye:isar-image-base',
@@ -198,6 +203,15 @@ class Standard(CIBaseTest):
'mc:de0-nano-soc-bookworm:isar-image-base',
]
+ self.init()
+ self.perform_build_test(targets)
+
+ def test_standard_debsrc(self):
+ targets = [
+ 'mc:qemuarm64-bullseye:isar-image-ci',
+ 'mc:hikey-bullseye:isar-image-base',
+ ]
+
self.init()
self.perform_build_test(targets, debsrc_cache=True)
@@ -224,7 +238,7 @@ class Standard(CIBaseTest):
self.init()
try:
- self.perform_build_test(targets, debsrc_cache=True)
+ self.perform_build_test(targets)
except:
self.cancel('KFAIL')
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 15/23] CI: Minimize kselftest checking
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (13 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 14/23] CI: Minimize debsrc_cache checking Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 16/23] CI: Do not include testsuites into each other Anton Mikanovich
` (7 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
There is no need in building kselftest for all the targets.
Move it to separate test case.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 1 -
testsuite/citest.py | 12 ++++++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 0f26c906..256534dc 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -124,7 +124,6 @@ class CIBuilder(Test):
f.write('ISAR_ENABLE_COMPAT_ARCH:arm64 = "1"\n')
f.write('IMAGE_INSTALL:remove:arm64 = "hello-isar"\n')
f.write('IMAGE_INSTALL:append:arm64 = " hello-isar-compat"\n')
- f.write('IMAGE_INSTALL += "kselftest"\n')
if cross:
f.write('ISAR_CROSS_COMPILE = "1"\n')
if debsrc_cache:
diff --git a/testsuite/citest.py b/testsuite/citest.py
index b3fd8ac2..799e742d 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -139,6 +139,18 @@ class Fast(CIBaseTest):
self.init()
self.perform_build_test(targets, debsrc_cache=True)
+ def test_fast_kselftest(self):
+ targets = [
+ 'mc:qemuamd64-bullseye:isar-image-ci',
+ 'mc:qemuarm-buster:isar-image-ci',
+ 'mc:qemuarm-bookworm:isar-image-ci',
+ 'mc:nanopi-neo-efi-bookworm:isar-image-base',
+ 'mc:qemuarm64-focal:isar-image-base',
+ ]
+
+ self.init()
+ self.perform_build_test(targets, image_install="kselftest")
+
def test_fast_rpi(self):
targets = [
'mc:rpi-arm-v7-bullseye:isar-image-base',
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 16/23] CI: Do not include testsuites into each other
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (14 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 15/23] CI: Minimize kselftest checking Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 17/23] CI: Mark some single testcases with tags Anton Mikanovich
` (6 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
>From now on fast testsuite will not also execute dev, standard/full
will not execute also fast and dev.
The same coverage should be done by running several testsuites.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 799e742d..5fb3ae41 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -22,7 +22,7 @@ class Dev(CIBaseTest):
"""
Developer's test
- :avocado: tags=dev,fast,standard,full
+ :avocado: tags=dev
"""
def test_dev_min(self):
targets = [
@@ -115,7 +115,7 @@ class Fast(CIBaseTest):
"""
Start fast build for the defined set of configurations
- :avocado: tags=fastbase,fast,standard,full
+ :avocado: tags=fastbase,fast
"""
def test_fast_min(self):
targets = [
@@ -370,7 +370,7 @@ class ContainerImage(CIBaseTest):
"""
Test containerized images creation
- :avocado: tags=containerbuild,fast,standard,full,container
+ :avocado: tags=containerbuild,fast,container
"""
@skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
def test_container_image(self):
@@ -388,7 +388,7 @@ class ContainerSdk(CIBaseTest):
"""
Test SDK container image creation
- :avocado: tags=containersdk,fast,standard,full,container
+ :avocado: tags=containersdk,fast,container
"""
@skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
def test_container_sdk(self):
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 17/23] CI: Mark some single testcases with tags
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (15 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 16/23] CI: Do not include testsuites into each other Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 18/23] CI: Execute startvm testcases after building Anton Mikanovich
` (5 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To improve testcase execution flexibility.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 5fb3ae41..efabd450 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -44,6 +44,9 @@ class Dev(CIBaseTest):
self.perform_build_test(targets)
def test_dev_rebuild(self):
+ """
+ :avocado: tags=rebuild
+ """
self.init()
layerdir_core = self.getVars('LAYERDIR_core')
@@ -58,13 +61,25 @@ class Dev(CIBaseTest):
finally:
self.restorefile(dpkgbase_file)
+ """
+ BEGIN: Run tests for test_dev_apps
+ """
def test_dev_run_amd64_bookworm(self):
+ """
+ :avocado: tags=startvm
+ """
self.init()
self.vm_start('amd64', 'bookworm', image='isar-image-ci')
def test_dev_run_arm64_bookworm(self):
+ """
+ :avocado: tags=startvm
+ """
self.init()
self.vm_start('arm64', 'bookworm', image='isar-image-ci')
+ """
+ END: Run tests for test_dev_apps
+ """
class Repro(CIBaseTest):
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 18/23] CI: Execute startvm testcases after building
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (16 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 17/23] CI: Mark some single testcases with tags Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 19/23] CI: Fix missing endline in generated config Anton Mikanovich
` (4 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Move startvm testcases up to be executed right after following targets
git built. Tag them as startvm to allow dsabling with '-startvm'.
Full testsuite was not changed.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 309 +++++++++++++++++++++++++++++++++++---------
1 file changed, 245 insertions(+), 64 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index efabd450..395b2965 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -145,6 +145,74 @@ class Fast(CIBaseTest):
self.init()
self.perform_build_test(targets)
+
+ """
+ BEGIN: Run tests for test_fast_min
+ """
+ def test_run_amd64_bullseye(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('amd64', 'bullseye', image='isar-image-ci')
+
+
+ def test_run_arm_buster_base(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'buster', image='isar-image-ci', keep=True)
+
+ def test_run_arm_buster_getty_target(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'buster', image='isar-image-ci',
+ cmd='systemctl is-active getty.target', keep=True)
+
+ def test_run_arm_buster_example_module(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'buster', image='isar-image-ci',
+ script='test_kernel_module.sh example_module')
+
+
+ def test_run_arm_bullseye_base(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bullseye', image='isar-image-ci', keep=True)
+
+ def test_run_arm_bullseye_example_module(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bullseye', image='isar-image-ci',
+ cmd='lsmod | grep example_module', keep=True)
+
+ def test_run_arm_bullseye_getty_target(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bullseye', image='isar-image-ci',
+ script='test_systemd_unit.sh getty.target 10')
+
+
+ def test_run_arm64_focal(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm64', 'focal', image='isar-image-base')
+
+
def test_fast_debsrc(self):
targets = [
'mc:qemuarm-bookworm:isar-image-ci',
@@ -154,6 +222,34 @@ class Fast(CIBaseTest):
self.init()
self.perform_build_test(targets, debsrc_cache=True)
+
+ def test_run_arm_bookworm_base(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bookworm', image='isar-image-ci', keep=True)
+
+ def test_run_arm_bookworm_example_module(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bookworm', image='isar-image-ci',
+ cmd='lsmod | grep example_module', keep=True)
+
+ def test_run_arm_bookworm_getty_target(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bookworm', image='isar-image-ci',
+ script='test_systemd_unit.sh getty.target 10')
+ """
+ END: Run tests for test_fast_min
+ """
+
+
def test_fast_kselftest(self):
targets = [
'mc:qemuamd64-bullseye:isar-image-ci',
@@ -233,6 +329,109 @@ class Standard(CIBaseTest):
self.init()
self.perform_build_test(targets)
+
+ """
+ BEGIN: Run tests for test_standard_cross
+ """
+ def test_run_i386_buster(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('i386', 'buster')
+
+
+ def test_run_i386_bullseye(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('i386', 'bullseye')
+
+
+ def test_run_amd64_buster(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ # test efi boot
+ self.vm_start('amd64', 'buster', image='isar-image-ci')
+ # test pcbios boot
+ self.vm_start('amd64', 'buster', True, image='isar-image-ci')
+
+
+ def test_run_mipsel_bullseye(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('mipsel', 'bullseye')
+
+
+ def test_run_amd64_focal_base(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('amd64', 'focal', image='isar-image-ci', keep=True)
+
+ def test_run_amd64_focal_example_module(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('amd64', 'focal', image='isar-image-ci',
+ cmd='lsmod | grep example_module', keep=True)
+
+ def test_run_amd64_focal_getty_target(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('amd64', 'focal', image='isar-image-ci',
+ script='test_systemd_unit.sh getty.target 10')
+
+
+ def test_run_amd64_bookworm(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('amd64', 'bookworm', image='isar-image-ci')
+
+
+ def test_run_i386_bookworm(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('i386', 'bookworm')
+
+
+ def test_run_mipsel_bookworm_base(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('mipsel', 'bookworm', image='isar-image-ci', keep=True)
+
+ def test_run_mipsel_bookworm_getty_target(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('mipsel', 'bookworm', image='isar-image-ci',
+ cmd='systemctl is-active getty.target', keep=True)
+
+ def test_run_mipsel_bookworm_example_module(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('mipsel', 'bookworm', image='isar-image-ci',
+ script='test_kernel_module.sh example_module')
+
+
def test_standard_debsrc(self):
targets = [
'mc:qemuarm64-bullseye:isar-image-ci',
@@ -242,6 +441,31 @@ class Standard(CIBaseTest):
self.init()
self.perform_build_test(targets, debsrc_cache=True)
+
+ def test_run_arm64_bullseye_base(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm64', 'bullseye', image='isar-image-ci', keep=True)
+
+ def test_run_arm64_bullseye_getty_target(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm64', 'bullseye', image='isar-image-ci',
+ cmd='systemctl is-active getty.target', keep=True)
+
+ def test_run_arm64_bullseye_example_module(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm64', 'bullseye', image='isar-image-ci',
+ script='test_kernel_module.sh example_module')
+
+
def test_standard_nocross(self):
targets = [
'mc:qemumipsel-buster:isar-image-base',
@@ -251,6 +475,26 @@ class Standard(CIBaseTest):
self.init()
self.perform_build_test(targets, cross=False)
+
+ def test_run_mipsel_buster(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('mipsel', 'buster')
+
+
+ def test_run_arm_bookworm(self):
+ """
+ :avocado: tags=startvm
+ """
+ self.init()
+ self.vm_start('arm', 'bookworm', image='isar-image-ci')
+ """
+ END: Run tests for test_standard_cross
+ """
+
+
def test_standard_rpi(self):
targets = [
'mc:rpi-arm-bullseye:isar-image-base',
@@ -470,75 +714,12 @@ class Source(CIBaseTest):
self.init()
self.perform_source_test(targets)
-class VmBootFast(CIBaseTest):
-
- """
- Test QEMU image start (fast)
-
- :avocado: tags=startvm,fast
- """
-
- def test_amd64_bullseye(self):
- self.init()
- self.vm_start('amd64', 'bullseye', image='isar-image-ci')
-
-
- def test_arm_bullseye_base(self):
- self.init()
- self.vm_start('arm','bullseye', image='isar-image-ci', keep=True)
-
- def test_arm_bullseye_example_module(self):
- self.init()
- self.vm_start('arm','bullseye', image='isar-image-ci',
- cmd='lsmod | grep example_module', keep=True)
-
- def test_arm_bullseye_getty_target(self):
- self.init()
- self.vm_start('arm','bullseye', image='isar-image-ci',
- script='test_systemd_unit.sh getty.target 10')
-
-
- def test_arm_buster_base(self):
- self.init()
- self.vm_start('arm','buster', image='isar-image-ci', keep=True)
-
- def test_arm_buster_getty_target(self):
- self.init()
- self.vm_start('arm','buster', image='isar-image-ci',
- cmd='systemctl is-active getty.target', keep=True)
-
- def test_arm_buster_example_module(self):
- self.init()
- self.vm_start('arm','buster', image='isar-image-ci',
- script='test_kernel_module.sh example_module')
-
-
- def test_arm_bookworm_base(self):
- self.init()
- self.vm_start('arm','bookworm', image='isar-image-ci', keep=True)
-
- def test_arm_bookworm_example_module(self):
- self.init()
- self.vm_start('arm','bookworm', image='isar-image-ci',
- cmd='lsmod | grep example_module', keep=True)
-
- def test_arm_bookworm_getty_target(self):
- self.init()
- self.vm_start('arm','bookworm', image='isar-image-ci',
- script='test_systemd_unit.sh getty.target 10')
-
-
- def test_arm64_focal(self):
- self.init()
- self.vm_start('arm64', 'focal', image='isar-image-base')
-
-
class VmBootFull(CIBaseTest):
"""
Test QEMU image start (full)
- :avocado: tags=startvm,standard,full
+ :avocado: tags=startvm,full
"""
def test_arm64_bullseye_base(self):
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 19/23] CI: Fix missing endline in generated config
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (17 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 18/23] CI: Execute startvm testcases after building Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 20/23] CI: Remove absolute path to citest.py Anton Mikanovich
` (3 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 256534dc..cc589b1f 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -149,7 +149,7 @@ class CIBuilder(Test):
if sstate_dir:
f.write('SSTATE_DIR = "%s"\n' % sstate_dir)
if image_install is not None:
- f.write('IMAGE_INSTALL = "%s"' % image_install)
+ f.write('IMAGE_INSTALL = "%s"\n' % image_install)
# include ci_build.conf in local.conf
with open(self.build_dir + '/conf/local.conf', 'r+') as f:
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 20/23] CI: Remove absolute path to citest.py
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (18 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 19/23] CI: Fix missing endline in generated config Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 21/23] CI: Create avocado.conf inside build_dir Anton Mikanovich
` (2 subsequent siblings)
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Do not pass absolute path to Avocado to simplify build logs.
Since every log line has path suffix we should try to minimize it.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 0aa2403d..7e841886 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -142,9 +142,11 @@ logs_dir = $(realpath "${BASE_DIR}")/job-results
EOF
export VIRTUAL_ENV="./"
+TESTFILE=$(realpath -s --relative-to=$(pwd) "${TESTSUITE_DIR}/citest.py")
+
# the real stuff starts here, trace commands from now on
set -x
-avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
+avocado ${VERBOSE} run "${TESTFILE}" \
-t "${TAGS}" --max-parallel-tasks=1 --disable-sysinfo \
${SSTATE} ${TIMEOUT}
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 21/23] CI: Create avocado.conf inside build_dir
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (19 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 20/23] CI: Remove absolute path to citest.py Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 22/23] CI: Set also cache dir for Avocado Anton Mikanovich
2024-03-25 15:55 ` [PATCH 23/23] CI: Add make world test case Anton Mikanovich
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To be sure all outputs are located inside it.
Also do not resolve symbolic links to keep original path naming.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 7e841886..cd4c8960 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -132,13 +132,13 @@ if echo "$TAGS" | grep -Fqive "-startvm"; then
fi
# Provide working path
-mkdir -p .config/avocado
-cat <<EOF > .config/avocado/avocado.conf
+mkdir -p "${BASE_DIR}"
+cat <<EOF > "${BASE_DIR}/avocado.conf"
[datadir.paths]
-base_dir = $(realpath "${BASE_DIR}")/
-test_dir = $(realpath "${BASE_DIR}")/tests
-data_dir = $(realpath "${BASE_DIR}")/data
-logs_dir = $(realpath "${BASE_DIR}")/job-results
+base_dir = $(realpath -s "${BASE_DIR}")/
+test_dir = $(realpath -s "${BASE_DIR}")/tests
+data_dir = $(realpath -s "${BASE_DIR}")/data
+logs_dir = $(realpath -s "${BASE_DIR}")/job-results
EOF
export VIRTUAL_ENV="./"
@@ -147,6 +147,6 @@ TESTFILE=$(realpath -s --relative-to=$(pwd) "${TESTSUITE_DIR}/citest.py")
# the real stuff starts here, trace commands from now on
set -x
-avocado ${VERBOSE} run "${TESTFILE}" \
+avocado ${VERBOSE} --config "${BASE_DIR}/avocado.conf" run "${TESTFILE}" \
-t "${TAGS}" --max-parallel-tasks=1 --disable-sysinfo \
${SSTATE} ${TIMEOUT}
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 22/23] CI: Set also cache dir for Avocado
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (20 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 21/23] CI: Create avocado.conf inside build_dir Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
2024-03-25 15:55 ` [PATCH 23/23] CI: Add make world test case Anton Mikanovich
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Otherwise it can fall back to unpredictable location.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index cd4c8960..5903cbe5 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -139,6 +139,7 @@ base_dir = $(realpath -s "${BASE_DIR}")/
test_dir = $(realpath -s "${BASE_DIR}")/tests
data_dir = $(realpath -s "${BASE_DIR}")/data
logs_dir = $(realpath -s "${BASE_DIR}")/job-results
+cache_dirs = ["$(realpath -s "${BASE_DIR}")/data/cache"]
EOF
export VIRTUAL_ENV="./"
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 23/23] CI: Add make world test case
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
` (21 preceding siblings ...)
2024-03-25 15:55 ` [PATCH 22/23] CI: Set also cache dir for Avocado Anton Mikanovich
@ 2024-03-25 15:55 ` Anton Mikanovich
22 siblings, 0 replies; 24+ messages in thread
From: Anton Mikanovich @ 2024-03-25 15:55 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Add test case which builds all available targets.
Single bitbake usage (directly from mc.conf):
$ avocado run testsuite/citest.py -t world
Multiple bitbakes usage (from generated yaml file):
$ avocado run testsuite/citest.py -t world -m testsuite/data/targets.yml
There is also a script to regenerate yaml from mc.conf:
$ ./testsuite/utils/targets_gen.py
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
.gitignore | 2 +-
scripts/ci_build.sh | 7 +-
testsuite/cibuilder.py | 21 ++++++
testsuite/citest.py | 24 +++++++
testsuite/data/targets.yml | 118 +++++++++++++++++++++++++++++++++
testsuite/utils/targets_gen.py | 27 ++++++++
6 files changed, 197 insertions(+), 2 deletions(-)
create mode 100644 testsuite/data/targets.yml
create mode 100755 testsuite/utils/targets_gen.py
diff --git a/.gitignore b/.gitignore
index b1b51dd4..7f5ab1ad 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,4 +2,4 @@
/*.patch
__pycache__
.config.yaml*
-build/
+build*
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 5903cbe5..c8ca84ad 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -49,6 +49,7 @@ show_help() {
echo " -b, --base BASE_DIR set path to base directory. If not set,"
echo " the tests will be started in current path."
echo " -d, --debug enable debug bitbake output."
+ echo " -m, --muxfile input file for yaml-to-mux plugin."
echo " -T, --tags specify basic avocado tags."
echo " --help display this message and exit."
echo
@@ -94,6 +95,10 @@ do
NORUN="1"
echo "warning: deprecated parameter '$key', consider using '-T <TAG>,-startvm' instead"
;;
+ -m|--muxfile)
+ MUXFILE="-m $2"
+ shift
+ ;;
-t|--timeout)
TIMEOUT="-p time_to_wait=$2"
shift
@@ -150,4 +155,4 @@ set -x
avocado ${VERBOSE} --config "${BASE_DIR}/avocado.conf" run "${TESTFILE}" \
-t "${TAGS}" --max-parallel-tasks=1 --disable-sysinfo \
- ${SSTATE} ${TIMEOUT}
+ ${SSTATE} ${MUXFILE} ${TIMEOUT}
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index cc589b1f..cef52e63 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -231,6 +231,15 @@ class CIBuilder(Test):
except FileNotFoundError:
self.log.warn(path + backup_prefix + ' not exist')
+ def get_test_images(self):
+ return ['isar-image-base', 'isar-image-ci']
+
+ def get_targets(self):
+ d = bb.data.init()
+ d.setVar('BBPATH', os.path.join(isar_root, 'meta-isar'))
+ d = bb.cookerdata.parse_config_file('conf/mc.conf', d, False)
+ return d.getVar('BBMULTICONFIG').split()
+
def getVars(self, *vars, target=None):
self.check_init()
def fixStream(stream):
@@ -263,6 +272,18 @@ class CIBuilder(Test):
values = values + (tinfoil.config_data.getVar(var, True) or 'None',)
return values if len(values) > 1 else values[0]
+ def gen_targets_yaml(self, fn='targets.yml'):
+ targetsfile = os.path.join(os.path.dirname(__file__), 'data', fn)
+ with open(targetsfile, 'w') as f:
+ f.write('a: !mux\n')
+ for target in self.get_targets():
+ f.write(f' {target}:\n name: {target}\n')
+ f.write('b: !mux\n')
+ prefix = 'isar-image-'
+ for image in self.get_test_images():
+ nodename = image[image.startswith(prefix) and len(prefix):]
+ f.write(f' {nodename}:\n image: {image}\n')
+
def create_tmp_layer(self):
tmp_layer_dir = os.path.join(isar_root, 'meta-tmp')
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 395b2965..7b504989 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -5,6 +5,7 @@ import os
from avocado import skipUnless
from avocado.utils import path
from cibase import CIBaseTest
+from cibuilder import isar_root
UMOCI_AVAILABLE = True
SKOPEO_AVAILABLE = True
@@ -808,3 +809,26 @@ class VmBootFull(CIBaseTest):
self.init()
self.vm_start('mipsel','bookworm', image='isar-image-ci',
script='test_kernel_module.sh example_module')
+
+class World(CIBaseTest):
+
+ """
+ All targets build test
+
+ :avocado: tags=world
+ """
+ def test_world(self):
+ name = self.params.get('name')
+ image = self.params.get('image', default='isar-image-ci')
+ targets = []
+
+ if name is None:
+ self.init()
+ for target in self.get_targets():
+ for image in self.get_test_images():
+ targets.append(f'mc:{target}:{image}')
+ else:
+ targets.append(f'mc:{name}:{image}')
+ self.init(f'build-{name}')
+
+ self.perform_build_test(targets, container=True)
diff --git a/testsuite/data/targets.yml b/testsuite/data/targets.yml
new file mode 100644
index 00000000..21fd0522
--- /dev/null
+++ b/testsuite/data/targets.yml
@@ -0,0 +1,118 @@
+a: !mux
+ qemuarm-buster:
+ name: qemuarm-buster
+ qemuarm-bullseye:
+ name: qemuarm-bullseye
+ qemuarm-bookworm:
+ name: qemuarm-bookworm
+ qemuarm-trixie:
+ name: qemuarm-trixie
+ qemuarm64-buster:
+ name: qemuarm64-buster
+ qemuarm64-bullseye:
+ name: qemuarm64-bullseye
+ qemuarm64-bookworm:
+ name: qemuarm64-bookworm
+ qemuarm64-trixie:
+ name: qemuarm64-trixie
+ qemui386-buster:
+ name: qemui386-buster
+ qemui386-bullseye:
+ name: qemui386-bullseye
+ qemui386-bookworm:
+ name: qemui386-bookworm
+ qemuamd64-buster:
+ name: qemuamd64-buster
+ qemuamd64-bullseye:
+ name: qemuamd64-bullseye
+ qemuamd64-sb-bullseye:
+ name: qemuamd64-sb-bullseye
+ qemuamd64-bookworm:
+ name: qemuamd64-bookworm
+ qemuamd64-trixie:
+ name: qemuamd64-trixie
+ container-amd64-buster:
+ name: container-amd64-buster
+ container-amd64-bullseye:
+ name: container-amd64-bullseye
+ container-amd64-bookworm:
+ name: container-amd64-bookworm
+ qemumipsel-buster:
+ name: qemumipsel-buster
+ qemumipsel-bullseye:
+ name: qemumipsel-bullseye
+ qemumipsel-bookworm:
+ name: qemumipsel-bookworm
+ qemuriscv64-sid:
+ name: qemuriscv64-sid
+ bananapi-buster:
+ name: bananapi-buster
+ bananapi-bullseye:
+ name: bananapi-bullseye
+ bananapi-bookworm:
+ name: bananapi-bookworm
+ beagleplay-bookworm:
+ name: beagleplay-bookworm
+ de0-nano-soc-buster:
+ name: de0-nano-soc-buster
+ de0-nano-soc-bullseye:
+ name: de0-nano-soc-bullseye
+ de0-nano-soc-bookworm:
+ name: de0-nano-soc-bookworm
+ hikey-bullseye:
+ name: hikey-bullseye
+ hikey-bookworm:
+ name: hikey-bookworm
+ imx6-sabrelite-buster:
+ name: imx6-sabrelite-buster
+ imx6-sabrelite-bullseye:
+ name: imx6-sabrelite-bullseye
+ phyboard-mira-bullseye:
+ name: phyboard-mira-bullseye
+ nanopi-neo-buster:
+ name: nanopi-neo-buster
+ nanopi-neo-bullseye:
+ name: nanopi-neo-bullseye
+ nanopi-neo-bookworm:
+ name: nanopi-neo-bookworm
+ nanopi-neo-efi-bookworm:
+ name: nanopi-neo-efi-bookworm
+ stm32mp15x-bullseye:
+ name: stm32mp15x-bullseye
+ virtualbox-bullseye:
+ name: virtualbox-bullseye
+ virtualbox-bookworm:
+ name: virtualbox-bookworm
+ rpi-arm-bullseye:
+ name: rpi-arm-bullseye
+ rpi-arm-bookworm:
+ name: rpi-arm-bookworm
+ rpi-arm-v7-bullseye:
+ name: rpi-arm-v7-bullseye
+ rpi-arm-v7-bookworm:
+ name: rpi-arm-v7-bookworm
+ rpi-arm-v7l-bullseye:
+ name: rpi-arm-v7l-bullseye
+ rpi-arm-v7l-bookworm:
+ name: rpi-arm-v7l-bookworm
+ rpi-arm64-v8-bullseye:
+ name: rpi-arm64-v8-bullseye
+ rpi-arm64-v8-bookworm:
+ name: rpi-arm64-v8-bookworm
+ sifive-fu540-sid:
+ name: sifive-fu540-sid
+ starfive-visionfive2-sid:
+ name: starfive-visionfive2-sid
+ qemuarm64-focal:
+ name: qemuarm64-focal
+ qemuarm64-jammy:
+ name: qemuarm64-jammy
+ qemuamd64-focal:
+ name: qemuamd64-focal
+ qemuamd64-jammy:
+ name: qemuamd64-jammy
+b: !mux
+ base:
+ image: isar-image-base
+ ci:
+ image: isar-image-ci
diff --git a/testsuite/utils/targets_gen.py b/testsuite/utils/targets_gen.py
new file mode 100755
index 00000000..49e00dd6
--- /dev/null
+++ b/testsuite/utils/targets_gen.py
@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+
+"""
+# This software is a part of Isar.
+# Copyright (C) 2024 ilbers GmbH
+
+# targets_gen.py: Generates yaml for yaml-to-mux Avocado varianter plugin.
+"""
+
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/..')
+from cibuilder import CIBuilder
+
+class TGen(CIBuilder):
+ def __init__(self):
+ super(CIBuilder, self).__init__()
+ self.gen_targets_yaml()
+ def test():
+ pass
+
+def main():
+ TGen()
+
+if __name__ == "__main__":
+ main()
--
2.34.1
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2024-03-25 15:56 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-25 15:55 [PATCH 00/23] Additional CI improvements Anton Mikanovich
2024-03-25 15:55 ` [PATCH 01/23] CI: Introduce Standard CI for cross building Anton Mikanovich
2024-03-25 15:55 ` [PATCH 02/23] CI: Rename cross to fast Anton Mikanovich
2024-03-25 15:55 ` [PATCH 03/23] CI: Rename nocross to full Anton Mikanovich
2024-03-25 15:55 ` [PATCH 04/23] CI: Remove test keyword from class names Anton Mikanovich
2024-03-25 15:55 ` [PATCH 05/23] CI: Rename base test cases Anton Mikanovich
2024-03-25 15:55 ` [PATCH 06/23] CI: Remove cleanup before nocross build Anton Mikanovich
2024-03-25 15:55 ` [PATCH 07/23] CI: Update default target to bookworm Anton Mikanovich
2024-03-25 15:55 ` [PATCH 08/23] CI: Remove duplicated targets Anton Mikanovich
2024-03-25 15:55 ` [PATCH 09/23] CI: Run startvm for all qemu targets Anton Mikanovich
2024-03-25 15:55 ` [PATCH 10/23] CI: Fix errors reporting during startvm Anton Mikanovich
2024-03-25 15:55 ` [PATCH 11/23] CI: Remove logging hack for job.log Anton Mikanovich
2024-03-25 15:55 ` [PATCH 12/23] CI: Do not lost errors on failed processes Anton Mikanovich
2024-03-25 15:55 ` [PATCH 13/23] CI: Remove container package list hack from testsuite Anton Mikanovich
2024-03-25 15:55 ` [PATCH 14/23] CI: Minimize debsrc_cache checking Anton Mikanovich
2024-03-25 15:55 ` [PATCH 15/23] CI: Minimize kselftest checking Anton Mikanovich
2024-03-25 15:55 ` [PATCH 16/23] CI: Do not include testsuites into each other Anton Mikanovich
2024-03-25 15:55 ` [PATCH 17/23] CI: Mark some single testcases with tags Anton Mikanovich
2024-03-25 15:55 ` [PATCH 18/23] CI: Execute startvm testcases after building Anton Mikanovich
2024-03-25 15:55 ` [PATCH 19/23] CI: Fix missing endline in generated config Anton Mikanovich
2024-03-25 15:55 ` [PATCH 20/23] CI: Remove absolute path to citest.py Anton Mikanovich
2024-03-25 15:55 ` [PATCH 21/23] CI: Create avocado.conf inside build_dir Anton Mikanovich
2024-03-25 15:55 ` [PATCH 22/23] CI: Set also cache dir for Avocado Anton Mikanovich
2024-03-25 15:55 ` [PATCH 23/23] CI: Add make world test case Anton Mikanovich
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox