[PATCH 0/3] Various improvements of the Isar testsuite

[PATCH 0/3] Various improvements of the Isar testsuite

['Felix Moessbauer' via isar-users]

The probably biggest improvement is to fully switch to cross build (p3).
There should really be no reason to build a large set of targets in
emulated mode, as the cross compile feature in Isar is mature and stable.

All patches of this series work standalone as well. Feel free to also
just partially apply them.

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (3):
  testsuite: compress sdk with zstd to speedup dev test
  testsuite: disable qemumipsel-bullseye tests in rootless mode
  testsuite: build all targets in cross mode

 testsuite/citest.py | 48 +++++++++++++++++++++++++++------------------
 1 file changed, 29 insertions(+), 19 deletions(-)

-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806103529.15597-1-felix.moessbauer%40siemens.com.

[PATCH 1/3] testsuite: compress sdk with zstd to speedup dev test

['Felix Moessbauer' via isar-users]

We only change it in the dev test, as this test is expected to finish as
quickly as possible. By that, we get even better test coverage, as other
tests still use xz compression.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/citest.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index fcd08c95..a41b77c8 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -73,7 +73,11 @@ class DevTest(CIBaseTest):
         ]
 
         self.init()
-        self.perform_build_test(targets, image_install='example-raw')
+        self.perform_build_test(
+            targets,
+            image_install='example-raw',
+            lines=['SDK_FORMATS = "tar.zst"'],
+        )
 
     def test_dev_run_arm_bookworm(self):
         self.init()
-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806103529.15597-2-felix.moessbauer%40siemens.com.

[PATCH 2/3] testsuite: disable qemumipsel-bullseye tests in rootless mode

['Felix Moessbauer' via isar-users]

The pam + adduser combination of qemumipsel on bullseye does not work in
unprivileged user namespaces. By that, no users can be added to the
image. This results in the following error during do_rootfs_install:

 chfn: PAM: System error
 adduser: `/bin/chfn -f systemd Network Management systemd-network' returned error code 1. Exiting.
 dpkg: error processing package systemd (--configure):
  installed systemd package post-installation script subprocess returned error exit status 1

As this target and distro combination is exotic, we just remove it from
the testsuite if running in rootless mode.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/citest.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index a41b77c8..211bc16d 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -928,7 +928,6 @@ class NoCrossTest(CIBaseTest):
             'mc:qemui386-bullseye:isar-image-base',
             'mc:qemuamd64-buster:isar-image-ci',
             'mc:qemuamd64-bullseye:isar-initramfs',
-            'mc:qemumipsel-bullseye:isar-image-base',
             'mc:phyboard-mira-bullseye:isar-image-base',
             'mc:hikey-bullseye:isar-image-base',
             'mc:virtualbox-bullseye:isar-image-base',
@@ -959,6 +958,11 @@ class NoCrossTest(CIBaseTest):
             'mc:x86-pc-bookworm:isar-image-base',
         ]
 
+        if not bool(int(self.params.get('rootless', default=0))):
+            # on bullseye mipsel, the pam + adduser combination does not
+            # work in unprivileged namespaces
+            targets.append('mc:qemumipsel-bullseye:isar-image-base')
+
         self.init()
         self.perform_build_test(targets, cross=False)
 
@@ -1120,6 +1124,8 @@ class NoCrossTest(CIBaseTest):
         """
         :avocado: tags=startvm
         """
+        if bool(int(self.params.get('rootless', default=0))):
+            self.cancel('mipsel bullseye not supported in rootless mode')
         self.init()
         self.vm_start('mipsel', 'bullseye')
 
-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806103529.15597-3-felix.moessbauer%40siemens.com.

[PATCH 3/3] testsuite: build all targets in cross mode

['Felix Moessbauer' via isar-users]

Cross compiling is supported by isar for a long time, and nowadays also
is the default. Recipes that explicitly don't support cross compilation
are already annotated.

By that, we switch all targets in CI to cross compilation and rename the
now-misleading NoCross test to AllTargets. This gives a massive speedup
on the full CI run, as also the kernels are not compiled via emulation
anymore.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/citest.py | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 211bc16d..2ba7962b 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -202,7 +202,7 @@ class ReproTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_repro_test(targets, cross=False)
+            self.perform_repro_test(targets)
         finally:
             self.move_in_build_dir('tmp', 'tmp_repro_unsigned')
 
@@ -437,7 +437,7 @@ class CrossTest(CIBaseTest):
         ]
 
         self.init()
-        self.perform_build_test(targets, cross=False)
+        self.perform_build_test(targets)
 
     def test_cross_kselftest(self):
         targets = [
@@ -910,15 +910,15 @@ class DtbDeployTest(CIBaseTest):
             self.move_in_build_dir('tmp', 'tmp_dtbdeploy')
 
 
-class NoCrossTest(CIBaseTest):
+class AllTargetsTest(CIBaseTest):
 
     """
-    Start non-cross build for the defined set of configurations
+    Build all defined targets
 
-    :avocado: tags=nocross,full
+    :avocado: tags=alltargets,full
     """
 
-    def test_nocross(self):
+    def test_all_targets(self):
         targets = [
             'mc:qemuarm-buster:isar-image-ci',
             'mc:qemuarm-bullseye:isar-image-base',
@@ -964,7 +964,7 @@ class NoCrossTest(CIBaseTest):
             targets.append('mc:qemumipsel-bullseye:isar-image-base')
 
         self.init()
-        self.perform_build_test(targets, cross=False)
+        self.perform_build_test(targets)
 
     def test_run_arm_bullseye(self):
         """
@@ -1234,7 +1234,7 @@ class NoCrossTest(CIBaseTest):
         self.vm_start('amd64-iso', 'bookworm', image='isar-image-ci',
                       script='test_system_running.sh 30')
 
-    def test_nocross_debsrc(self):
+    def test_debsrc(self):
         targets = [
             'mc:qemuarm-bookworm:isar-image-ci',
             'mc:stm32mp15x-bullseye:isar-image-base',
@@ -1242,7 +1242,7 @@ class NoCrossTest(CIBaseTest):
         ]
 
         self.init()
-        self.perform_build_test(targets, cross=False, debsrc_cache=True)
+        self.perform_build_test(targets, debsrc_cache=True)
 
     def test_run_arm_bookworm(self):
         """
@@ -1251,7 +1251,7 @@ class NoCrossTest(CIBaseTest):
         self.init()
         self.vm_start('arm', 'bookworm', image='isar-image-ci')
 
-    def test_nocross_rpi(self):
+    def test_rpi(self):
         targets = [
             'mc:rpi-arm-bullseye:isar-image-base',
             'mc:rpi-arm-trixie:isar-image-base',
@@ -1264,27 +1264,27 @@ class NoCrossTest(CIBaseTest):
         ]
 
         self.init()
-        self.perform_build_test(targets, cross=False)
+        self.perform_build_test(targets)
 
-    def test_nocross_rpi_debsrc(self):
+    def test_rpi_debsrc(self):
         targets = [
             'mc:rpi-arm-bookworm:isar-image-base',
             'mc:rpi-arm-v7l-bookworm:isar-image-base',
         ]
 
         self.init()
-        self.perform_build_test(targets, cross=False, debsrc_cache=True)
+        self.perform_build_test(targets, debsrc_cache=True)
 
-    def test_nocross_riscv_trixie(self):
+    def test_riscv_trixie(self):
         targets = [
             'mc:sifive-fu540-trixie:isar-image-base',
             'mc:starfive-visionfive2-trixie:isar-image-base',
         ]
 
         self.init()
-        self.perform_build_test(targets, cross=False)
+        self.perform_build_test(targets)
 
-    def test_nocross_sid(self):
+    def test_sid(self):
         targets = [
             'mc:qemuamd64-sid:isar-image-base',
             'mc:qemuarm64-sid:isar-image-base',
@@ -1292,7 +1292,7 @@ class NoCrossTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_build_test(targets, cross=False)
+            self.perform_build_test(targets)
         except exceptions.TestFail:
             self.cancel('KFAIL')
 
-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806103529.15597-4-felix.moessbauer%40siemens.com.

Re: [PATCH 3/3] testsuite: build all targets in cross mode

[Zhihang Wei]

On 8/6/26 12:35, 'Felix Moessbauer' via isar-users wrote:
> Cross compiling is supported by isar for a long time, and nowadays also
> is the default. Recipes that explicitly don't support cross compilation
> are already annotated.
>
> By that, we switch all targets in CI to cross compilation and rename the
> now-misleading NoCross test to AllTargets. This gives a massive speedup
> on the full CI run, as also the kernels are not compiled via emulation
> anymore.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Hi,

Native compilation was originally requested because some packages are not
cross-buildable. Those packages still exist, so I don't think we can drop
native building entirely.

If we keep it, the question is what the proper testing scope should be.
I'm afraid the scope is still full.

So we probably still need to keep both aspects (cross vs. native, and
some-targets vs. all-targets) but split them explicitly in the code. The
exact implementation is open, e.g. separate full-cross / full-native jobs,
or only full with like TEST_CROSS=1.

IIRC Anton has been working on this topic, see [1]. It's better to hear
his opinion as well, he'll be back next week.

[1] 
https://lists.isar-build.org/isar-users/20240325155540.4162990-1-amikan@ilbers.de/

Zhihang

> ---
>   testsuite/citest.py | 34 +++++++++++++++++-----------------
>   1 file changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/testsuite/citest.py b/testsuite/citest.py
> index 211bc16d..2ba7962b 100644
> --- a/testsuite/citest.py
> +++ b/testsuite/citest.py
> @@ -202,7 +202,7 @@ class ReproTest(CIBaseTest):
>   
>           self.init()
>           try:
> -            self.perform_repro_test(targets, cross=False)
> +            self.perform_repro_test(targets)
>           finally:
>               self.move_in_build_dir('tmp', 'tmp_repro_unsigned')
>   
> @@ -437,7 +437,7 @@ class CrossTest(CIBaseTest):
>           ]
>   
>           self.init()
> -        self.perform_build_test(targets, cross=False)
> +        self.perform_build_test(targets)
>   
>       def test_cross_kselftest(self):
>           targets = [
> @@ -910,15 +910,15 @@ class DtbDeployTest(CIBaseTest):
>               self.move_in_build_dir('tmp', 'tmp_dtbdeploy')
>   
>   
> -class NoCrossTest(CIBaseTest):
> +class AllTargetsTest(CIBaseTest):
>   
>       """
> -    Start non-cross build for the defined set of configurations
> +    Build all defined targets
>   
> -    :avocado: tags=nocross,full
> +    :avocado: tags=alltargets,full
>       """
>   
> -    def test_nocross(self):
> +    def test_all_targets(self):
>           targets = [
>               'mc:qemuarm-buster:isar-image-ci',
>               'mc:qemuarm-bullseye:isar-image-base',
> @@ -964,7 +964,7 @@ class NoCrossTest(CIBaseTest):
>               targets.append('mc:qemumipsel-bullseye:isar-image-base')
>   
>           self.init()
> -        self.perform_build_test(targets, cross=False)
> +        self.perform_build_test(targets)
>   
>       def test_run_arm_bullseye(self):
>           """
> @@ -1234,7 +1234,7 @@ class NoCrossTest(CIBaseTest):
>           self.vm_start('amd64-iso', 'bookworm', image='isar-image-ci',
>                         script='test_system_running.sh 30')
>   
> -    def test_nocross_debsrc(self):
> +    def test_debsrc(self):
>           targets = [
>               'mc:qemuarm-bookworm:isar-image-ci',
>               'mc:stm32mp15x-bullseye:isar-image-base',
> @@ -1242,7 +1242,7 @@ class NoCrossTest(CIBaseTest):
>           ]
>   
>           self.init()
> -        self.perform_build_test(targets, cross=False, debsrc_cache=True)
> +        self.perform_build_test(targets, debsrc_cache=True)
>   
>       def test_run_arm_bookworm(self):
>           """
> @@ -1251,7 +1251,7 @@ class NoCrossTest(CIBaseTest):
>           self.init()
>           self.vm_start('arm', 'bookworm', image='isar-image-ci')
>   
> -    def test_nocross_rpi(self):
> +    def test_rpi(self):
>           targets = [
>               'mc:rpi-arm-bullseye:isar-image-base',
>               'mc:rpi-arm-trixie:isar-image-base',
> @@ -1264,27 +1264,27 @@ class NoCrossTest(CIBaseTest):
>           ]
>   
>           self.init()
> -        self.perform_build_test(targets, cross=False)
> +        self.perform_build_test(targets)
>   
> -    def test_nocross_rpi_debsrc(self):
> +    def test_rpi_debsrc(self):
>           targets = [
>               'mc:rpi-arm-bookworm:isar-image-base',
>               'mc:rpi-arm-v7l-bookworm:isar-image-base',
>           ]
>   
>           self.init()
> -        self.perform_build_test(targets, cross=False, debsrc_cache=True)
> +        self.perform_build_test(targets, debsrc_cache=True)
>   
> -    def test_nocross_riscv_trixie(self):
> +    def test_riscv_trixie(self):
>           targets = [
>               'mc:sifive-fu540-trixie:isar-image-base',
>               'mc:starfive-visionfive2-trixie:isar-image-base',
>           ]
>   
>           self.init()
> -        self.perform_build_test(targets, cross=False)
> +        self.perform_build_test(targets)
>   
> -    def test_nocross_sid(self):
> +    def test_sid(self):
>           targets = [
>               'mc:qemuamd64-sid:isar-image-base',
>               'mc:qemuarm64-sid:isar-image-base',
> @@ -1292,7 +1292,7 @@ class NoCrossTest(CIBaseTest):
>   
>           self.init()
>           try:
> -            self.perform_build_test(targets, cross=False)
> +            self.perform_build_test(targets)
>           except exceptions.TestFail:
>               self.cancel('KFAIL')
>   

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/3da676b0-3e97-41ef-a489-a75c81e6e58d%40ilbers.de.

Re: [PATCH 3/3] testsuite: build all targets in cross mode

['MOESSBAUER, Felix' via isar-users]

On Thu, 2026-08-06 at 16:06 +0200, Zhihang Wei wrote:
> On 8/6/26 12:35, 'Felix Moessbauer' via isar-users wrote:
> > Cross compiling is supported by isar for a long time, and nowadays also
> > is the default. Recipes that explicitly don't support cross compilation
> > are already annotated.
> > 
> > By that, we switch all targets in CI to cross compilation and rename the
> > now-misleading NoCross test to AllTargets. This gives a massive speedup
> > on the full CI run, as also the kernels are not compiled via emulation
> > anymore.
> > 
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> 
> Hi,
> 
> Native compilation was originally requested because some packages are not
> cross-buildable. Those packages still exist, so I don't think we can drop
> native building entirely.

Agree. But this should IMHO be done on a per-package level (we can
simply sprinkle a ISAR_CROSS_COMPILE=0 into the recipes of the
incompatible packages). We already do this today. This series is
actually tested on -t alltargets :)

> 
> If we keep it, the question is what the proper testing scope should be.
> I'm afraid the scope is still full.

Can't we use the -p cross override, maybe by making it a tristate
(0,1,auto) or something like that? The emulated compilation of the
kernels is *horribly* slow, while we at the same time miss targets by
not running the full-ci because it takes too long.

My hope would be, that we can build the whole testsuite in both cross
and no-cross, whereby the recipes internally fall back to emulation if
cross is not supported.

> 
> So we probably still need to keep both aspects (cross vs. native, and
> some-targets vs. all-targets) but split them explicitly in the code. The
> exact implementation is open, e.g. separate full-cross / full-native jobs,
> or only full with like TEST_CROSS=1.

Both possible. For me it is important to have a fast CI with broad
coverage of targets. Next week I will send out patches to support
bitbake hash equivalence . This makes the testsuite blazing fast when
re-running in most cases. I'm already running this locally for a couple
of weeks and fine-tuning it. And it only needs ~150GB of sstate cache.

> 
> IIRC Anton has been working on this topic, see [1]. It's better to hear
> his opinion as well, he'll be back next week.

Sure! It's not urgent.

PS: feel free to apply patches 1 and 2 already.

Felix

> 
> [1] 
> https://lists.isar-build.org/isar-users/20240325155540.4162990-1-amikan@ilbers.de/
> 
> Zhihang
> 
> > ---
> >   testsuite/citest.py | 34 +++++++++++++++++-----------------
> >   1 file changed, 17 insertions(+), 17 deletions(-)
> > 
> > diff --git a/testsuite/citest.py b/testsuite/citest.py
> > index 211bc16d..2ba7962b 100644
> > --- a/testsuite/citest.py
> > +++ b/testsuite/citest.py
> > @@ -202,7 +202,7 @@ class ReproTest(CIBaseTest):
> >   
> >           self.init()
> >           try:
> > -            self.perform_repro_test(targets, cross=False)
> > +            self.perform_repro_test(targets)
> >           finally:
> >               self.move_in_build_dir('tmp', 'tmp_repro_unsigned')
> >   
> > @@ -437,7 +437,7 @@ class CrossTest(CIBaseTest):
> >           ]
> >   
> >           self.init()
> > -        self.perform_build_test(targets, cross=False)
> > +        self.perform_build_test(targets)
> >   
> >       def test_cross_kselftest(self):
> >           targets = [
> > @@ -910,15 +910,15 @@ class DtbDeployTest(CIBaseTest):
> >               self.move_in_build_dir('tmp', 'tmp_dtbdeploy')
> >   
> >   
> > -class NoCrossTest(CIBaseTest):
> > +class AllTargetsTest(CIBaseTest):
> >   
> >       """
> > -    Start non-cross build for the defined set of configurations
> > +    Build all defined targets
> >   
> > -    :avocado: tags=nocross,full
> > +    :avocado: tags=alltargets,full
> >       """
> >   
> > -    def test_nocross(self):
> > +    def test_all_targets(self):
> >           targets = [
> >               'mc:qemuarm-buster:isar-image-ci',
> >               'mc:qemuarm-bullseye:isar-image-base',
> > @@ -964,7 +964,7 @@ class NoCrossTest(CIBaseTest):
> >               targets.append('mc:qemumipsel-bullseye:isar-image-base')
> >   
> >           self.init()
> > -        self.perform_build_test(targets, cross=False)
> > +        self.perform_build_test(targets)
> >   
> >       def test_run_arm_bullseye(self):
> >           """
> > @@ -1234,7 +1234,7 @@ class NoCrossTest(CIBaseTest):
> >           self.vm_start('amd64-iso', 'bookworm', image='isar-image-ci',
> >                         script='test_system_running.sh 30')
> >   
> > -    def test_nocross_debsrc(self):
> > +    def test_debsrc(self):
> >           targets = [
> >               'mc:qemuarm-bookworm:isar-image-ci',
> >               'mc:stm32mp15x-bullseye:isar-image-base',
> > @@ -1242,7 +1242,7 @@ class NoCrossTest(CIBaseTest):
> >           ]
> >   
> >           self.init()
> > -        self.perform_build_test(targets, cross=False, debsrc_cache=True)
> > +        self.perform_build_test(targets, debsrc_cache=True)
> >   
> >       def test_run_arm_bookworm(self):
> >           """
> > @@ -1251,7 +1251,7 @@ class NoCrossTest(CIBaseTest):
> >           self.init()
> >           self.vm_start('arm', 'bookworm', image='isar-image-ci')
> >   
> > -    def test_nocross_rpi(self):
> > +    def test_rpi(self):
> >           targets = [
> >               'mc:rpi-arm-bullseye:isar-image-base',
> >               'mc:rpi-arm-trixie:isar-image-base',
> > @@ -1264,27 +1264,27 @@ class NoCrossTest(CIBaseTest):
> >           ]
> >   
> >           self.init()
> > -        self.perform_build_test(targets, cross=False)
> > +        self.perform_build_test(targets)
> >   
> > -    def test_nocross_rpi_debsrc(self):
> > +    def test_rpi_debsrc(self):
> >           targets = [
> >               'mc:rpi-arm-bookworm:isar-image-base',
> >               'mc:rpi-arm-v7l-bookworm:isar-image-base',
> >           ]
> >   
> >           self.init()
> > -        self.perform_build_test(targets, cross=False, debsrc_cache=True)
> > +        self.perform_build_test(targets, debsrc_cache=True)
> >   
> > -    def test_nocross_riscv_trixie(self):
> > +    def test_riscv_trixie(self):
> >           targets = [
> >               'mc:sifive-fu540-trixie:isar-image-base',
> >               'mc:starfive-visionfive2-trixie:isar-image-base',
> >           ]
> >   
> >           self.init()
> > -        self.perform_build_test(targets, cross=False)
> > +        self.perform_build_test(targets)
> >   
> > -    def test_nocross_sid(self):
> > +    def test_sid(self):
> >           targets = [
> >               'mc:qemuamd64-sid:isar-image-base',
> >               'mc:qemuarm64-sid:isar-image-base',
> > @@ -1292,7 +1292,7 @@ class NoCrossTest(CIBaseTest):
> >   
> >           self.init()
> >           try:
> > -            self.perform_build_test(targets, cross=False)
> > +            self.perform_build_test(targets)
> >           except exceptions.TestFail:
> >               self.cancel('KFAIL')
> >   

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/6fa22d8e134b96132d2a23e9ce8e64c4072900b0.camel%40siemens.com.