[PATCH v2 00/15] Add developers test

[PATCH v2 00/15] Add developers test

[Anton Mikanovich]

Add separate quick 'developers' test for fast checking new patches.
It will test only general Isar logic on basic targets without dealing with
all the functionality.
To execute dev test run avocado testsuite like:

$ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1

Estimated execution time for dev test is ~1.5h.

Changes sinse v1:
- Rebase on next.
- Integrate some changes from other patchsets.
- Remove ci_build.sh params.
- Improve testsuite API.
- Improve testsuite readme.
- Add single target test case.

Anton Mikanovich (10):
  CI: Enable downloads dir sharing
  CI: Add IMAGE_INSTALL override
  CI: Add developers test
  CI: Remove duplicated targets
  CI: Get rid of setting cross from outside
  CI: Make cross compile to be default
  CI: Get rid of setting repro from outside
  CI: Allow example module checking disabling
  CI: Update testsuite documentation
  CI: Add single target test case

Henning Schild (4):
  CI: move to avocado to 99.0
  CI: fix shell coding style
  CI: install qemu-system when qemu testing is requested
  testsuite: remove tests from "fast" set

Uladzimir Bely (1):
  testsuite: Fix terminal broken after run test executed

 .gitlab-ci.yml         |   4 +-
 scripts/ci_build.sh    |  46 +++++---------
 testsuite/README.md    |  26 ++++++--
 testsuite/cibuilder.py |  24 +++++--
 testsuite/citest.py    | 141 ++++++++++++++++++++++++-----------------
 5 files changed, 139 insertions(+), 102 deletions(-)

-- 
2.34.1

[PATCH v2 01/15] CI: Enable downloads dir sharing

[Anton Mikanovich]

Test cases performed in different build dir now can share DL_DIR.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/cibuilder.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 2e4702a..d874393 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -54,7 +54,8 @@ class CIBuilder(Test):
 
     def configure(self, compat_arch=True, cross=None, debsrc_cache=False,
                   container=False, ccache=False, sstate=False, offline=False,
-                  gpg_pub_key=None, wic_deploy_parts=False, source_date_epoch=None, **kwargs):
+                  gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
+                  source_date_epoch=None, **kwargs):
         # write configuration file and set bitbake_args
         # can run multiple times per test case
         self.check_init()
@@ -64,6 +65,9 @@ class CIBuilder(Test):
         if cross is None:
             cross = bool(int(self.params.get('cross', default=0)))
 
+        if dl_dir is None:
+            dl_dir = os.path.join(isar_root, 'downloads')
+
         # get parameters from environment
         distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS')
 
@@ -79,6 +83,7 @@ class CIBuilder(Test):
                       f'  gpg_pub_key = {gpg_pub_key}\n'
                       f'  wic_deploy_parts = {wic_deploy_parts}\n'
                       f'  source_date_epoch = {source_date_epoch} \n'
+                      f'  dl_dir = {dl_dir}\n'
                       f'===================================================')
 
         # determine bitbake_args
@@ -115,6 +120,8 @@ class CIBuilder(Test):
                 f.write('CCACHE_TOP_DIR = "${TOPDIR}/ccache"\n')
             if source_date_epoch:
                 f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch)
+            if dl_dir:
+                f.write('DL_DIR = "%s"\n' % dl_dir)
 
         # include ci_build.conf in local.conf
         with open(self.build_dir + '/conf/local.conf', 'r+') as f:
-- 
2.34.1

[PATCH v2 02/15] CI: Add IMAGE_INSTALL override

[Anton Mikanovich]

Allow user to override the list of packages to be built.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/cibuilder.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index d874393..4e55121 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -55,7 +55,7 @@ class CIBuilder(Test):
     def configure(self, compat_arch=True, cross=None, debsrc_cache=False,
                   container=False, ccache=False, sstate=False, offline=False,
                   gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
-                  source_date_epoch=None, **kwargs):
+                  source_date_epoch=None, image_install=None, **kwargs):
         # write configuration file and set bitbake_args
         # can run multiple times per test case
         self.check_init()
@@ -84,6 +84,7 @@ class CIBuilder(Test):
                       f'  wic_deploy_parts = {wic_deploy_parts}\n'
                       f'  source_date_epoch = {source_date_epoch} \n'
                       f'  dl_dir = {dl_dir}\n'
+                      f'  image_install = {image_install}\n'
                       f'===================================================')
 
         # determine bitbake_args
@@ -122,6 +123,8 @@ class CIBuilder(Test):
                 f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch)
             if dl_dir:
                 f.write('DL_DIR = "%s"\n' % dl_dir)
+            if image_install is not None:
+                f.write('IMAGE_INSTALL = "%s"' % image_install)
 
         # include ci_build.conf in local.conf
         with open(self.build_dir + '/conf/local.conf', 'r+') as f:
-- 
2.34.1

[PATCH v2 03/15] CI: Add developers test

[Anton Mikanovich]

Developers test consist of minimal test cases for patches checking.
To ensure dev test is 'always green' include it into full CI.

Dev test can be executed using 'dev' avocado tag.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/citest.py | 54 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 7aa1e6b..ec5bbf4 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -17,6 +17,60 @@ try:
 except path.CmdNotFoundError:
     SKOPEO_AVAILABLE = False
 
+class DevTest(CIBaseTest):
+
+    """
+    Developer's test
+
+    :avocado: tags=dev,fast,full
+    """
+    def test_dev(self):
+        targets = [
+            'mc:qemuamd64-bullseye:isar-image-base',
+            'mc:qemuarm-bullseye:isar-image-base',
+            'mc:qemuarm-bullseye:isar-image-base:do_populate_sdk',
+            'mc:qemuarm64-bullseye:isar-image-base'
+                  ]
+
+        self.init()
+        self.perform_build_test(targets, cross=True, image_install="example-raw")
+
+    def test_dev_apps(self):
+        targets = [
+            'mc:qemuamd64-bullseye:isar-image-base',
+            'mc:qemuarm64-bullseye:isar-image-base'
+                  ]
+
+        self.init()
+        self.perform_build_test(targets)
+
+    def test_dev_rebuild(self):
+        self.init()
+        layerdir_core = self.getlayerdir('core')
+
+        dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass'
+
+        self.backupfile(dpkgbase_file)
+        with open(dpkgbase_file, 'a') as file:
+            file.write('do_fetch_append() {\n\n}')
+
+        try:
+            self.perform_build_test('mc:qemuamd64-bullseye:isar-image-base')
+        finally:
+            self.restorefile(dpkgbase_file)
+
+    def test_dev_run_amd64_bullseye(self):
+        self.init()
+        self.vm_start('amd64', 'bullseye')
+
+    def test_dev_run_arm64_bullseye(self):
+        self.init()
+        self.vm_start('arm64', 'bullseye')
+
+    def test_dev_run_arm_bullseye(self):
+        self.init()
+        self.vm_start('arm', 'bullseye')
+
 class ReproTest(CIBaseTest):
 
     """
-- 
2.34.1

[PATCH v2 04/15] CI: Remove duplicated targets

[Anton Mikanovich]

Some targets are already covered by dev tests, so remove retesting.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/citest.py | 49 ---------------------------------------------
 1 file changed, 49 deletions(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index ec5bbf4..8414ef4 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -125,7 +125,6 @@ class CrossTest(CIBaseTest):
         targets = [
             'mc:qemuarm-buster:isar-image-base',
             'mc:qemuarm-bullseye:isar-image-base',
-            'mc:qemuarm64-bullseye:isar-image-base',
             'mc:de0-nano-soc-bullseye:isar-image-base',
             'mc:stm32mp15x-buster:isar-image-base'
                   ]
@@ -167,19 +166,6 @@ class CrossTest(CIBaseTest):
         except:
             self.cancel('KFAIL')
 
-class SdkTest(CIBaseTest):
-
-    """
-    In addition test SDK creation
-
-    :avocado: tags=sdk,fast,full
-    """
-    def test_sdk(self):
-        targets = ['mc:qemuarm-bullseye:isar-image-base']
-
-        self.init()
-        self.perform_build_test(targets, bitbake_cmd='do_populate_sdk')
-
 class WicTest(CIBaseTest):
 
     """
@@ -218,7 +204,6 @@ class NoCrossTest(CIBaseTest):
             'mc:qemui386-buster:isar-image-base',
             'mc:qemui386-bullseye:isar-image-base',
             'mc:qemuamd64-buster:isar-image-base',
-            'mc:qemuamd64-bullseye:isar-image-base',
             'mc:qemuamd64-bullseye:isar-initramfs',
             'mc:qemumipsel-buster:isar-image-base',
             'mc:qemumipsel-bullseye:isar-image-base',
@@ -278,28 +263,6 @@ class NoCrossTest(CIBaseTest):
         except:
             self.cancel('KFAIL')
 
-class RebuildTest(CIBaseTest):
-
-    """
-    Test image rebuild
-
-    :avocado: tags=rebuild,fast,full
-    """
-    def test_rebuild(self):
-        self.init()
-        layerdir_core = self.getlayerdir('core')
-
-        dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass'
-
-        self.backupfile(dpkgbase_file)
-        with open(dpkgbase_file, 'a') as file:
-            file.write('do_fetch_append() {\n\n}')
-
-        try:
-            self.perform_build_test('mc:qemuamd64-bullseye:isar-image-base', debsrc_cache=True)
-        finally:
-            self.restorefile(dpkgbase_file)
-
 class ContainerImageTest(CIBaseTest):
 
     """
@@ -361,14 +324,6 @@ class VmBootTestFast(CIBaseTest):
         self.init()
         self.vm_start('arm','buster')
 
-    def test_arm64_bullseye(self):
-        self.init()
-        self.vm_start('arm64','bullseye')
-
-    def test_amd64_bullseye(self):
-        self.init()
-        self.vm_start('amd64','bullseye')
-
     def test_arm_bookworm(self):
         self.init()
         self.vm_start('arm','bookworm')
@@ -392,10 +347,6 @@ class VmBootTestFull(CIBaseTest):
         self.init()
         self.vm_start('arm64','bullseye')
 
-    def test_amd64_bullseye(self):
-        self.init()
-        self.vm_start('amd64','bullseye')
-
     def test_i386_buster(self):
         self.init()
         self.vm_start('i386','buster')
-- 
2.34.1

[PATCH v2 05/15] CI: move to avocado to 99.0

[Anton Mikanovich]

From: Henning Schild <henning.schild@siemens.com>

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 scripts/ci_build.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index fa7208c..84a3e47 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
     rm -rf /tmp/avocado_venv
     virtualenv --python python3 /tmp/avocado_venv
     source /tmp/avocado_venv/bin/activate
-    pip install avocado-framework==96.0
+    pip install avocado-framework==99.0
 fi
 
 # Get Avocado build tests path
-- 
2.34.1

Re: [PATCH v2 05/15] CI: move to avocado to 99.0

[Henning Schild]

Am Tue, 24 Jan 2023 22:45:48 +0200
schrieb Anton Mikanovich <amikan@ilbers.de>:

> From: Henning Schild <henning.schild@siemens.com>
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  scripts/ci_build.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index fa7208c..84a3e47 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
>      rm -rf /tmp/avocado_venv
>      virtualenv --python python3 /tmp/avocado_venv
>      source /tmp/avocado_venv/bin/activate
> -    pip install avocado-framework==96.0
> +    pip install avocado-framework==99.0

100.1 in the meantime, or we maybe step back to 92.0 LTS

Not sure, we can also take that 99.

Henning

>  fi
>  
>  # Get Avocado build tests path

Re: [PATCH v2 05/15] CI: move to avocado to 99.0

[Anton Mikanovich]

25/01/2023 09:10, Henning Schild wrote:
> 100.1 in the meantime, or we maybe step back to 92.0 LTS
>
> Not sure, we can also take that 99.
>
> Henning

Avocado version 100 has some incompatible params syntax changes (prefixes
--nrunner- were removed and no more supported), so it looks like 99 is the
latest compatible version currently.

Re: [PATCH v2 05/15] CI: move to avocado to 99.0

[Henning Schild]

Am Wed, 25 Jan 2023 09:42:10 +0200
schrieb Anton Mikanovich <amikan@ilbers.de>:

> 25/01/2023 09:10, Henning Schild wrote:
> > 100.1 in the meantime, or we maybe step back to 92.0 LTS
> >
> > Not sure, we can also take that 99.
> >
> > Henning  
> 
> Avocado version 100 has some incompatible params syntax changes
> (prefixes --nrunner- were removed and no more supported), so it looks
> like 99 is the latest compatible version currently.

Again a user interface change ...

Maybe we can skip that nrunner thing now and ilbers also need to go to
100+

Henning

[PATCH v2 06/15] CI: fix shell coding style

[Anton Mikanovich]

From: Henning Schild <henning.schild@siemens.com>

Apply some suggestions from shellcheck and make sure that runs fine.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 scripts/ci_build.sh | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 84a3e47..e5f20d4 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -20,6 +20,7 @@ if ! command -v avocado > /dev/null; then
     sudo apt-get install -y virtualenv
     rm -rf /tmp/avocado_venv
     virtualenv --python python3 /tmp/avocado_venv
+    # shellcheck source=/dev/null
     source /tmp/avocado_venv/bin/activate
     pip install avocado-framework==99.0
 fi
@@ -33,8 +34,8 @@ BASE_DIR=./build
 # Check dependencies
 DEPENDENCIES="umoci skopeo"
 for prog in ${DEPENDENCIES} ; do
-    if [ ! -x "$(which $prog)" ] ; then
-        echo "missing $prog in PATH" >&2
+    if ! command -v "${prog}" > /dev/null; then
+        echo "missing ${prog} in PATH" >&2
     fi
 done
 
@@ -132,16 +133,16 @@ fi
 mkdir -p .config/avocado
 cat <<EOF > .config/avocado/avocado.conf
 [datadir.paths]
-base_dir = $(realpath $BASE_DIR)/
-test_dir = $(realpath $BASE_DIR)/tests
-data_dir = $(realpath $BASE_DIR)/data
-logs_dir = $(realpath $BASE_DIR)/job-results
+base_dir = $(realpath "${BASE_DIR}")/
+test_dir = $(realpath "${BASE_DIR}")/tests
+data_dir = $(realpath "${BASE_DIR}")/data
+logs_dir = $(realpath "${BASE_DIR}")/job-results
 EOF
 export VIRTUAL_ENV="./"
 
 # the real stuff starts here, trace commands from now on
 set -x
 
-avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
-    -t $TAGS --nrunner-max-parallel-tasks=1 --disable-sysinfo \
-    -p quiet=$QUIET -p cross=$CROSS_BUILD -p time_to_wait=$TIMEOUT
+avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
+    -t "${TAGS}" --nrunner-max-parallel-tasks=1 --disable-sysinfo \
+    -p quiet="${QUIET}" -p cross="${CROSS_BUILD}" -p time_to_wait="${TIMEOUT}"
-- 
2.34.1

[PATCH v2 07/15] CI: install qemu-system when qemu testing is requested

[Anton Mikanovich]

From: Henning Schild <henning.schild@siemens.com>

And also change gitlab ci config to request qemu testing.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .gitlab-ci.yml      | 4 ++--
 scripts/ci_build.sh | 5 +++++
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fcdefa8..6a49ce4 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,7 +27,7 @@ fast-ci:
   except:
     - schedules
   script:
-    - scripts/ci_build.sh -q -f -n
+    - scripts/ci_build.sh -q -f
 
 full-ci:
   <<: *common-build
@@ -37,7 +37,7 @@ full-ci:
     - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
     - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
           echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
-          scripts/ci_build.sh -q -c -r -n;
+          scripts/ci_build.sh -q -c -r;
       fi
   cache:
     key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index e5f20d4..960ee94 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -127,6 +127,11 @@ fi
 
 if [ -n "$NORUN" ]; then
     TAGS="$TAGS,-startvm"
+else
+    if ! command -v qemu-system-x86_64 > /dev/null; then
+        sudo apt-get update -qq
+        sudo apt-get install -y qemu-system
+    fi
 fi
 
 # Provide working path
-- 
2.34.1

Re: [PATCH v2 07/15] CI: install qemu-system when qemu testing is requested

[Henning Schild]

Am Tue, 24 Jan 2023 22:45:50 +0200
schrieb Anton Mikanovich <amikan@ilbers.de>:

> From: Henning Schild <henning.schild@siemens.com>
> 
> And also change gitlab ci config to request qemu testing.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  .gitlab-ci.yml      | 4 ++--
>  scripts/ci_build.sh | 5 +++++
>  2 files changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fcdefa8..6a49ce4 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -27,7 +27,7 @@ fast-ci:
>    except:
>      - schedules
>    script:
> -    - scripts/ci_build.sh -q -f -n
> +    - scripts/ci_build.sh -q -f
>  
>  full-ci:
>    <<: *common-build
> @@ -37,7 +37,7 @@ full-ci:
>      - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
>      - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
>            echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
> -          scripts/ci_build.sh -q -c -r -n;
> +          scripts/ci_build.sh -q -c -r;
>        fi
>    cache:
>      key: "$CI_COMMIT_REF_SLUG"
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index e5f20d4..960ee94 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -127,6 +127,11 @@ fi
>  
>  if [ -n "$NORUN" ]; then
>      TAGS="$TAGS,-startvm"
> +else
> +    if ! command -v qemu-system-x86_64 > /dev/null; then

I think i would have done

if [ ! -f /usr/share/doc/qemu-system/copyright ] 

after the last review. That way we check what we install and will not
miss the arm qemu should the amd64 be the only one installed.

Henning

> +        sudo apt-get update -qq
> +        sudo apt-get install -y qemu-system
> +    fi
>  fi
>  
>  # Provide working path

[PATCH v2 08/15] testsuite: remove tests from "fast" set

[Anton Mikanovich]

From: Henning Schild <henning.schild@siemens.com>

Ccache and Sstate are test infrastructure bits and should not be done
when we try to be "fast".

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/citest.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 8414ef4..5ed2df4 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -107,7 +107,7 @@ class CcacheTest(CIBaseTest):
     """
     Test rebuild speed improve with ccache
 
-    :avocado: tags=ccache,fast,full
+    :avocado: tags=ccache,full
     """
     def test_ccache_rebuild(self):
         targets = ['mc:qemuamd64-bullseye:hello-isar']
@@ -300,7 +300,7 @@ class SstateTest(CIBaseTest):
     """
     Test builds with artifacts taken from sstate cache
 
-    :avocado: tags=sstate,fast,full
+    :avocado: tags=sstate,full
     """
     def test_sstate(self):
         image_target = 'mc:qemuamd64-bullseye:isar-image-base'
-- 
2.34.1

[PATCH v2 09/15] CI: Get rid of setting cross from outside

[Anton Mikanovich]

There is no need in setting cross compilation mode from outside
because it was already done inside test cases.
This also makes test cases more unified.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .gitlab-ci.yml         | 2 +-
 scripts/ci_build.sh    | 9 +--------
 testsuite/cibuilder.py | 4 +---
 3 files changed, 3 insertions(+), 12 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 6a49ce4..d1d22a6 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -37,7 +37,7 @@ full-ci:
     - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
     - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
           echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
-          scripts/ci_build.sh -q -c -r;
+          scripts/ci_build.sh -q -r;
       fi
   cache:
     key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 960ee94..02aeba5 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -48,7 +48,6 @@ show_help() {
     echo "Parameters:"
     echo "    -b, --base BASE_DIR      set path to base directory. If not set,"
     echo "                             the tests will be started in current path."
-    echo "    -c, --cross              enable cross-compilation."
     echo "    -d, --debug              enable debug bitbake output."
     echo "    -f, --fast               cross build reduced set of configurations."
     echo "    -q, --quiet              suppress verbose bitbake output."
@@ -64,7 +63,6 @@ show_help() {
 }
 
 TAGS="full"
-CROSS_BUILD="0"
 QUIET="0"
 TIMEOUT=300
 
@@ -82,17 +80,12 @@ do
         BASE_DIR="$2"
         shift
         ;;
-    -c|--cross)
-        CROSS_BUILD="1"
-        ;;
     -d|--debug)
         VERBOSE="--show=app,test"
         ;;
     -f|--fast)
         # Start build for the reduced set of configurations
-        # Enforce cross-compilation to speed up the build
         TAGS="fast"
-        CROSS_BUILD="1"
         ;;
     -q|--quiet)
         QUIET="1"
@@ -150,4 +143,4 @@ set -x
 
 avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
     -t "${TAGS}" --nrunner-max-parallel-tasks=1 --disable-sysinfo \
-    -p quiet="${QUIET}" -p cross="${CROSS_BUILD}" -p time_to_wait="${TIMEOUT}"
+    -p quiet="${QUIET}" -p time_to_wait="${TIMEOUT}"
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 4e55121..830918d 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -52,7 +52,7 @@ class CIBuilder(Test):
         if not hasattr(self, 'build_dir'):
             self.error("Broken test implementation: need to call init().")
 
-    def configure(self, compat_arch=True, cross=None, debsrc_cache=False,
+    def configure(self, compat_arch=True, cross=False, debsrc_cache=False,
                   container=False, ccache=False, sstate=False, offline=False,
                   gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
                   source_date_epoch=None, image_install=None, **kwargs):
@@ -62,8 +62,6 @@ class CIBuilder(Test):
 
         # get parameters from avocado cmdline
         quiet = bool(int(self.params.get('quiet', default=0)))
-        if cross is None:
-            cross = bool(int(self.params.get('cross', default=0)))
 
         if dl_dir is None:
             dl_dir = os.path.join(isar_root, 'downloads')
-- 
2.34.1

Re: [PATCH v2 09/15] CI: Get rid of setting cross from outside

[Henning Schild]

Am Tue, 24 Jan 2023 22:45:52 +0200
schrieb Anton Mikanovich <amikan@ilbers.de>:

> There is no need in setting cross compilation mode from outside
> because it was already done inside test cases.
> This also makes test cases more unified.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  .gitlab-ci.yml         | 2 +-
>  scripts/ci_build.sh    | 9 +--------
>  testsuite/cibuilder.py | 4 +---
>  3 files changed, 3 insertions(+), 12 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 6a49ce4..d1d22a6 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -37,7 +37,7 @@ full-ci:
>      - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
>      - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
>            echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
> -          scripts/ci_build.sh -q -c -r;
> +          scripts/ci_build.sh -q -r;
>        fi
>    cache:
>      key: "$CI_COMMIT_REF_SLUG"
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index 960ee94..02aeba5 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -48,7 +48,6 @@ show_help() {
>      echo "Parameters:"
>      echo "    -b, --base BASE_DIR      set path to base directory.
> If not set," echo "                             the tests will be
> started in current path."
> -    echo "    -c, --cross              enable cross-compilation."

This is a user-interface change and people calling it the old way will
get an error.

Not sure we want to do anything about that, i do not care because
gitlab never used that and i hardly ever called that manually.

Henning

>      echo "    -d, --debug              enable debug bitbake output."
>      echo "    -f, --fast               cross build reduced set of
> configurations." echo "    -q, --quiet              suppress verbose
> bitbake output." @@ -64,7 +63,6 @@ show_help() {
>  }
>  
>  TAGS="full"
> -CROSS_BUILD="0"
>  QUIET="0"
>  TIMEOUT=300
>  
> @@ -82,17 +80,12 @@ do
>          BASE_DIR="$2"
>          shift
>          ;;
> -    -c|--cross)
> -        CROSS_BUILD="1"
> -        ;;
>      -d|--debug)
>          VERBOSE="--show=app,test"
>          ;;
>      -f|--fast)
>          # Start build for the reduced set of configurations
> -        # Enforce cross-compilation to speed up the build
>          TAGS="fast"
> -        CROSS_BUILD="1"
>          ;;
>      -q|--quiet)
>          QUIET="1"
> @@ -150,4 +143,4 @@ set -x
>  
>  avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
>      -t "${TAGS}" --nrunner-max-parallel-tasks=1 --disable-sysinfo \
> -    -p quiet="${QUIET}" -p cross="${CROSS_BUILD}" -p
> time_to_wait="${TIMEOUT}"
> +    -p quiet="${QUIET}" -p time_to_wait="${TIMEOUT}"
> diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
> index 4e55121..830918d 100755
> --- a/testsuite/cibuilder.py
> +++ b/testsuite/cibuilder.py
> @@ -52,7 +52,7 @@ class CIBuilder(Test):
>          if not hasattr(self, 'build_dir'):
>              self.error("Broken test implementation: need to call
> init().") 
> -    def configure(self, compat_arch=True, cross=None,
> debsrc_cache=False,
> +    def configure(self, compat_arch=True, cross=False,
> debsrc_cache=False, container=False, ccache=False, sstate=False,
> offline=False, gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
>                    source_date_epoch=None, image_install=None,
> **kwargs): @@ -62,8 +62,6 @@ class CIBuilder(Test):
>  
>          # get parameters from avocado cmdline
>          quiet = bool(int(self.params.get('quiet', default=0)))
> -        if cross is None:
> -            cross = bool(int(self.params.get('cross', default=0)))
>  
>          if dl_dir is None:
>              dl_dir = os.path.join(isar_root, 'downloads')

[PATCH v2 10/15] CI: Make cross compile to be default

[Anton Mikanovich]

Cross compile is stable enough to be enabled by default.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/cibuilder.py |  2 +-
 testsuite/citest.py    | 14 +++++++-------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 830918d..d229ecb 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -52,7 +52,7 @@ class CIBuilder(Test):
         if not hasattr(self, 'build_dir'):
             self.error("Broken test implementation: need to call init().")
 
-    def configure(self, compat_arch=True, cross=False, debsrc_cache=False,
+    def configure(self, compat_arch=True, cross=True, debsrc_cache=False,
                   container=False, ccache=False, sstate=False, offline=False,
                   gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
                   source_date_epoch=None, image_install=None, **kwargs):
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 5ed2df4..2c17361 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -33,7 +33,7 @@ class DevTest(CIBaseTest):
                   ]
 
         self.init()
-        self.perform_build_test(targets, cross=True, image_install="example-raw")
+        self.perform_build_test(targets, image_install="example-raw")
 
     def test_dev_apps(self):
         targets = [
@@ -86,7 +86,7 @@ class ReproTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_repro_test(targets, signed=True, cross=True)
+            self.perform_repro_test(targets, signed=True)
         finally:
             self.move_in_build_dir('tmp', 'tmp_repro_signed')
 
@@ -98,7 +98,7 @@ class ReproTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_repro_test(targets)
+            self.perform_repro_test(targets, cross=False)
         finally:
             self.move_in_build_dir('tmp', 'tmp_repro_unsigned')
 
@@ -130,7 +130,7 @@ class CrossTest(CIBaseTest):
                   ]
 
         self.init()
-        self.perform_build_test(targets, cross=True, debsrc_cache=True)
+        self.perform_build_test(targets, debsrc_cache=True)
 
     def test_cross_rpi(self):
         targets = [
@@ -139,7 +139,7 @@ class CrossTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_build_test(targets, cross=True, debsrc_cache=True)
+            self.perform_build_test(targets, debsrc_cache=True)
         except:
             self.cancel('KFAIL')
 
@@ -150,7 +150,7 @@ class CrossTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_build_test(targets, cross=True)
+            self.perform_build_test(targets)
         except:
             self.cancel('KFAIL')
 
@@ -162,7 +162,7 @@ class CrossTest(CIBaseTest):
 
         self.init()
         try:
-            self.perform_build_test(targets, cross=True)
+            self.perform_build_test(targets)
         except:
             self.cancel('KFAIL')
 
-- 
2.34.1

[PATCH v2 11/15] CI: Get rid of setting repro from outside

[Anton Mikanovich]

To make test cases more unified do not allow to change repro test
execution. This test case will always run for "full" now.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .gitlab-ci.yml      |  2 +-
 scripts/ci_build.sh | 13 -------------
 2 files changed, 1 insertion(+), 14 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d1d22a6..d1e8249 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -37,7 +37,7 @@ full-ci:
     - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
     - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
           echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
-          scripts/ci_build.sh -q -r;
+          scripts/ci_build.sh -q;
       fi
   cache:
     key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 02aeba5..dbac96f 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -51,7 +51,6 @@ show_help() {
     echo "    -d, --debug              enable debug bitbake output."
     echo "    -f, --fast               cross build reduced set of configurations."
     echo "    -q, --quiet              suppress verbose bitbake output."
-    echo "    -r, --repro              enable use of cached base repository."
     echo "    -n, --norun              do not execute QEMU run tests."
     echo "    -t, --timeout SEC        specify time in seconds to wait before stop QEMU."
     echo "                             The default is: 300"
@@ -90,14 +89,6 @@ do
     -q|--quiet)
         QUIET="1"
         ;;
-    -r|--repro)
-        REPRO_BUILD="1"
-        # This switch is deprecated, just here to not cause failing CI on
-        # legacy configs
-        case "$2" in
-        -s|--sign) shift ;;
-        esac
-        ;;
     -n|--norun)
         NORUN="1"
         ;;
@@ -114,10 +105,6 @@ do
     shift
 done
 
-if [ -z "$REPRO_BUILD" ]; then
-    TAGS="$TAGS,-repro"
-fi
-
 if [ -n "$NORUN" ]; then
     TAGS="$TAGS,-startvm"
 else
-- 
2.34.1

[PATCH v2 12/15] CI: Allow example module checking disabling

[Anton Mikanovich]

If the target do not have example-module package compiled it can fail
on startvm test currently. Provide internal testsuite API for skipping
this test.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/cibuilder.py | 6 ++++--
 testsuite/citest.py    | 2 +-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index d229ecb..f373829 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -209,7 +209,8 @@ class CIBuilder(Test):
 
         return env['LAYERDIR_' + layer].strip('"')
 
-    def vm_start(self, arch='amd64', distro='buster', enforce_pcbios=False):
+    def vm_start(self, arch='amd64', distro='buster', enforce_pcbios=False,
+                 skip_modulecheck=False):
         time_to_wait = self.params.get('time_to_wait', default=60)
 
         self.log.info('===================================================')
@@ -291,7 +292,8 @@ class CIBuilder(Test):
         if os.path.exists(output_file) and os.path.getsize(output_file) > 0:
             with open(output_file, "rb") as f1:
                 data = f1.read()
-                if module_output in data and login_prompt in data:
+                if (module_output in data or skip_modulecheck) \
+                   and login_prompt in data:
                     if resize_output:
                         if resize_output in data:
                             return
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 2c17361..1b4336c 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -69,7 +69,7 @@ class DevTest(CIBaseTest):
 
     def test_dev_run_arm_bullseye(self):
         self.init()
-        self.vm_start('arm', 'bullseye')
+        self.vm_start('arm', 'bullseye', skip_modulecheck=True)
 
 class ReproTest(CIBaseTest):
 
-- 
2.34.1

Re: [PATCH v2 12/15] CI: Allow example module checking disabling

[Henning Schild]

Am Tue, 24 Jan 2023 22:45:55 +0200
schrieb Anton Mikanovich <amikan@ilbers.de>:

> If the target do not have example-module package compiled it can fail
> on startvm test currently. Provide internal testsuite API for skipping
> this test.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  testsuite/cibuilder.py | 6 ++++--
>  testsuite/citest.py    | 2 +-
>  2 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
> index d229ecb..f373829 100755
> --- a/testsuite/cibuilder.py
> +++ b/testsuite/cibuilder.py
> @@ -209,7 +209,8 @@ class CIBuilder(Test):
>  
>          return env['LAYERDIR_' + layer].strip('"')
>  
> -    def vm_start(self, arch='amd64', distro='buster',
> enforce_pcbios=False):
> +    def vm_start(self, arch='amd64', distro='buster',
> enforce_pcbios=False,
> +                 skip_modulecheck=False):
>          time_to_wait = self.params.get('time_to_wait', default=60)
>  
>          self.log.info('===================================================')
> @@ -291,7 +292,8 @@ class CIBuilder(Test):
>          if os.path.exists(output_file) and
> os.path.getsize(output_file) > 0: with open(output_file, "rb") as f1:
>                  data = f1.read()
> -                if module_output in data and login_prompt in data:
> +                if (module_output in data or skip_modulecheck) \
> +                   and login_prompt in data:

This all looks a little weird to be honest. And will likely be replaced
once we ssh in and "lsmod" anyhow. Maybe we need this in the meantime.

>                      if resize_output:
>                          if resize_output in data:
>                              return
> diff --git a/testsuite/citest.py b/testsuite/citest.py
> index 2c17361..1b4336c 100755
> --- a/testsuite/citest.py
> +++ b/testsuite/citest.py
> @@ -69,7 +69,7 @@ class DevTest(CIBaseTest):
>  
>      def test_dev_run_arm_bullseye(self):
>          self.init()
> -        self.vm_start('arm', 'bullseye')
> +        self.vm_start('arm', 'bullseye', skip_modulecheck=True)

Not sure why this one does not have or support the module, maybe adding
the module if possible would be better.

Henning
 
>  class ReproTest(CIBaseTest):
>  

[PATCH v2 13/15] testsuite: Fix terminal broken after run test executed

[Anton Mikanovich]

From: Uladzimir Bely <ubely@ilbers.de>

If some of run tests executed from interactive terminal by hand,
user input goes broken after VM powered off. Commands user enters
become invisible and terminal can be fixed only by executing
either 'reset' or 'stty echo' commands.

Fix this by specifying 'stdin' value for qemu subprocess.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/cibuilder.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index f373829..ec6ba39 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -262,7 +262,7 @@ class CIBuilder(Test):
 
         timeout = time.time() + int(time_to_wait)
 
-        p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE,
+        p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE, stdin=subprocess.PIPE,
                               stderr=subprocess.PIPE, universal_newlines=True)
         try:
             poller = select.poll()
-- 
2.34.1

[PATCH v2 14/15] CI: Update testsuite documentation

[Anton Mikanovich]

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/README.md | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/testsuite/README.md b/testsuite/README.md
index b83a8f9..90ba5c3 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -4,34 +4,42 @@ The framework could be installed by using standard HOWTO:
 
   https://github.com/avocado-framework/avocado#installing-with-standard-python-tools
 
-## For Debian (tested on Debian 10.x)
+## For Debian (tested on Debian 11.x)
 
-        $ sudo dpkg -i avocado_91.0_all.deb
+```
+$ pip install avocado-framework==99.0
+```
 
 # Run test
 
+## Quick developers test
+
+```
+$ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1
+```
+
 ## Fast build test
 
 ```
-$ avocado run build_test.py -t fast -p quiet=1 -p cross=1
+$ avocado run ../testsuite/citest.py -t fast --nrunner-max-parallel-tasks=1 -p quiet=1
 ```
 
 ## Full build test
 
 ```
-$ avocado run build_test.py -t full -p quiet=1
+$ avocado run ../testsuite/citest.py -t full --nrunner-max-parallel-tasks=1 -p quiet=1
 ```
 
 ## Fast boot test
 
 ```
-$ avocado run vm_boot_test.py -t fast -p build_dir="$BUILDDIR" -p time_to_wait=300
+$ avocado run ../testsuite/citest.py -t startvm,fast -p time_to_wait=300
 ```
 
 ## Full boot test
 
 ```
-$ avocado run vm_boot_test.py -t full -p build_dir="$BUILDDIR" -p time_to_wait=300
+$ avocado run ../testsuite/citest.py -t startvm,full -p time_to_wait=300
 ```
 
 # Other
-- 
2.34.1

Re: [PATCH v2 14/15] CI: Update testsuite documentation

[Henning Schild]

Am Tue, 24 Jan 2023 22:45:57 +0200
schrieb Anton Mikanovich <amikan@ilbers.de>:

> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  testsuite/README.md | 20 ++++++++++++++------
>  1 file changed, 14 insertions(+), 6 deletions(-)
> 
> diff --git a/testsuite/README.md b/testsuite/README.md
> index b83a8f9..90ba5c3 100644
> --- a/testsuite/README.md
> +++ b/testsuite/README.md
> @@ -4,34 +4,42 @@ The framework could be installed by using standard
> HOWTO: 
>    https://github.com/avocado-framework/avocado#installing-with-standard-python-tools
>  
> -## For Debian (tested on Debian 10.x)
> +## For Debian (tested on Debian 11.x)
>  
> -        $ sudo dpkg -i avocado_91.0_all.deb
> +```
> +$ pip install avocado-framework==99.0
> +```

maybe suggest venv, pip is evil and if people do that on productive
machines they might get into real trouble at some point

Henning

>  
>  # Run test
>  
> +## Quick developers test
> +
> +```
> +$ avocado run ../testsuite/citest.py -t dev
> --nrunner-max-parallel-tasks=1 +```
> +
>  ## Fast build test
>  
>  ```
> -$ avocado run build_test.py -t fast -p quiet=1 -p cross=1
> +$ avocado run ../testsuite/citest.py -t fast
> --nrunner-max-parallel-tasks=1 -p quiet=1 ```
>  
>  ## Full build test
>  
>  ```
> -$ avocado run build_test.py -t full -p quiet=1
> +$ avocado run ../testsuite/citest.py -t full
> --nrunner-max-parallel-tasks=1 -p quiet=1 ```
>  
>  ## Fast boot test
>  
>  ```
> -$ avocado run vm_boot_test.py -t fast -p build_dir="$BUILDDIR" -p
> time_to_wait=300 +$ avocado run ../testsuite/citest.py -t
> startvm,fast -p time_to_wait=300 ```
>  
>  ## Full boot test
>  
>  ```
> -$ avocado run vm_boot_test.py -t full -p build_dir="$BUILDDIR" -p
> time_to_wait=300 +$ avocado run ../testsuite/citest.py -t
> startvm,full -p time_to_wait=300 ```
>  
>  # Other

[PATCH v2 15/15] CI: Add single target test case

[Anton Mikanovich]

Add special test case for single target build and run testing.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 testsuite/README.md |  6 ++++++
 testsuite/citest.py | 22 ++++++++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/testsuite/README.md b/testsuite/README.md
index 90ba5c3..4f2a8da 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -18,6 +18,12 @@ $ pip install avocado-framework==99.0
 $ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1
 ```
 
+## Single target test
+
+```
+$ avocado run ../testsuite/citest.py -t single --nrunner-max-parallel-tasks=1 -p machine=qemuamd64 -p distro=bullseye
+```
+
 ## Fast build test
 
 ```
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 1b4336c..c858449 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -309,6 +309,28 @@ class SstateTest(CIBaseTest):
         self.init('build-sstate')
         self.perform_sstate_test(image_target, package_target)
 
+class SingleTest(CIBaseTest):
+
+    """
+    Single test for selected target
+
+    :avocado: tags=single
+    """
+    def test_single_build(self):
+        self.init()
+        machine = self.params.get('machine', default='qemuamd64')
+        distro = self.params.get('distro', default='bullseye')
+        image = self.params.get('image', default='isar-image-base')
+
+        self.perform_build_test('mc:%s-%s:%s' % (machine, distro, image))
+
+    def test_single_run(self):
+        self.init()
+        machine = self.params.get('machine', default='qemuamd64')
+        distro = self.params.get('distro', default='bullseye')
+
+        self.vm_start(machine.removeprefix('qemu'), distro)
+
 class VmBootTestFast(CIBaseTest):
 
     """
-- 
2.34.1