[PATCH 1/3] testsuite: fix SignatureTest by avoiding absolute path in bblayers

[PATCH 1/3] testsuite: fix SignatureTest by avoiding absolute path in bblayers

['Felix Moessbauer' via isar-users]

The SignatureTest currently fails, as it correctly detects absolute path
that influence the sstate signatures. These path come from the ISARROOT
substitution done in the isar-init-build-env script. Depending on how
the bblayers is setup (which again depends on who does it), we might end
up with absolute paths.

To fix this, we apply the same strategy as yocto and kas uses: Making
the layer imports relative to the TOPDIR. As we can only guarantee the
correctness of this path within the testsuite, we also only apply the
fix there.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta-test/conf/bblayers.conf.sample | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta-test/conf/bblayers.conf.sample b/meta-test/conf/bblayers.conf.sample
index dcec6cf6..f37625d2 100644
--- a/meta-test/conf/bblayers.conf.sample
+++ b/meta-test/conf/bblayers.conf.sample
@@ -9,10 +9,10 @@ BBPATH = "${TOPDIR}"
 BBFILES ?= ""
 
 BBLAYERS ?= " \
-  ##ISARROOT##/meta \
-  ##ISARROOT##/meta-isar \
-  ##ISARROOT##/meta-test \
+  ${TOPDIR}/../meta \
+  ${TOPDIR}/../meta-isar \
+  ${TOPDIR}/../meta-test \
   "
 BBLAYERS_NON_REMOVABLE ?= " \
-  ##ISARROOT##/meta \
+  ${TOPDIR}/../meta \
   "
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251218102343.1074824-1-felix.moessbauer%40siemens.com.

[PATCH 2/3] testsuite: use more recent distros in SignatureTest

['Felix Moessbauer' via isar-users]

In general, that should not make a difference. However, we better test
things that users actually use than old-old-stable distros. As this test
is just a parser test, it is fast. Hence add the fast tag.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/citest.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 95ce891c..cc2bdf41 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -700,16 +700,16 @@ class SignatureTest(CIBaseTest):
     SstateTest also checks for these, but this test is faster and will check
     more cases.
 
-    :avocado: tags=signatures,sstate
+    :avocado: tags=signatures,sstate,fast
     """
 
     def test_signature_lint(self):
         verbose = bool(int(self.params.get('verbose', default=0)))
         targets = [
-            'mc:qemuamd64-bullseye:isar-image-ci',
-            'mc:qemuarm-bullseye:isar-image-base',
-            'mc:qemuarm-bullseye:isar-image-base:do_populate_sdk',
-            'mc:qemuamd64-focal:isar-image-base',
+            'mc:qemuamd64-trixie:isar-image-ci',
+            'mc:qemuarm-trixie:isar-image-base',
+            'mc:qemuarm-trixie:isar-image-base:do_populate_sdk',
+            'mc:qemuamd64-noble:isar-image-base',
         ]
 
         self.init()
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251218102343.1074824-2-felix.moessbauer%40siemens.com.

Re: [PATCH 2/3] testsuite: use more recent distros in SignatureTest

[Anton Mikanovich]

Hello Felix,

18/12/2025 12:23, 'Felix Moessbauer' via isar-users wrote:
> In general, that should not make a difference. However, we better test
> things that users actually use than old-old-stable distros. As this test
> is just a parser test, it is fast. Hence add the fast tag.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>   testsuite/citest.py | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/testsuite/citest.py b/testsuite/citest.py
> index 95ce891c..cc2bdf41 100755
> --- a/testsuite/citest.py
> +++ b/testsuite/citest.py
> @@ -700,16 +700,16 @@ class SignatureTest(CIBaseTest):
>       SstateTest also checks for these, but this test is faster and will check
>       more cases.
>   
> -    :avocado: tags=signatures,sstate
> +    :avocado: tags=signatures,sstate,fast
Adding this test case to fast makes it to cleanup isar-image-ci images which
are used later on startvm tests. You can try to run fast CI to reproduce 
this.

To prevent issues like that in future, we should perform startvm tests 
in the
same places were artifacts got generated, like I've tried to do in:

[18/23] CI: Execute startvm testcases after building
>       """
>   
>       def test_signature_lint(self):
>           verbose = bool(int(self.params.get('verbose', default=0)))
>           targets = [
> -            'mc:qemuamd64-bullseye:isar-image-ci',
> -            'mc:qemuarm-bullseye:isar-image-base',
> -            'mc:qemuarm-bullseye:isar-image-base:do_populate_sdk',
> -            'mc:qemuamd64-focal:isar-image-base',
> +            'mc:qemuamd64-trixie:isar-image-ci',
> +            'mc:qemuarm-trixie:isar-image-base',
> +            'mc:qemuarm-trixie:isar-image-base:do_populate_sdk',
> +            'mc:qemuamd64-noble:isar-image-base',
>           ]
>   
>           self.init()

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/5c49b11a-1d63-42cc-8fb3-3555e923912f%40ilbers.de.

[PATCH 3/3] testsuite: make SignatureTest idempotent

['Felix Moessbauer' via isar-users]

The SignatureTest copies the tmp dir for later comparison. However, this
currently fails in case the test was run before as the target dir then
already exists. We fix it by clearing the target dir upfront. We further
reduce the disk consumption by removing the tmp dir copy in case the
test was successfull. On error, we keep the copy for manual
analysis.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/cibase.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index 5b0139de..5ef1a5b5 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -171,6 +171,7 @@ class CIBaseTest(CIBuilder):
         Generate signature data for target(s) and check for cacheability issues
         """
         self.configure(**kwargs)
+        self.delete_from_build_dir('tmp_before_sstate')
         self.move_in_build_dir('tmp', 'tmp_before_sstate')
         self.bitbake(targets, sig_handler='none')
 
@@ -190,6 +191,9 @@ class CIBaseTest(CIBuilder):
             for line in output.splitlines():
                 self.log.error(ansi_escape.sub('', line))
             self.fail("Detected cacheability issues")
+        else:
+            # on success, cleanup temporary copy (keep on failure to inspect)
+            self.delete_from_build_dir('tmp_before_sstate')
 
     def perform_sstate_test(self, image_target, package_target, **kwargs):
         def check_executed_tasks(target, expected):
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251218102343.1074824-3-felix.moessbauer%40siemens.com.