* [PATCH 1/4] CI: support negative build tests
2025-11-01 11:13 [PATCH 0/4] Add test-cases for dracut-based initrds 'Cedric Hombourger' via isar-users
@ 2025-11-01 11:13 ` 'Cedric Hombourger' via isar-users
2025-11-01 11:13 ` [PATCH 2/4] CI: add smoke-tests for switch to dracut in isar-image-ci 'Cedric Hombourger' via isar-users
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: 'Cedric Hombourger' via isar-users @ 2025-11-01 11:13 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
It appears that our build tests are only check for successful
builds. We also want to make sure builds fail when they are
expected to. This adds a should_fail boolean to bitbake() and
perform_build_test()
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
testsuite/cibase.py | 4 ++--
testsuite/cibuilder.py | 12 ++++++++----
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index 6ffedcea..0a006a21 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -14,12 +14,12 @@ from avocado.utils import process
class CIBaseTest(CIBuilder):
- def perform_build_test(self, targets, **kwargs):
+ def perform_build_test(self, targets, should_fail=False, **kwargs):
self.configure(**kwargs)
self.log.info("Starting build...")
- self.bitbake(targets, **kwargs)
+ self.bitbake(targets, should_fail=should_fail, **kwargs)
def perform_wic_partition_test(self, targets, wic_deploy_parts, **kwargs):
self.configure(wic_deploy_parts=wic_deploy_parts, **kwargs)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index cd5e70ba..366f6a1f 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -281,7 +281,8 @@ class CIBuilder(Test):
if os.path.exists(self.build_dir + '/' + src):
shutil.move(self.build_dir + '/' + src, self.build_dir + '/' + dst)
- def bitbake(self, target, bitbake_cmd=None, sig_handler=None, **kwargs):
+ def bitbake(self, target, bitbake_cmd=None, should_fail=False,
+ sig_handler=None, **kwargs):
self.check_init()
self.log.info("===================================================")
self.log.info(f"Building {str(target)}")
@@ -318,13 +319,16 @@ class CIBuilder(Test):
continue
if fd == p1.stdout.fileno():
self.log.info(p1.stdout.readline().rstrip())
- if fd == p1.stderr.fileno():
+ if fd == p1.stderr.fileno() and should_fail is False:
app_log.error(p1.stderr.readline().rstrip())
if p1.poll() is not None:
break
p1.wait()
- if p1.returncode:
- self.fail("Bitbake failed")
+ if should_fail is False:
+ if p1.returncode:
+ self.fail("Bitbake failed")
+ elif p1.returncode == 0:
+ self.fail("Bitbake suceeded but was expected to fail!")
def backupfile(self, path):
self.check_init()
--
2.47.3
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251101111326.600581-2-cedric.hombourger%40siemens.com.
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 2/4] CI: add smoke-tests for switch to dracut in isar-image-ci
2025-11-01 11:13 [PATCH 0/4] Add test-cases for dracut-based initrds 'Cedric Hombourger' via isar-users
2025-11-01 11:13 ` [PATCH 1/4] CI: support negative build tests 'Cedric Hombourger' via isar-users
@ 2025-11-01 11:13 ` 'Cedric Hombourger' via isar-users
2025-11-07 13:09 ` Zhihang Wei
2025-11-01 11:13 ` [PATCH 3/4] CI: add build tests for isar-dracut initrd images 'Cedric Hombourger' via isar-users
2025-11-01 11:13 ` [PATCH 4/4] CI: check if the build fails when dracut fails 'Cedric Hombourger' via isar-users
3 siblings, 1 reply; 7+ messages in thread
From: 'Cedric Hombourger' via isar-users @ 2025-11-01 11:13 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
Use IMAGE_PREINSTALL to install dracut in the isar-image-ci
image and verify that it builds and boots. We also check that
a dracut-shutdown service is active (as an indication that
the system was booted using a dracut initrd). A single test
is provided as well as cross-{machine,distro} tests.
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
testsuite/citest.py | 58 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 58 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index a1214e9c..23900a6c 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -278,6 +278,64 @@ class KernelTests(CIBaseTest):
self.perform_build_test(targets, image_install=' '.join(modules), lines=lines)
+class InitRdBaseTest(CIBaseTest):
+ """
+ Helper class for InitRd build tests
+ """
+
+ DRACUT_CONF = [
+ "IMAGE_PREINSTALL:append = ' dracut'",
+ "IMAGE_INSTALL:remove = 'enable-fsck'",
+ ]
+
+ def dracut_in_image(self, targets):
+ machine = 'qemuamd64'
+ distro = 'bookworm'
+ image = 'isar-image-ci'
+ self.init()
+ self.perform_build_test(targets, lines=InitRdBaseTest.DRACUT_CONF)
+
+ for target in targets:
+ config = target.split(':')
+ machine = config[1].split('-')[0]
+ distro = config[1].split('-')[1]
+ image = config[2]
+
+ # systems booted via a dracut initrd have a dracut-shutdown service
+ # check its presence
+ self.vm_start(
+ machine.removeprefix('qemu'), distro, image=image,
+ cmd="systemctl is-active dracut-shutdown"
+ )
+
+
+class InitRdTest(InitRdBaseTest):
+ """
+ Tests for initrd generation
+ :avocado: tags=initrd,single,full
+ """
+
+ def test_dracut_in_image(self):
+ """Test switch to dracut in an image recipe."""
+ self.dracut_in_image(['mc:qemuamd64-bookworm:isar-image-ci'])
+
+
+class InitRdCrossTests(InitRdBaseTest):
+ """
+ Tests for initrd generation
+ :avocado: tags=initrd,full
+ """
+
+ def test_dracut_in_image(self):
+ """Test switch to dracut in an image recipe."""
+ self.dracut_in_image([
+ 'mc:qemuamd64-bookworm:isar-image-ci',
+ 'mc:qemuamd64-trixie:isar-image-ci',
+ 'mc:qemuarm64-bookworm:isar-image-ci',
+ 'mc:qemuarm64-trixie:isar-image-ci'
+ ])
+
+
class WicTest(CIBaseTest):
"""
--
2.47.3
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251101111326.600581-3-cedric.hombourger%40siemens.com.
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH 2/4] CI: add smoke-tests for switch to dracut in isar-image-ci
2025-11-01 11:13 ` [PATCH 2/4] CI: add smoke-tests for switch to dracut in isar-image-ci 'Cedric Hombourger' via isar-users
@ 2025-11-07 13:09 ` Zhihang Wei
2025-11-07 13:42 ` 'cedric.hombourger@siemens.com' via isar-users
0 siblings, 1 reply; 7+ messages in thread
From: Zhihang Wei @ 2025-11-07 13:09 UTC (permalink / raw)
To: Cedric Hombourger, isar-users
On 11/1/25 12:13, 'Cedric Hombourger' via isar-users wrote:
> Use IMAGE_PREINSTALL to install dracut in the isar-image-ci
> image and verify that it builds and boots. We also check that
> a dracut-shutdown service is active (as an indication that
> the system was booted using a dracut initrd). A single test
> is provided as well as cross-{machine,distro} tests.
>
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
> testsuite/citest.py | 58 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 58 insertions(+)
>
> diff --git a/testsuite/citest.py b/testsuite/citest.py
> index a1214e9c..23900a6c 100755
> --- a/testsuite/citest.py
> +++ b/testsuite/citest.py
> @@ -278,6 +278,64 @@ class KernelTests(CIBaseTest):
> self.perform_build_test(targets, image_install=' '.join(modules), lines=lines)
>
>
> +class InitRdBaseTest(CIBaseTest):
> + """
> + Helper class for InitRd build tests
> + """
> +
> + DRACUT_CONF = [
> + "IMAGE_PREINSTALL:append = ' dracut'",
> + "IMAGE_INSTALL:remove = 'enable-fsck'",
> + ]
> +
> + def dracut_in_image(self, targets):
> + machine = 'qemuamd64'
> + distro = 'bookworm'
> + image = 'isar-image-ci'
> + self.init()
> + self.perform_build_test(targets, lines=InitRdBaseTest.DRACUT_CONF)
> +
> + for target in targets:
> + config = target.split(':')
> + machine = config[1].split('-')[0]
> + distro = config[1].split('-')[1]
> + image = config[2]
> +
> + # systems booted via a dracut initrd have a dracut-shutdown service
> + # check its presence
> + self.vm_start(
> + machine.removeprefix('qemu'), distro, image=image,
> + cmd="systemctl is-active dracut-shutdown"
> + )
> +
> +
I would suggest to move the definition of class InitRdBaseTest(CIBaseTest)
to cibase.py. In citest.py there should only be definition of tests that we
can directly run.
> +class InitRdTest(InitRdBaseTest):
> + """
> + Tests for initrd generation
> + :avocado: tags=initrd,single,full
> + """
> +
> + def test_dracut_in_image(self):
> + """Test switch to dracut in an image recipe."""
> + self.dracut_in_image(['mc:qemuamd64-bookworm:isar-image-ci'])
> +
> +
> +class InitRdCrossTests(InitRdBaseTest):
> + """
> + Tests for initrd generation
> + :avocado: tags=initrd,full
> + """
> +
> + def test_dracut_in_image(self):
> + """Test switch to dracut in an image recipe."""
> + self.dracut_in_image([
> + 'mc:qemuamd64-bookworm:isar-image-ci',
> + 'mc:qemuamd64-trixie:isar-image-ci',
> + 'mc:qemuarm64-bookworm:isar-image-ci',
> + 'mc:qemuarm64-trixie:isar-image-ci'
> + ])
> +
> +
> class WicTest(CIBaseTest):
>
> """
Thanks for adding new test cases. I have tested the v7 dracut patch set
against these test cases and others in current CI, didn't show any error.
We'll continue with this patch set after dracut can be applied.
Best regards,
Zhihang
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/bda24ac5-21de-4881-bc64-704a58147f6b%40ilbers.de.
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH 2/4] CI: add smoke-tests for switch to dracut in isar-image-ci
2025-11-07 13:09 ` Zhihang Wei
@ 2025-11-07 13:42 ` 'cedric.hombourger@siemens.com' via isar-users
0 siblings, 0 replies; 7+ messages in thread
From: 'cedric.hombourger@siemens.com' via isar-users @ 2025-11-07 13:42 UTC (permalink / raw)
To: wzh; +Cc: isar-users
On Fri, 2025-11-07 at 14:09 +0100, Zhihang Wei wrote:
>
>
> On 11/1/25 12:13, 'Cedric Hombourger' via isar-users wrote:
> > Use IMAGE_PREINSTALL to install dracut in the isar-image-ci
> > image and verify that it builds and boots. We also check that
> > a dracut-shutdown service is active (as an indication that
> > the system was booted using a dracut initrd). A single test
> > is provided as well as cross-{machine,distro} tests.
> >
> > Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> > ---
> > testsuite/citest.py | 58
> > +++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 58 insertions(+)
> >
> > diff --git a/testsuite/citest.py b/testsuite/citest.py
> > index a1214e9c..23900a6c 100755
> > --- a/testsuite/citest.py
> > +++ b/testsuite/citest.py
> > @@ -278,6 +278,64 @@ class KernelTests(CIBaseTest):
> > self.perform_build_test(targets, image_install='
> > '.join(modules), lines=lines)
> >
> >
> > +class InitRdBaseTest(CIBaseTest):
> > + """
> > + Helper class for InitRd build tests
> > + """
> > +
> > + DRACUT_CONF = [
> > + "IMAGE_PREINSTALL:append = ' dracut'",
> > + "IMAGE_INSTALL:remove = 'enable-fsck'",
> > + ]
> > +
> > + def dracut_in_image(self, targets):
> > + machine = 'qemuamd64'
> > + distro = 'bookworm'
> > + image = 'isar-image-ci'
> > + self.init()
> > + self.perform_build_test(targets,
> > lines=InitRdBaseTest.DRACUT_CONF)
> > +
> > + for target in targets:
> > + config = target.split(':')
> > + machine = config[1].split('-')[0]
> > + distro = config[1].split('-')[1]
> > + image = config[2]
> > +
> > + # systems booted via a dracut initrd have a dracut-
> > shutdown service
> > + # check its presence
> > + self.vm_start(
> > + machine.removeprefix('qemu'), distro, image=image,
> > + cmd="systemctl is-active dracut-shutdown"
> > + )
> > +
> > +
> I would suggest to move the definition of class
> InitRdBaseTest(CIBaseTest)
> to cibase.py. In citest.py there should only be definition of tests
> that we
> can directly run.
Are we sure we want to do this? While that's an easy change, I fear
that it would negatively impact readability of the tests that are
defined below. cibase is really at the framework level. The above class
is getting into the specifics.
We will anyhow need to revisit the structure of our test. If we are
successful in getting more test-case contributions (as I am really
hoping and now pushing for), then we would end-up with an enormous
citest.py file. We will need to break it down.
I would therefore like to give us some time to re-think the overall
structure of our testsuite/.
Is that agreeable?
> > +class InitRdTest(InitRdBaseTest):
> > + """
> > + Tests for initrd generation
> > + :avocado: tags=initrd,single,full
> > + """
> > +
> > + def test_dracut_in_image(self):
> > + """Test switch to dracut in an image recipe."""
> > + self.dracut_in_image(['mc:qemuamd64-bookworm:isar-image-
> > ci'])
> > +
> > +
> > +class InitRdCrossTests(InitRdBaseTest):
> > + """
> > + Tests for initrd generation
> > + :avocado: tags=initrd,full
> > + """
> > +
> > + def test_dracut_in_image(self):
> > + """Test switch to dracut in an image recipe."""
> > + self.dracut_in_image([
> > + 'mc:qemuamd64-bookworm:isar-image-ci',
> > + 'mc:qemuamd64-trixie:isar-image-ci',
> > + 'mc:qemuarm64-bookworm:isar-image-ci',
> > + 'mc:qemuarm64-trixie:isar-image-ci'
> > + ])
> > +
> > +
> > class WicTest(CIBaseTest):
> >
> > """
> Thanks for adding new test cases. I have tested the v7 dracut patch
> set
> against these test cases and others in current CI, didn't show any
> error.
> We'll continue with this patch set after dracut can be applied.
>
> Best regards,
> Zhihang
--
Cedric Hombourger
Siemens AG
www.siemens.com
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/a25db0cef1c44cf29a1da221316ae251be8a034d.camel%40siemens.com.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/4] CI: add build tests for isar-dracut initrd images
2025-11-01 11:13 [PATCH 0/4] Add test-cases for dracut-based initrds 'Cedric Hombourger' via isar-users
2025-11-01 11:13 ` [PATCH 1/4] CI: support negative build tests 'Cedric Hombourger' via isar-users
2025-11-01 11:13 ` [PATCH 2/4] CI: add smoke-tests for switch to dracut in isar-image-ci 'Cedric Hombourger' via isar-users
@ 2025-11-01 11:13 ` 'Cedric Hombourger' via isar-users
2025-11-01 11:13 ` [PATCH 4/4] CI: check if the build fails when dracut fails 'Cedric Hombourger' via isar-users
3 siblings, 0 replies; 7+ messages in thread
From: 'Cedric Hombourger' via isar-users @ 2025-11-01 11:13 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
testsuite/citest.py | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 23900a6c..7f046a7f 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -319,6 +319,11 @@ class InitRdTest(InitRdBaseTest):
"""Test switch to dracut in an image recipe."""
self.dracut_in_image(['mc:qemuamd64-bookworm:isar-image-ci'])
+ def test_dracut_build_initrd(self):
+ """ Test build of an initrd image that uses dracut."""
+ self.init()
+ self.perform_build_test(['mc:qemuamd64-bookworm:isar-dracut'])
+
class InitRdCrossTests(InitRdBaseTest):
"""
@@ -335,6 +340,16 @@ class InitRdCrossTests(InitRdBaseTest):
'mc:qemuarm64-trixie:isar-image-ci'
])
+ def test_dracut_build_initrd(self):
+ """ Test build of initrd images that use dracut."""
+ self.init()
+ self.perform_build_test([
+ 'mc:qemuamd64-bookworm:isar-dracut',
+ 'mc:qemuamd64-trixie:isar-dracut',
+ 'mc:qemuarm64-bookworm:isar-dracut',
+ 'mc:qemuarm64-trixie:isar-dracut'
+ ])
+
class WicTest(CIBaseTest):
--
2.47.3
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251101111326.600581-4-cedric.hombourger%40siemens.com.
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 4/4] CI: check if the build fails when dracut fails
2025-11-01 11:13 [PATCH 0/4] Add test-cases for dracut-based initrds 'Cedric Hombourger' via isar-users
` (2 preceding siblings ...)
2025-11-01 11:13 ` [PATCH 3/4] CI: add build tests for isar-dracut initrd images 'Cedric Hombourger' via isar-users
@ 2025-11-01 11:13 ` 'Cedric Hombourger' via isar-users
3 siblings, 0 replies; 7+ messages in thread
From: 'Cedric Hombourger' via isar-users @ 2025-11-01 11:13 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
The build should not succeed if dracut fails to generate an
initrd. Amend the dracut command line to add an invalid option
(--unknown-option) to force a build failure and check that
bitbake does fail (it did not in the "Add support for dracut"
patch series v5).
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
testsuite/citest.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 7f046a7f..e53dfa9b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -324,6 +324,14 @@ class InitRdTest(InitRdBaseTest):
self.init()
self.perform_build_test(['mc:qemuamd64-bookworm:isar-dracut'])
+ def test_dracut_build_failure(self):
+ """ Check if the build fails if dracut fails to generate an initrd."""
+ lines = InitRdBaseTest.DRACUT_CONF
+ lines.append("ROOTFS_INITRAMFS_GENERATOR_CMDLINE:append = ' --unknown-option'")
+ self.init()
+ self.perform_build_test('mc:qemuamd64-bookworm:isar-image-ci',
+ should_fail=True, lines=lines)
+
class InitRdCrossTests(InitRdBaseTest):
"""
--
2.47.3
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20251101111326.600581-5-cedric.hombourger%40siemens.com.
^ permalink raw reply [flat|nested] 7+ messages in thread