Re: [PATCH 0/6] Various improvements to the testsuite

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Fri, 2025-12-19 at 13:14 +0200, Anton Mikanovich wrote:
> 19/12/2025 13:06, MOESSBAUER, Felix wrote:
> > Hi,
> > 
> > I plan to send out a v2 of this series by end of today. This will
> > contain all "testsuite:" and "ci:" patches recently sent to the ML and
> > should make applying easier. There are also a couple of fixes in the
> > more recent parts (not this one), which will be included and marked as
> > well.
> > 
> > @Anton: Did you already test parts of the series and already have
> > feedback if it is conceptually ok and also passes your CI?
> > 
> > Best regards,
> > Felix
> Hello Felix,
> 
> Yes, I've tried this patchset on CI and it fails on full CI.
> 
> Following test cases failed:
> test_amd64_bookworm_prebuilt_containers
> test_arm64_bookworm_prebuilt_containers
> 
> It happens because ssh connection got "identification changed" warning
> (sshd-regen-keys issue?) and started waiting for "ssh-keygen -f" command.

The issues seems to be a test-ordering issue. Prior to the "testsuite:
make prebuilt container a feature test", the containers (and also sshd-
regen-keys) package have always been added to the image. However, now
some tests only install specific packages, breaking the otherwise
unrelated VM boot tests later on.

The correct way to fix this is to always build the needed image prior
to the VM execution. By that, we can also avoid the imaging step in
most (non VM) tests.

This is something that is really hard to fix as we always have to run
the whole testsuite to expose the hidden inter-test dependencies. I'm
thinking of instead encoding the requirements of the VM tests in a
simple check script that parses the manifest of a image build and then
skips the test if dependencies are not there. By that, we can work on
the rebuilds one-by-one without breaking the testsuite.

I already have a similar patch in my queue to skip the VM tests if the
image itself is not available.

> 
> Regarding the contents on the patchset I like the ways it is done, and I 
> hope
> I will also rebuild my "Additional CI improvements" patchset sometimes, 
> which
> will also speedup our CI.

Sounds good. The other reported issues will be fixed in a v2.

Felix

> 
> -- 
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/5510e9b4-168e-4432-a966-832994290b65%40ilbers.de.

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/bf8c377f2f4649acb753c50933ebe190a676c8df.camel%40siemens.com.