Re: [PATCH v2 00/17] Various improvements to the testsuite

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Fri, 2025-12-19 at 14:19 +0100, Felix Moessbauer wrote:
> This series aims at simplifying and speeding up the testsuite execution.
> This should enable contributors to actually run it and test their patches.

Hi,

is there anything needed from my side?
Does the series pass your full CI?

Best regards,
Felix

> 
> As pointed out in [1], the testsuite currently tests various features over
> and over again, while others are not tested at all. We try to improve this
> by focusing on "feature" tests, whereby each test case checks a set of
> features instead of a whole image. We further identified long running tasks
> (like the kernel builds) and move them to a dedicated test.
> 
> The execution time reduction varies greatly depending on the hardware,
> network and sstate usage. When applying these changes on recently added
> patches, the time was often decreased by more than 50% on "fast".
> 
> Note, that more refactorings of the testsuite are needed - and planned,
> but this series should already address the biggest pain points.
> 
> [1] https://groups.google.com/g/isar-users/c/uZBTIHzLp8Q/m/8k0L1vfMDAAJ
> 
> Changes since v1:
> 
> - continued work on testsuite refactoring, this supersedes:
>   - testsuite: fix SignatureTest by avoiding absolute path in bblayers
>   - testsuite: add rootfs target for rootfs only tests
>   - Enable SState cache in GitLab CI
>   - testsuite: fix typo in log message in perform_signature_lint
> - completed exclusion / refactoring of db2m sbuild chroot
> - GitLab CI: increased after script timeout to give sstate upload more time
> - GitLab CI: do not upload sstate artifacts of images (as almost no reuse)
> - do not add SignatureTest to fast tag (as otherwise VM tests break...)
> - rebuild container test images in VM test to not depend on test order
> - rebased onto next
> 
> As always, the patches are written in a linear way, so only parts of them
> can already be integrated.
> 
> That's a wrap for this year, isar developers!
> See you next year!
> 
> Best regards,
> Felix Moessbauer
> Siemens AG
> 
> Felix Moessbauer (17):
>   testsuite: move targets with custom kernel to separate test
>   testsuite: enable ccache on kernel tests
>   testsuite: make prebuilt container a feature test
>   testsuite: make compat test standalone test
>   testsuite: handle IMAGE_INSTALL solely in cibuilder.py
>   testsuite: limit cross_debsrc test to subset of packages
>   testsuite: forward SSTATE_MIRRORS into CI env on sstate
>   ci: add support for sstate cache
>   testsuite: make test_cross_deps more specific
>   testsuite: fix typo in log message in perform_signature_lint
>   testsuite: fix SignatureTest by avoiding absolute path in bblayers
>   testsuite: use more recent distros in SignatureTest
>   testsuite: make SignatureTest idempotent
>   testsuite: add rootfs target for rootfs only tests
>   testsuite: refactor sbom tests to avoid overhead
>   testsuite: make sbuild-flavor test standalone
>   testsuite: skip VM tests if images are not available
> 
>  .gitlab-ci.yml                                |  36 ++++-
>  meta-test/conf/bblayers.conf.sample           |   8 +-
>  meta-test/conf/local.conf.sample              |  10 --
>  .../hello-isar/hello-isar.bbappend            |   3 +
>  .../recipes-app/libhello/libhello.bbappend    |   5 +
>  .../recipes-core/images/isar-image-ci.bb      |   2 -
>  .../recipes-core/images/isar-rootfs-ci.bb     |  17 +++
>  testsuite/cibase.py                           |  34 ++++-
>  testsuite/cibuilder.py                        |  49 ++++++-
>  testsuite/citest.py                           | 131 ++++++++++++++++--
>  10 files changed, 255 insertions(+), 40 deletions(-)
>  create mode 100644 meta-test/recipes-app/libhello/libhello.bbappend
>  create mode 100644 meta-test/recipes-core/images/isar-rootfs-ci.bb
> 
> -- 
> 2.51.0

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/508eb585ca5334136e71e56eac9a621e990d5b5a.camel%40siemens.com.