Re: [PATCH v3 00/20] Various improvements to the testsuite

[Zhihang Wei <wzh@ilbers.de>]

On 1/23/26 09:24, Zhihang Wei wrote:
> This series aims at simplifying and speeding up the testsuite execution.
> This should enable contributors to actually run it and test their patches.
>
> As pointed out in [1], the testsuite currently tests various features over
> and over again, while others are not tested at all. We try to improve this
> by focusing on "feature" tests, whereby each test case checks a set of
> features instead of a whole image. We further identified long running tasks
> (like the kernel builds) and move them to a dedicated test.
>
> The execution time reduction varies greatly depending on the hardware,
> network and sstate usage. When applying these changes on recently added
> patches, the time was often decreased by more than 50% on "fast".
>
> Note, that more refactorings of the testsuite are needed - and planned,
> but this series should already address the biggest pain points.
>
> [1] https://groups.google.com/g/isar-users/c/uZBTIHzLp8Q/m/8k0L1vfMDAAJ
>
> Changes since v2:
>
> - move startvm tests after each build to avoid inter-testsuite dependency
> - remove SstateTest from full

v3 has passed fast and full CI.

Zhihang
> Changes since v1:
>
> - continued work on testsuite refactoring, this supersedes:
>    - testsuite: fix SignatureTest by avoiding absolute path in bblayers
>    - testsuite: add rootfs target for rootfs only tests
>    - Enable SState cache in GitLab CI
>    - testsuite: fix typo in log message in perform_signature_lint
> - completed exclusion / refactoring of db2m sbuild chroot
> - GitLab CI: increased after script timeout to give sstate upload more time
> - GitLab CI: do not upload sstate artifacts of images (as almost no reuse)
> - do not add SignatureTest to fast tag (as otherwise VM tests break...)
> - rebuild container test images in VM test to not depend on test order
> - rebased onto next
>
> Anton Mikanovich (1):
>    testsuite: Group prebuilt_containers related test cases together
>
> MOESSBAUER, Felix (17):
>    testsuite: move targets with custom kernel to separate test
>    testsuite: enable ccache on kernel tests
>    testsuite: make prebuilt container a feature test
>    testsuite: make compat test standalone test
>    testsuite: handle IMAGE_INSTALL solely in cibuilder.py
>    testsuite: limit cross_debsrc test to subset of packages
>    testsuite: forward SSTATE_MIRRORS into CI env on sstate
>    ci: add support for sstate cache
>    testsuite: make test_cross_deps more specific
>    testsuite: fix typo in log message in perform_signature_lint
>    testsuite: fix SignatureTest by avoiding absolute path in bblayers
>    testsuite: use more recent distros in SignatureTest
>    testsuite: make SignatureTest idempotent
>    testsuite: add rootfs target for rootfs only tests
>    testsuite: refactor sbom tests to avoid overhead
>    testsuite: make sbuild-flavor test standalone
>    testsuite: skip VM tests if images are not available
>
> Zhihang Wei (2):
>    testsuite: Execute startvm testcases after building
>    testsuite: Remove SstateTest from full testsuite
>
>   .gitlab-ci.yml                                |  36 +-
>   meta-test/conf/bblayers.conf.sample           |   8 +-
>   meta-test/conf/local.conf.sample              |  10 -
>   .../hello-isar/hello-isar.bbappend            |   3 +
>   .../recipes-app/libhello/libhello.bbappend    |   5 +
>   .../recipes-core/images/isar-image-ci.bb      |   2 -
>   .../recipes-core/images/isar-rootfs-ci.bb     |  17 +
>   testsuite/cibase.py                           |  34 +-
>   testsuite/cibuilder.py                        |  49 +-
>   testsuite/citest.py                           | 740 +++++++++++-------
>   10 files changed, 602 insertions(+), 302 deletions(-)
>   create mode 100644 meta-test/recipes-app/libhello/libhello.bbappend
>   create mode 100644 meta-test/recipes-core/images/isar-rootfs-ci.bb
>

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/27f0e44f-5f2f-4cd2-990d-c232bda4ff38%40ilbers.de.