Re: [PATCH 0/8] CI rework of gitlab fast job

[Baurzhan Ismagulov <ibr@radix50.net>]

On Fri, Jan 13, 2023 at 12:43:15AM +0100, Henning Schild wrote:
> At the end of the day i want any variant of the testsuite to run on any
> setup just like it does on ilbers jenkins.
> 
> That should be based on the kas containers for anyone not using your
> jenkins.
> 
> With that any contributor would be able to run it manually or in their
> CI.
> 
> We currently can not run "full" in gitlab CI because it takes too much
> time and our gitlab runners would take such jobs for at most 12h. But
> that is a value i could adjust if we find that it really needs to take
> that long. I hope we can get much faster and still have the same
> coverage.

Thanks for the summary, that's fully in line with our last off-list discussion:

* The preferred downstream testsuite is "dev" (runs for ~ 2 h IIRC).

* The testsuite runs manually on the host or in kas.

* The testsuite runs automatically in GitLab, Jenkins or Azure using kas.

* Separate testcases for every arch / distro combo. One can run the complete
  testsuite or individual testcases.

* The maintainer CI provides test status and build control on a publicly
  accessible, easily scalable infrastructure.

* The CI setup is easily clonable for the downstreams.

* Downstreams can run Isar testsuites and / or their own testsuites.


So, I propose that we update and apply the "dev" testsuite series, then apply
yours with the following changes:

1. Use "dev" in "[PATCH 3/8] CI: install qemu-system when qemu testing is
   requested".

2. Patches 4-8 "testsuite: Remove X test from "fast" set" will be unnecessary
   (we'd take this into account during the "dev" series update -- those
   testcases are not part of the "dev" v1 anyway).

For the final review of the use cases I'll wait for Anton.


With kind regards,
Baurzhan