Re: [PATCH 0/3] Make CI targets be configured

[Henning Schild <henning.schild@siemens.com>]

The test suite already has too many knobs, now including env variables
would be the number one cause of "works for me" ... does not work in
CI-A and even different in CI-B.

I think if i pushed this series to our CI it would simply fail, because
i would not have any of the variables set there ... and no clue how you
did set them on your jenkins.

Tests that are known to not work should probably be skipped in general,
not kfailed because that would just waste time.

If we ever see certain tests not working in certain setups we can see
how we make skipping configurable, but it is imho a very bad idea to
introduce that without a need.

So i would say let us get rid of some of the KFAILs and turn the still
valid ones into SKIPs ... and see about the rest laster.

Henning

Am Wed, 27 Apr 2022 15:32:02 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> This patchset removes hardcoded KFAIL protection from targets and
> makes any target be KFAILed or SKIPed based on execution environment.
> ISAR_CI_KFAIL and ISAR_CI_SKIP variables are used to store those
> lists.
> 
> Anton Mikanovich (3):
>   ci: Implement dynamic KFAIL checking
>   ci: Implement dynamic tests skipping
>   ci: Correct container test case name
> 
>  testsuite/cibuilder.py | 56
> ++++++++++++++++++++++++++++-------------- testsuite/citest.py    |
> 32 ++++++------------------ 2 files changed, 45 insertions(+), 43
> deletions(-)
>