Re: [PATCH v10 00/17] Update Avocado testsuite

[Henning Schild <henning.schild@siemens.com>]

I was only playing with what we currently have in next. But that seems
to serialize the builds listed in that yaml. While before bitbake did
run them all at once.
I think the tests from that yaml should run in parallel to cause the
same kind of stress ... just parallel in the testsuite instead of
bitbake. And to hopefully speed things up a bit.

Henning

Am Fri, 30 Jul 2021 11:27:22 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> This patchset provides implementation of build and vm_start tests
> functionality in Python with help of Avocado framework.
> Scripts ci_build and vm_smoke_test are rewritten to call Avocado.
> ci_build and vm_smoke_test are left for compatibility and
> should be replaced by direct avocado calls later.
> Original ci_build and vm_smoke_test are renamed to *_legacy.
> Build and run logs are hidden now until an error occurs.
> 
> ---
> Changes since v9:
> - Rebase on next.
> - Fix avocado artifacts path.
> Changes since v8:
> - Remove duplication in logs.
> - Disable sysinfo plugin.
> - Show full errors in default mode.
> Changes since v7:
> - Remove code duplication.
> - Fix build_dir parsing.
> - Add missed protect files backup logic.
> Changes since v6:
> - Improve environment parsing.
> - Protect files backup logic.
> - Rebuild full log output option.
> - Protect ubuntu target with KFAIL.
> Changes since v5:
> - Rebase on current next.
> Changes since v4:
> - Add container tests.
> - Implement testcase skipping logic.
> - Separate and protect Bullseye tests.
> - Add checks for container dependencies.
> - Add Ilbers repo key.
> - Improve error handling.
> - Fix Ubuntu qemu image name.
> Changes since v3:
> - Add automatic bitbake init.
> - Fix line lengths.
> - Fix build without repro.
> - Fix build artifact paths.
> - Prepare for Gitlab build.
> - Change debug settings.
> - Move log splitting to start_vm.
> - Add deb package install.
> Changes since v2:
> - Make CI call Avocado tests.
> - Wait less on vm_start test.
> - Split Roman's vm_start commit.
> - Put logs to build folder.
> - Rearrange patches.
> - Split build test to classes.
> - Add test case tagging.
> - Get rid of avocado-framework-plugin-varianter-yaml-to-mux.
> - Get rid of python-subprocess32.
> - Improve logging.
> Changes since v1:
> - Merge start_vm rebuild patches.
> - Fix patch comments.
> 
> Anton Mikanovich (14):
>   start_vm.py: Fix ubuntu image name
>   vm_boot_test: Fix log file path in vm_boot_test
>   vm_boot_test: Remove external varianter
>   vm_boot_test: Improve QEMU images checking
>   build_test: Refactoring build tests cases
>   testsuite: Add Python generations for testsuite in gitignore
>   testsuite: Fix test suite prepare guide
>   gitlab-ci: Add Avocado build artifacts
>   gitlab-ci: Add debug flag
>   vm_boot_test: Add automatic bitbake init
>   build_test: Protect ubuntu target with KFAIL
>   ci_build: Migrate to Avocado
>   vm_smoke_test: Migrate to Avocado
>   ci_build: Install Avocado if needed
> 
> Roman Pletnev (3):
>   start_vm.py: Fix target name handling
>   start_vm.py: Add output and PID file vm_start.py options
>   start_vm.py: Add MIPS support
> 
>  .gitlab-ci.yml                         |   5 +-
>  scripts/ci_build.sh                    | 213 ++++----------------
>  scripts/ci_build_legacy.sh             | 267
> +++++++++++++++++++++++++ scripts/vm_smoke_test                  |
> 110 ++-------- scripts/vm_smoke_test_legacy           | 168
> ++++++++++++++++ testsuite/.gitignore                   |   1 +
>  testsuite/README.md                    |  21 +-
>  testsuite/build_test/build_test.py     | 217 ++++++++++++++++++--
>  testsuite/build_test/cibase.py         | 110 ++++++++++
>  testsuite/build_test/cibuilder.py      | 130 ++++++++++++
>  testsuite/build_test/run.sh            |   3 -
>  testsuite/build_test/run_fast.sh       |   3 +
>  testsuite/build_test/run_full.sh       |   3 +
>  testsuite/build_test/variant.yaml      |  22 --
>  testsuite/start_vm.py                  |  27 ++-
>  testsuite/vm_boot_test/run.sh          |   3 -
>  testsuite/vm_boot_test/run_fast.sh     |   3 +
>  testsuite/vm_boot_test/run_full.sh     |   3 +
>  testsuite/vm_boot_test/variant.yaml    |  22 --
>  testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++--
>  20 files changed, 1070 insertions(+), 382 deletions(-)
>  create mode 100755 scripts/ci_build_legacy.sh
>  create mode 100755 scripts/vm_smoke_test_legacy
>  create mode 100644 testsuite/.gitignore
>  create mode 100644 testsuite/build_test/cibase.py
>  create mode 100644 testsuite/build_test/cibuilder.py
>  delete mode 100755 testsuite/build_test/run.sh
>  create mode 100755 testsuite/build_test/run_fast.sh
>  create mode 100755 testsuite/build_test/run_full.sh
>  delete mode 100644 testsuite/build_test/variant.yaml
>  delete mode 100755 testsuite/vm_boot_test/run.sh
>  create mode 100755 testsuite/vm_boot_test/run_fast.sh
>  create mode 100755 testsuite/vm_boot_test/run_full.sh
>  delete mode 100644 testsuite/vm_boot_test/variant.yaml
>