From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7008446828784910336 X-Received: by 2002:a2e:bf07:: with SMTP id c7mr10988493ljr.394.1632485932672; Fri, 24 Sep 2021 05:18:52 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3f14:: with SMTP id y20ls2404534lfa.1.gmail; Fri, 24 Sep 2021 05:18:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzrttRhfk+lfJp0Aw5LdaJ2MGtyQxa9oqNLG/SKieT0sjtjfP3NIwa7Uyw3QiQpOoAt1LvS X-Received: by 2002:a19:6512:: with SMTP id z18mr9320797lfb.106.1632485931625; Fri, 24 Sep 2021 05:18:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1632485931; cv=none; d=google.com; s=arc-20160816; b=EArazStJlUfiKQ+ZlIG7TLkwjzi1DETrSV0eD1IgtiQ5o+48E90UwCJbsB4vQw67eo IoEjJQ/6mvKXD2ujY1uFnxMM3vnQ6WM4Z9S/AAmyp58zpBdX3kcafQG53l2m4SEe2dP9 mrVNe2O8umqoGnoKxHmEpAwjfD9opmsQjlR/2zRcrRh4vcJCk5wL/hgSPETOG4ZzUs0B azaVwM3fv8wjTOtzZIH3bkLHQFNa2OO5J7o7SHmu21Scrkkyo8kUjnDuXyTbyCfHA/W1 YY7t0f2bl42EXP7ZKyI9rUn0KWZcfBcHjAS4mfhzFR/9AEJG/oHUt65vbRyC5idT99s5 ye1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=4RfXyZERCJuDDFsGbAR6rQugO/bPffywbeErBGJkjik=; b=dlxygNDOl2Nt046jCx54gCQ0X8aFcR1r6AptaFw4JzHNiMjZ+x6wDYxPgwfxkNLCGk tGtNJgWpvVLEQreOCY7tCHYf0O2OihPkuaJHDKGihziU2mT0tHIZOgS9VDRC57TUv5vT fEIs77GRdyqZN+LaC0JCAEUikIc1ebA1AdmJ/V2NUrpSeSEFMn1Ye2dnHqSn4Q0J0iaP KEILGUbIQ8L6FCjYGIFXElxz5X0ZWzzmgP5PjU/gK+ljfApNgrNH4giS85S7+NUzy9TO Dz2opKeWVt+zOXfoSNwGt33d6GyQLbtlHdpnkvE66PrmRYfCyGkVvzltgfZyWiMJzRN4 prFQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id z21si553262ljn.1.2021.09.24.05.18.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Sep 2021 05:18:51 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 18OCIogH024613 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Sep 2021 14:18:50 +0200 Received: from md1za8fc.ad001.siemens.net ([139.25.0.59]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 18OCIo8o025207; Fri, 24 Sep 2021 14:18:50 +0200 Date: Fri, 24 Sep 2021 14:18:50 +0200 From: Henning Schild To: Anton Mikanovich Cc: isar-users@googlegroups.com Subject: Re: [PATCH v12 00/16] Update Avocado testsuite Message-ID: <20210924141850.6cf5d6fd@md1za8fc.ad001.siemens.net> In-Reply-To: <20210917170033.74797-1-amikan@ilbers.de> References: <20210917170033.74797-1-amikan@ilbers.de> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: EP0jgHkzdB4K Am Fri, 17 Sep 2021 20:00:17 +0300 schrieb Anton Mikanovich : > This patchset provides implementation of build and vm_start tests > functionality in Python with help of Avocado framework. > Scripts ci_build and vm_smoke_test are rewritten to call Avocado. > ci_build and vm_smoke_test are left for compatibility and > should be replaced by direct avocado calls later. What to stay compatible with? I think the old one should be dropped right when the new one comes. Or do we maintain both wile we have both? I am all for dropping the old and merging this rather soon. Because any change made is likely to add tests ... and will have merge conflicts with this not being in the tree yet. Henning > Original ci_build and vm_smoke_test are renamed to *_legacy. > Build and run logs are hidden by default now until an error occurs. > > --- > Changes since v11: > - Rebase on next. > - Init environment in vm_smoke_test. > Changes since v10: > - Rebase on next. > - Add DISTRO_APT_PREMIRRORS usage. > - Remove avocado deb package install. > - Set default runner. > Changes since v9: > - Rebase on next. > - Fix avocado artifacts path. > Changes since v8: > - Remove duplication in logs. > - Disable sysinfo plugin. > - Show full errors in default mode. > Changes since v7: > - Remove code duplication. > - Fix build_dir parsing. > - Add missed protect files backup logic. > Changes since v6: > - Improve environment parsing. > - Protect files backup logic. > - Rebuild full log output option. > - Protect ubuntu target with KFAIL. > Changes since v5: > - Rebase on current next. > Changes since v4: > - Add container tests. > - Implement testcase skipping logic. > - Separate and protect Bullseye tests. > - Add checks for container dependencies. > - Add Ilbers repo key. > - Improve error handling. > - Fix Ubuntu qemu image name. > Changes since v3: > - Add automatic bitbake init. > - Fix line lengths. > - Fix build without repro. > - Fix build artifact paths. > - Prepare for Gitlab build. > - Change debug settings. > - Move log splitting to start_vm. > - Add deb package install. > Changes since v2: > - Make CI call Avocado tests. > - Wait less on vm_start test. > - Split Roman's vm_start commit. > - Put logs to build folder. > - Rearrange patches. > - Split build test to classes. > - Add test case tagging. > - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. > - Get rid of python-subprocess32. > - Improve logging. > Changes since v1: > - Merge start_vm rebuild patches. > - Fix patch comments. > > Anton Mikanovich (13): > start_vm.py: Fix ubuntu image name > vm_boot_test: Fix log file path in vm_boot_test > vm_boot_test: Remove external varianter > vm_boot_test: Improve QEMU images checking > build_test: Refactoring build tests cases > testsuite: Add Python generations for testsuite in gitignore > testsuite: Fix test suite prepare guide > gitlab-ci: Add Avocado build artifacts > gitlab-ci: Add debug flag > vm_boot_test: Add automatic bitbake init > build_test: Protect ubuntu target with KFAIL > ci_build: Migrate to Avocado > vm_smoke_test: Migrate to Avocado > > Roman Pletnev (3): > start_vm.py: Fix target name handling > start_vm.py: Add output and PID file vm_start.py options > start_vm.py: Add MIPS support > > .gitlab-ci.yml | 5 +- > scripts/ci_build.sh | 208 +++---------------- > scripts/ci_build_legacy.sh | 275 > +++++++++++++++++++++++++ scripts/vm_smoke_test | > 120 +++-------- scripts/vm_smoke_test_legacy | 168 > +++++++++++++++ testsuite/.gitignore | 1 + > testsuite/README.md | 21 +- > testsuite/build_test/build_test.py | 219 ++++++++++++++++++-- > testsuite/build_test/cibase.py | 116 +++++++++++ > testsuite/build_test/cibuilder.py | 132 ++++++++++++ > testsuite/build_test/run.sh | 3 - > testsuite/build_test/run_fast.sh | 3 + > testsuite/build_test/run_full.sh | 3 + > testsuite/build_test/variant.yaml | 22 -- > testsuite/start_vm.py | 27 ++- > testsuite/vm_boot_test/run.sh | 3 - > testsuite/vm_boot_test/run_fast.sh | 3 + > testsuite/vm_boot_test/run_full.sh | 3 + > testsuite/vm_boot_test/variant.yaml | 22 -- > testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++-- > 20 files changed, 1084 insertions(+), 391 deletions(-) > create mode 100755 scripts/ci_build_legacy.sh > create mode 100755 scripts/vm_smoke_test_legacy > create mode 100644 testsuite/.gitignore > create mode 100644 testsuite/build_test/cibase.py > create mode 100644 testsuite/build_test/cibuilder.py > delete mode 100755 testsuite/build_test/run.sh > create mode 100755 testsuite/build_test/run_fast.sh > create mode 100755 testsuite/build_test/run_full.sh > delete mode 100644 testsuite/build_test/variant.yaml > delete mode 100755 testsuite/vm_boot_test/run.sh > create mode 100755 testsuite/vm_boot_test/run_fast.sh > create mode 100755 testsuite/vm_boot_test/run_full.sh > delete mode 100644 testsuite/vm_boot_test/variant.yaml >