From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6969105759827132416 X-Received: by 2002:a19:ca0e:: with SMTP id a14mr2654210lfg.384.1625207351274; Thu, 01 Jul 2021 23:29:11 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:508b:: with SMTP id f11ls151926lfm.2.gmail; Thu, 01 Jul 2021 23:29:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwbkKE8pUcVkthHAefmWkbaxLzeGe4g/OSQJcQX0hEC/SuDr0Uv1QQT01ZKC6asgJE2eLtH X-Received: by 2002:a19:f51a:: with SMTP id j26mr2703494lfb.390.1625207350109; Thu, 01 Jul 2021 23:29:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625207350; cv=none; d=google.com; s=arc-20160816; b=fiB3BHtupxjl7w7NwJVKYGx25MV6/XsSy7PKu/rnqilJjoU3trD2r9uj4opVltQpvc jwqxyIm9gQkX0MlW2V5+xUe3like9XE8939tda8/NPv8bRiGFnxWTxqRmvYghJdfDJlm r7vz5QZ3qKlZnrWgJEYASn5nusRY0oyZqpG0eJ4WYF3ApLdhU7H3R6jfcZuh3izzkk4J YFAtl7HU3bEXY2ClLT/PvuI0HsEk+lcMB9YHRJtxL5IH+mTtmn4yn92w6gOxigV6suYY sTQZg16N7XfYQUlfwPtEZJWvp1mP5gtdTdAfbx4adY65jpJZEdKBV0md+mHnhuy+hg/T yAEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=Pj+muSmc7G6F0iGEF3VPT+W2g458rEVJyItBMGBQAsw=; b=QGnVm9VE6HuAOHkDzqSycayXO6Y+Kg+ePcBl3J4OWFA3WkqxesnBUAJH5Xq3hvJAU0 3i4mHSgznWI5xFj4Bn+smJn/PRgt66J6w8/zUjPGhkyJMGtIyKDUdUHW2lajM3K5J6gQ 7TQiI4wf3JxiCOmaq4gUEaEYQQ3vQw3NLpI3VuN5lIn7DszkXUeu7P6njNLEtjERkRU4 N3hD9j+J/jJJCSzmjisJZT5E4dXSfsG4iaT8D1QfkHUmukkm8+OYAFgfkyQqJo3h4CRG Vdh9x0ZPfulgNlgSjCepTTMrrwpBgFri9ztd1GgwsTepvJUZM05kNzzPQrtPI2ivvxw0 ggjQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id a16si52860ljq.8.2021.07.01.23.29.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Jul 2021 23:29:10 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 1626T9He017346 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Jul 2021 08:29:09 +0200 Received: from [139.22.38.250] ([139.22.38.250]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 1626T8Z9014360; Fri, 2 Jul 2021 08:29:08 +0200 Subject: Re: [PATCH v9 00/17] Update Avocado testsuite To: Anton Mikanovich , isar-users@googlegroups.com References: <20210630112909.87092-1-amikan@ilbers.de> From: Jan Kiszka Message-ID: <7c8c0e32-ebf1-6c28-66d3-536e261c9320@siemens.com> Date: Fri, 2 Jul 2021 08:29:08 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210630112909.87092-1-amikan@ilbers.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: iDuP/ZLVtfWj On 30.06.21 13:28, Anton Mikanovich wrote: > This patchset provides implementation of build and vm_start tests > functionality in Python with help of Avocado framework. > Scripts ci_build and vm_smoke_test are rewritten to call Avocado. > ci_build and vm_smoke_test are left for compatibility and > should be replaced by direct avocado calls later. > Original ci_build and vm_smoke_test are renamed to *_legacy. > Build and run logs are hidden now until an error occurs. > Do I find this version as git branch somewhere? Looks like amikan/avocado was not updated. Thanks, Jan > --- > Changes since v8: > - Remove duplication in logs. > - Disable sysinfo plugin. > - Show full errors in default mode. > Changes since v7: > - Remove code duplication. > - Fix build_dir parsing. > - Add missed protect files backup logic. > Changes since v6: > - Improve environment parsing. > - Protect files backup logic. > - Rebuild full log output option. > - Protect ubuntu target with KFAIL. > Changes since v5: > - Rebase on current next. > Changes since v4: > - Add container tests. > - Implement testcase skipping logic. > - Separate and protect Bullseye tests. > - Add checks for container dependencies. > - Add Ilbers repo key. > - Improve error handling. > - Fix Ubuntu qemu image name. > Changes since v3: > - Add automatic bitbake init. > - Fix line lengths. > - Fix build without repro. > - Fix build artifact paths. > - Prepare for Gitlab build. > - Change debug settings. > - Move log splitting to start_vm. > - Add deb package install. > Changes since v2: > - Make CI call Avocado tests. > - Wait less on vm_start test. > - Split Roman's vm_start commit. > - Put logs to build folder. > - Rearrange patches. > - Split build test to classes. > - Add test case tagging. > - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. > - Get rid of python-subprocess32. > - Improve logging. > Changes since v1: > - Merge start_vm rebuild patches. > - Fix patch comments. > > Anton Mikanovich (14): > start_vm.py: Fix ubuntu image name > vm_boot_test: Fix log file path in vm_boot_test > vm_boot_test: Remove external varianter > vm_boot_test: Improve QEMU images checking > build_test: Refactoring build tests cases > testsuite: Add Python generations for testsuite in gitignore > testsuite: Fix test suite prepare guide > gitlab-ci: Add Avocado build artifacts > gitlab-ci: Add debug flag > vm_boot_test: Add automatic bitbake init > build_test: Protect ubuntu target with KFAIL > ci_build: Migrate to Avocado > vm_smoke_test: Migrate to Avocado > ci_build: Install Avocado if needed > > Roman Pletnev (3): > start_vm.py: Fix target name handling > start_vm.py: Add output and PID file vm_start.py options > start_vm.py: Add MIPS support > > .gitlab-ci.yml | 5 +- > scripts/ci_build.sh | 213 ++++---------------- > scripts/ci_build_legacy.sh | 267 +++++++++++++++++++++++++ > scripts/vm_smoke_test | 110 ++-------- > scripts/vm_smoke_test_legacy | 168 ++++++++++++++++ > testsuite/.gitignore | 1 + > testsuite/README.md | 21 +- > testsuite/build_test/build_test.py | 217 ++++++++++++++++++-- > testsuite/build_test/cibase.py | 110 ++++++++++ > testsuite/build_test/cibuilder.py | 130 ++++++++++++ > testsuite/build_test/run.sh | 3 - > testsuite/build_test/run_fast.sh | 3 + > testsuite/build_test/run_full.sh | 3 + > testsuite/build_test/variant.yaml | 22 -- > testsuite/start_vm.py | 27 ++- > testsuite/vm_boot_test/run.sh | 3 - > testsuite/vm_boot_test/run_fast.sh | 3 + > testsuite/vm_boot_test/run_full.sh | 3 + > testsuite/vm_boot_test/variant.yaml | 22 -- > testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++-- > 20 files changed, 1070 insertions(+), 382 deletions(-) > create mode 100755 scripts/ci_build_legacy.sh > create mode 100755 scripts/vm_smoke_test_legacy > create mode 100644 testsuite/.gitignore > create mode 100644 testsuite/build_test/cibase.py > create mode 100644 testsuite/build_test/cibuilder.py > delete mode 100755 testsuite/build_test/run.sh > create mode 100755 testsuite/build_test/run_fast.sh > create mode 100755 testsuite/build_test/run_full.sh > delete mode 100644 testsuite/build_test/variant.yaml > delete mode 100755 testsuite/vm_boot_test/run.sh > create mode 100755 testsuite/vm_boot_test/run_fast.sh > create mode 100755 testsuite/vm_boot_test/run_full.sh > delete mode 100644 testsuite/vm_boot_test/variant.yaml > -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux