From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6990633379674718208 X-Received: by 2002:a05:6402:5210:: with SMTP id s16mr6483174edd.343.1628784802915; Thu, 12 Aug 2021 09:13:22 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:907:20e9:: with SMTP id rh9ls2780722ejb.10.gmail; Thu, 12 Aug 2021 09:13:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyg6EzQ0ZqxDifbYfAabTvOg/oWjH2StCh15eZAC7Xhc5fa9R50nTLJG7uGVoDAHgPEqw14 X-Received: by 2002:a17:906:b48:: with SMTP id v8mr4434387ejg.258.1628784801859; Thu, 12 Aug 2021 09:13:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628784801; cv=none; d=google.com; s=arc-20160816; b=uH484nok3cUkFfMDxePIE0x6nVEx7TtyJxzGJQHTqYvSiNaBh3VPV/ofTvDIPSGJ8l 8CpDRs7j93MjKsBLN6zc8usqYnDd1roCn2JeOyz4GBAIJVM30h8Tq2Wl0+Z5gmGNEunh fVxDIvoCmlH504lvi9pMQ+hM9HN5MhJGqPcXoozohiWa2wMiJjgWAUc2kJEGYgue5u0d lkAqRa8C4M68/vNxQNMarfdMKlix+DTRaEU7aFJSkwGudErKa9MS1WsSj2v39odg3/mt EYtiOE/UB7XAVvEJRSLB4yQlR1j5iTXsFhtb/xWzUIDb+9jfvAihqlpn/ZSlIqnT0NEW ydng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=rilJEmKTgQD4IZVZQ++Yib3gxTqHndRQHaYzO+1chkg=; b=MLxnUzXBikZORNsUgM1HJBJa6ik4bqFCCb9dN1aS7OqGgw0JhG4RFQomlRvYOOCDTc e6XsCcVt/gQNQapdKj0WE5ApDTiRo7X29r4NDVRARfukTcZmH3umwEHNYGbbXf8qIvRN 4fLrxy2cj6qMUSvWuOlLYsEp3rJpu0VBHjYiJWI463iRvJXT1Olu3K6FCx6qxEYAylXG YQd9bn8QHKbTjHMVwsSDmObCawkJhWFyvv20glnWPBDd1lmyHg0hHXfp65P6IzjN6ccd llrEuxUvqPHYAmzRHtFknumDLoHkIB6Bz+7TxQ6buLUrKLsYLrMCpBkAlYB5uHHlO2UX u0qw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id s18si252550ejo.1.2021.08.12.09.13.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Aug 2021 09:13:21 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 17CGDLtv029043 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 12 Aug 2021 18:13:21 +0200 Received: from md1za8fc.ad001.siemens.net ([139.25.0.59]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 17CGDLUN004061; Thu, 12 Aug 2021 18:13:21 +0200 Date: Thu, 12 Aug 2021 18:13:19 +0200 From: Henning Schild To: Anton Mikanovich Cc: isar-users@googlegroups.com Subject: Re: [PATCH v10 00/17] Update Avocado testsuite Message-ID: <20210812181319.5bd465e7@md1za8fc.ad001.siemens.net> In-Reply-To: <20210730082739.15042-1-amikan@ilbers.de> References: <20210730082739.15042-1-amikan@ilbers.de> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: FOdsQXqPkUkl I started looking into the avocado tests we currently have. Seems that they are still pretty different from what we used to have. Will this new version implement all the tests we used to have? My solution was to install that very unlucky choice of software with virtualenv (what i always do when forced to pip). And i worked on the gitlab-ci stuff. Came up with something similar like you. Just a few more bits on nice integration i will propose later when this series got merged. Henning Am Fri, 30 Jul 2021 11:27:22 +0300 schrieb Anton Mikanovich : > This patchset provides implementation of build and vm_start tests > functionality in Python with help of Avocado framework. > Scripts ci_build and vm_smoke_test are rewritten to call Avocado. > ci_build and vm_smoke_test are left for compatibility and > should be replaced by direct avocado calls later. > Original ci_build and vm_smoke_test are renamed to *_legacy. > Build and run logs are hidden now until an error occurs. > > --- > Changes since v9: > - Rebase on next. > - Fix avocado artifacts path. > Changes since v8: > - Remove duplication in logs. > - Disable sysinfo plugin. > - Show full errors in default mode. > Changes since v7: > - Remove code duplication. > - Fix build_dir parsing. > - Add missed protect files backup logic. > Changes since v6: > - Improve environment parsing. > - Protect files backup logic. > - Rebuild full log output option. > - Protect ubuntu target with KFAIL. > Changes since v5: > - Rebase on current next. > Changes since v4: > - Add container tests. > - Implement testcase skipping logic. > - Separate and protect Bullseye tests. > - Add checks for container dependencies. > - Add Ilbers repo key. > - Improve error handling. > - Fix Ubuntu qemu image name. > Changes since v3: > - Add automatic bitbake init. > - Fix line lengths. > - Fix build without repro. > - Fix build artifact paths. > - Prepare for Gitlab build. > - Change debug settings. > - Move log splitting to start_vm. > - Add deb package install. > Changes since v2: > - Make CI call Avocado tests. > - Wait less on vm_start test. > - Split Roman's vm_start commit. > - Put logs to build folder. > - Rearrange patches. > - Split build test to classes. > - Add test case tagging. > - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. > - Get rid of python-subprocess32. > - Improve logging. > Changes since v1: > - Merge start_vm rebuild patches. > - Fix patch comments. > > Anton Mikanovich (14): > start_vm.py: Fix ubuntu image name > vm_boot_test: Fix log file path in vm_boot_test > vm_boot_test: Remove external varianter > vm_boot_test: Improve QEMU images checking > build_test: Refactoring build tests cases > testsuite: Add Python generations for testsuite in gitignore > testsuite: Fix test suite prepare guide > gitlab-ci: Add Avocado build artifacts > gitlab-ci: Add debug flag > vm_boot_test: Add automatic bitbake init > build_test: Protect ubuntu target with KFAIL > ci_build: Migrate to Avocado > vm_smoke_test: Migrate to Avocado > ci_build: Install Avocado if needed > > Roman Pletnev (3): > start_vm.py: Fix target name handling > start_vm.py: Add output and PID file vm_start.py options > start_vm.py: Add MIPS support > > .gitlab-ci.yml | 5 +- > scripts/ci_build.sh | 213 ++++---------------- > scripts/ci_build_legacy.sh | 267 > +++++++++++++++++++++++++ scripts/vm_smoke_test | > 110 ++-------- scripts/vm_smoke_test_legacy | 168 > ++++++++++++++++ testsuite/.gitignore | 1 + > testsuite/README.md | 21 +- > testsuite/build_test/build_test.py | 217 ++++++++++++++++++-- > testsuite/build_test/cibase.py | 110 ++++++++++ > testsuite/build_test/cibuilder.py | 130 ++++++++++++ > testsuite/build_test/run.sh | 3 - > testsuite/build_test/run_fast.sh | 3 + > testsuite/build_test/run_full.sh | 3 + > testsuite/build_test/variant.yaml | 22 -- > testsuite/start_vm.py | 27 ++- > testsuite/vm_boot_test/run.sh | 3 - > testsuite/vm_boot_test/run_fast.sh | 3 + > testsuite/vm_boot_test/run_full.sh | 3 + > testsuite/vm_boot_test/variant.yaml | 22 -- > testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++-- > 20 files changed, 1070 insertions(+), 382 deletions(-) > create mode 100755 scripts/ci_build_legacy.sh > create mode 100755 scripts/vm_smoke_test_legacy > create mode 100644 testsuite/.gitignore > create mode 100644 testsuite/build_test/cibase.py > create mode 100644 testsuite/build_test/cibuilder.py > delete mode 100755 testsuite/build_test/run.sh > create mode 100755 testsuite/build_test/run_fast.sh > create mode 100755 testsuite/build_test/run_full.sh > delete mode 100644 testsuite/build_test/variant.yaml > delete mode 100755 testsuite/vm_boot_test/run.sh > create mode 100755 testsuite/vm_boot_test/run_fast.sh > create mode 100755 testsuite/vm_boot_test/run_full.sh > delete mode 100644 testsuite/vm_boot_test/variant.yaml >