From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7008446828784910336 X-Received: by 2002:a1c:3b44:: with SMTP id i65mr4414620wma.161.1634201680209; Thu, 14 Oct 2021 01:54:40 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:35cd:: with SMTP id r13ls2482443wmq.2.gmail; Thu, 14 Oct 2021 01:54:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzkcSP/ZWcbIOAkFj5TvQUK/jflsBYWTQ6goZedHUgusOP1Ho9KMsyrtvNNH8JnHpPyxmGK X-Received: by 2002:a05:600c:190a:: with SMTP id j10mr17171922wmq.149.1634201679183; Thu, 14 Oct 2021 01:54:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634201679; cv=none; d=google.com; s=arc-20160816; b=L6otAJ7EL/Vgu4KEQjQ3wh1VYG9jprwXs9pMp0b/1pPj2QHFWydAgzIabWHhr7df8d vBlPJiZFn++QHv7EToggWG3804FyRwKKBJVa8nf+SmVbyeIHxNCdQTKGaCUNd5irv/wf wKmyp1o5q6Tle4Fj5u9Usys7I7hIabqsv+9pLWGYXplmFCVEt1AC4lOQChO9BE2JblLo VMP6E+5237WX3BAMXkQGm7ZCZ5lULKzjC8PbvWOQfOATs/syp+eE2KpK9EzbMYN4wTfl dfC3Jsm87TZO4z9ieV92yGMwKrQ93XxJoe4nTZ6TMuQejDbyKhu32BQKhuhiTSZdhZjf T84g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:cc:references:to:subject; bh=BRAtW8e51UkiHV61WfhUQo/Ee1gr91jpy4jSP4FehG0=; b=V1SJs88cUf/WDLj9hQXRxWrcmy42Vgt1rUjyg+2j7wLgqwcbHyszD4ACeDNLaCj5n6 pLtpalYJXhLITxmVrOp11Jx1ywxkfMmZTtfxXD9NuRxCbHUy5G7NxkgUvY/XKXPMCXOO XAYi8HO5ORSuTN1joot3fOJ4Ymwghc1TiHBSe6DlKRVSr5xizzvC7dN7I1CD7rPReq9S KumhdBDha3LZ3KysKlr0u9exBvrEHl1/BnGZvKjwTDj9UO811l9jjf4Ox99Vbg3tMRg/ pfbXFlPPw+4Qioe1tFrCFt3N1E+FFaIbcfEg79GspbDsQQb/hg37Mo78LxuG629JhBKn KN8w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id h11si104070wrs.3.2021.10.14.01.54.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Oct 2021 01:54:39 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 19E8scfb003604 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Oct 2021 10:54:38 +0200 Received: from [167.87.2.42] ([167.87.2.42]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 19E8sbb1019856; Thu, 14 Oct 2021 10:54:37 +0200 Subject: Re: [PATCH v14 00/16] Update Avocado testsuite To: Anton Mikanovich , isar-users@googlegroups.com References: <1633020341-26062-1-git-send-email-anton.mikanovich@promwad.com> Cc: "Schmidt, Adriaan" From: Jan Kiszka Message-ID: <98b328db-e1ef-0a2a-8ddd-7fe8f9c341ab@siemens.com> Date: Thu, 14 Oct 2021 10:54:37 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <1633020341-26062-1-git-send-email-anton.mikanovich@promwad.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: OlxhZ6pGJEzI On 30.09.21 18:45, Anton Mikanovich wrote: > This patchset provides implementation of build and vm_start tests > functionality in Python with help of Avocado framework. > Scripts ci_build and vm_smoke_test are rewritten to call Avocado. > Build and run logs are hidden by default now until an error occurs. > > Resending v14 because v13 was just a copy of v12 by mistake. > > --- > Changes since v13: > - Remove legacy scripts. > Changes since v12: > - No changes, wrong patchset. > Changes since v11: > - Rebase on next. > - Init environment in vm_smoke_test. > Changes since v10: > - Rebase on next. > - Add DISTRO_APT_PREMIRRORS usage. > - Remove avocado deb package install. > - Set default runner. > Changes since v9: > - Rebase on next. > - Fix avocado artifacts path. > Changes since v8: > - Remove duplication in logs. > - Disable sysinfo plugin. > - Show full errors in default mode. > Changes since v7: > - Remove code duplication. > - Fix build_dir parsing. > - Add missed protect files backup logic. > Changes since v6: > - Improve environment parsing. > - Protect files backup logic. > - Rebuild full log output option. > - Protect ubuntu target with KFAIL. > Changes since v5: > - Rebase on current next. > Changes since v4: > - Add container tests. > - Implement testcase skipping logic. > - Separate and protect Bullseye tests. > - Add checks for container dependencies. > - Add Ilbers repo key. > - Improve error handling. > - Fix Ubuntu qemu image name. > Changes since v3: > - Add automatic bitbake init. > - Fix line lengths. > - Fix build without repro. > - Fix build artifact paths. > - Prepare for Gitlab build. > - Change debug settings. > - Move log splitting to start_vm. > - Add deb package install. > Changes since v2: > - Make CI call Avocado tests. > - Wait less on vm_start test. > - Split Roman's vm_start commit. > - Put logs to build folder. > - Rearrange patches. > - Split build test to classes. > - Add test case tagging. > - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. > - Get rid of python-subprocess32. > - Improve logging. > Changes since v1: > - Merge start_vm rebuild patches. > - Fix patch comments. > > Anton Mikanovich (13): > start_vm.py: Fix ubuntu image name > vm_boot_test: Fix log file path in vm_boot_test > vm_boot_test: Remove external varianter > vm_boot_test: Improve QEMU images checking > build_test: Refactoring build tests cases > testsuite: Add Python generations for testsuite in gitignore > testsuite: Fix test suite prepare guide > gitlab-ci: Add Avocado build artifacts > gitlab-ci: Add debug flag > vm_boot_test: Add automatic bitbake init > build_test: Protect ubuntu target with KFAIL > ci_build: Migrate to Avocado > vm_smoke_test: Migrate to Avocado > > Roman Pletnev (3): > start_vm.py: Fix target name handling > start_vm.py: Add output and PID file vm_start.py options > start_vm.py: Add MIPS support > > .gitlab-ci.yml | 5 +- > scripts/ci_build.sh | 208 ++++--------------------------- > scripts/vm_smoke_test | 120 +++++------------- > testsuite/.gitignore | 1 + > testsuite/README.md | 21 +--- > testsuite/build_test/build_test.py | 219 ++++++++++++++++++++++++++++++--- > testsuite/build_test/cibase.py | 116 +++++++++++++++++ > testsuite/build_test/cibuilder.py | 132 ++++++++++++++++++++ > testsuite/build_test/run.sh | 3 - > testsuite/build_test/run_fast.sh | 3 + > testsuite/build_test/run_full.sh | 3 + > testsuite/build_test/variant.yaml | 22 ---- > testsuite/start_vm.py | 27 ++-- > testsuite/vm_boot_test/run.sh | 3 - > testsuite/vm_boot_test/run_fast.sh | 3 + > testsuite/vm_boot_test/run_full.sh | 3 + > testsuite/vm_boot_test/variant.yaml | 22 ---- > testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++++++++--- > 18 files changed, 641 insertions(+), 391 deletions(-) > create mode 100644 testsuite/.gitignore > create mode 100644 testsuite/build_test/cibase.py > create mode 100644 testsuite/build_test/cibuilder.py > delete mode 100755 testsuite/build_test/run.sh > create mode 100755 testsuite/build_test/run_fast.sh > create mode 100755 testsuite/build_test/run_full.sh > delete mode 100644 testsuite/build_test/variant.yaml > delete mode 100755 testsuite/vm_boot_test/run.sh > create mode 100755 testsuite/vm_boot_test/run_fast.sh > create mode 100755 testsuite/vm_boot_test/run_full.sh > delete mode 100644 testsuite/vm_boot_test/variant.yaml > Didn't you once installed avocado for execution in CI? This is no longer happening - unless I missed something - and, thus, broke gitlab-ci. Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux