From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6969105759827132416 X-Received: by 2002:a05:6512:3b2a:: with SMTP id f42mr14541980lfv.425.1624287430040; Mon, 21 Jun 2021 07:57:10 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3b99:: with SMTP id g25ls110902lfv.2.gmail; Mon, 21 Jun 2021 07:57:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyDfMzQ/OWhWMVCIDLSzpbIzS/gp0u7kVOCBmgrPgfh/qRy6wDFteOW7Etwcb4I4W/WF/2g X-Received: by 2002:a05:6512:3a4:: with SMTP id v4mr15070385lfp.127.1624287428900; Mon, 21 Jun 2021 07:57:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1624287428; cv=none; d=google.com; s=arc-20160816; b=uUEH4A1FSEKq6w6l+FJ1vntDWt1ChPEQUqLuOi/GZ2iegl8hTgwdXW+01obgb/+02X 4NBOJZ5nuEVpRX3xixbsQVqKmPQ5H2SNfPIsxBy/ZvLQfVx0qRWN0uoCfgvCFyAIcBHY 6bjtqy5D2I1Rv3Myig7ud3JsH7OY8H1WsklMygMMarabkK5+hzEmQ3u6/2GaJt4hEDrs j3qq+iw6XPsf59jr+cQjt57NRCvX3oy0m6x5qACWPQxDNk+jJAvM1G+8uoNb6+tBBC1i oVfKMjYzM62mfFCZ3i1jUMtMOQ9s+EVSIn28UOBrfh6NKRJX6nVX+d4h6k5a7DdslBoE GO2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:to:from:subject; bh=WuK2P9ESJaSmTfmDfwbPUMpkO6eKi+niLe/k2gZvFO4=; b=gRSDkac/FZJI8/f9mtyMbWfAMTyOl1D3Z9mb9d2qlVeDSLUTRhqho849B2QjVV6yN6 Y2cXO99HyQocc4ZRj91tXeVTFtzwHq4Zwa26/HTIJ5Go+BRZp27OGmL0ZCS0skm1VYOB tIsAInzmCQBDVTg7rt0QUgSJI2O06hqa7OQc+aeg1d4s6iZdMcALqDpJ4yu4yM8S2jdd 6d9kT6ly5u12VeHYBMMPWeDbFrTq0kQVjPiB2yVFfWarH4DtEfcVqpnurl+nLwiPvNhT FQxf2EofZPvd1t3jwdC3ofd5mhKpK5GoN5gflPCGqcK2EtmbIqtDwZtPwsu42+S0sym9 IjpQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id z16si363379lfq.13.2021.06.21.07.57.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jun 2021 07:57:07 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 15LEv7lm028593 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 21 Jun 2021 16:57:07 +0200 Received: from [167.87.243.122] ([167.87.243.122]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 15LEv6ra018017; Mon, 21 Jun 2021 16:57:06 +0200 Subject: Re: [PATCH v8 00/17] Update Avocado testsuite From: Jan Kiszka To: Anton Mikanovich , isar-users@googlegroups.com References: <20210602080922.16435-1-amikan@ilbers.de> Message-ID: Date: Mon, 21 Jun 2021 16:57:06 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: Wk4ddG/0f2pk On 21.06.21 08:49, Jan Kiszka wrote: > On 02.06.21 10:09, Anton Mikanovich wrote: >> This patchset provides implementation of build and vm_start tests >> functionality in Python with help of Avocado framework. >> Scripts ci_build and vm_smoke_test are rewritten to call Avocado. >> ci_build and vm_smoke_test are left for compatibility and >> should be replaced by direct avocado calls later. >> Original ci_build and vm_smoke_test are renamed to *_legacy. >> Build and run logs are hidden now until an error occurs. >> >> --- >> Changes since v7: >> - Remove code duplication. >> - Fix build_dir parsing. >> - Add missed protect files backup logic. >> Changes since v6: >> - Improve environment parsing. >> - Protect files backup logic. >> - Rebuild full log output option. >> - Protect ubuntu target with KFAIL. >> Changes since v5: >> - Rebase on current next. >> Changes since v4: >> - Add container tests. >> - Implement testcase skipping logic. >> - Separate and protect Bullseye tests. >> - Add checks for container dependencies. >> - Add Ilbers repo key. >> - Improve error handling. >> - Fix Ubuntu qemu image name. >> Changes since v3: >> - Add automatic bitbake init. >> - Fix line lengths. >> - Fix build without repro. >> - Fix build artifact paths. >> - Prepare for Gitlab build. >> - Change debug settings. >> - Move log splitting to start_vm. >> - Add deb package install. >> Changes since v2: >> - Make CI call Avocado tests. >> - Wait less on vm_start test. >> - Split Roman's vm_start commit. >> - Put logs to build folder. >> - Rearrange patches. >> - Split build test to classes. >> - Add test case tagging. >> - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. >> - Get rid of python-subprocess32. >> - Improve logging. >> Changes since v1: >> - Merge start_vm rebuild patches. >> - Fix patch comments. >> >> Anton Mikanovich (14): >> start_vm.py: Fix ubuntu image name >> vm_boot_test: Fix log file path in vm_boot_test >> vm_boot_test: Remove external varianter >> vm_boot_test: Improve QEMU images checking >> build_test: Refactoring build tests cases >> testsuite: Add Python generations for testsuite in gitignore >> testsuite: Fix test suite prepare guide >> gitlab-ci: Add Avocado build artifacts >> gitlab-ci: Add debug flag >> vm_boot_test: Add automatic bitbake init >> build_test: Protect ubuntu target with KFAIL >> ci_build: Migrate to Avocado >> vm_smoke_test: Migrate to Avocado >> ci_build: Install Avocado if needed >> >> Roman Pletnev (3): >> start_vm.py: Fix target name handling >> start_vm.py: Add output and PID file vm_start.py options >> start_vm.py: Add MIPS support >> >> .gitlab-ci.yml | 5 +- >> scripts/ci_build.sh | 213 ++++---------------- >> scripts/ci_build_legacy.sh | 267 +++++++++++++++++++++++++ >> scripts/vm_smoke_test | 110 ++-------- >> scripts/vm_smoke_test_legacy | 168 ++++++++++++++++ >> testsuite/.gitignore | 1 + >> testsuite/README.md | 21 +- >> testsuite/build_test/build_test.py | 217 ++++++++++++++++++-- >> testsuite/build_test/cibase.py | 110 ++++++++++ >> testsuite/build_test/cibuilder.py | 103 ++++++++++ >> testsuite/build_test/run.sh | 3 - >> testsuite/build_test/run_fast.sh | 3 + >> testsuite/build_test/run_full.sh | 3 + >> testsuite/build_test/variant.yaml | 22 -- >> testsuite/start_vm.py | 27 ++- >> testsuite/vm_boot_test/run.sh | 3 - >> testsuite/vm_boot_test/run_fast.sh | 3 + >> testsuite/vm_boot_test/run_full.sh | 3 + >> testsuite/vm_boot_test/variant.yaml | 22 -- >> testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++-- >> 20 files changed, 1043 insertions(+), 382 deletions(-) >> create mode 100755 scripts/ci_build_legacy.sh >> create mode 100755 scripts/vm_smoke_test_legacy >> create mode 100644 testsuite/.gitignore >> create mode 100644 testsuite/build_test/cibase.py >> create mode 100644 testsuite/build_test/cibuilder.py >> delete mode 100755 testsuite/build_test/run.sh >> create mode 100755 testsuite/build_test/run_fast.sh >> create mode 100755 testsuite/build_test/run_full.sh >> delete mode 100644 testsuite/build_test/variant.yaml >> delete mode 100755 testsuite/vm_boot_test/run.sh >> create mode 100755 testsuite/vm_boot_test/run_fast.sh >> create mode 100755 testsuite/vm_boot_test/run_full.sh >> delete mode 100644 testsuite/vm_boot_test/variant.yaml >> > > I tried that revision in our CI, but all attempts are failing, > apparently due to connectivity issues. That would be something to blame > on our infrastructure - if current next wouldn't pass... Retrying once > again. Seems there to be some changing fetch patterns, and that upset our local proxies once again. Isar is not to blaim for the proxy mess. I was shortly confused that do_rootfs_postprocess is always failing with fetches, but those are related to cache_deb_src pulling all related sources. Wasn't that tested so far, and therefore current next didn't trigger that for us? Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux