From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7008446828784910336 X-Received: by 2002:a5d:648b:: with SMTP id o11mr7458823wri.315.1633020347244; Thu, 30 Sep 2021 09:45:47 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:c782:: with SMTP id l2ls9058947wrg.2.gmail; Thu, 30 Sep 2021 09:45:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxO4KZkxTMQAQTGuHfyE61RFns2Ybs81jn/2h34m6RTHOZdZVvUpPFTJKOWZBaptwYITGeX X-Received: by 2002:a05:6000:144e:: with SMTP id v14mr6113381wrx.228.1633020346340; Thu, 30 Sep 2021 09:45:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1633020346; cv=none; d=google.com; s=arc-20160816; b=InJEgo/grlbFp0fVQRyjk5q/MXKsKFc1N8zQ28pJs10fN0CST5ft36/l+Jd1p4/lpP 2Q1F79RpA3dttikdVTJDX+zKpU2nKdJ0/xM2CLVB1q8zsL7vF1YbxVrgnFJnYWc8LGbl EOr/TnZkINRZtFyAl+/7lmFXNMBNbpSzhGsBA+YX0o3GAUFsxOAoVXFelRz0dwgdedsJ xGkp5DTa+PQX3Vt3Vkdkjua8iRzYxJj6lM9oXb+4T3mqJ7mf+rsxgw95UP8cvuqXCeYn CNGKUVStzqMaH115DCTZdw3DI9NN+my7M70ubV4O70aVH7RjquheBT94DVCFd6nHVPbV QWQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=0+8lFzNR19DQWPHK/k884KssWDuzE/p+orzM6/iNrRU=; b=TU0SH9oUz9qBPstP/avxBFzidkGCdiZkd2TOSmNsrE9jdH20fZ/o0c08K8IjKUv4kW IueMFdhZdTyN4jQ34c9Z6tYtyISkkuTNIZHiOlfFYzzOZO265bolOEZhCi88kPh1Q50h vIvThd69HmjDoSMA2Uj3uJRbHZEUIbQIZMFoBHyE9QkimZtNpL1PO7KTgMBSTVfVDZW8 3rNHannJh54C0L6eCMMXzH5NvCjtizpc4niSqd6r7cHjtwHRHK5H6ZIpNeAjJV6IevMw 5tjk6FxQarymBGN9MK7HHtjpy9o4Js8BXbjEHsCuGBK25v9cXVVWGdx0FlfQVtVw+6i/ Y2qQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=softfail (google.com: domain of transitioning anton.mikanovich@promwad.com does not designate 85.214.156.166 as permitted sender) smtp.mailfrom=anton.mikanovich@promwad.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=promwad.com Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id i66si123493wma.0.2021.09.30.09.45.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 30 Sep 2021 09:45:45 -0700 (PDT) Received-SPF: softfail (google.com: domain of transitioning anton.mikanovich@promwad.com does not designate 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=softfail (google.com: domain of transitioning anton.mikanovich@promwad.com does not designate 85.214.156.166 as permitted sender) smtp.mailfrom=anton.mikanovich@promwad.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=promwad.com Received: from ws-323.promwad.corp (leased-line-st-87-252-252-30.telecom.by [87.252.252.30]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 18UGjhuY018892 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA256 bits=128 verify=NOT); Thu, 30 Sep 2021 18:45:43 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v14 00/16] Update Avocado testsuite Date: Thu, 30 Sep 2021 19:45:25 +0300 Message-Id: <1633020341-26062-1-git-send-email-anton.mikanovich@promwad.com> X-Mailer: git-send-email 2.7.4 X-TUID: WZeinLuBWbD8 This patchset provides implementation of build and vm_start tests functionality in Python with help of Avocado framework. Scripts ci_build and vm_smoke_test are rewritten to call Avocado. Build and run logs are hidden by default now until an error occurs. Resending v14 because v13 was just a copy of v12 by mistake. --- Changes since v13: - Remove legacy scripts. Changes since v12: - No changes, wrong patchset. Changes since v11: - Rebase on next. - Init environment in vm_smoke_test. Changes since v10: - Rebase on next. - Add DISTRO_APT_PREMIRRORS usage. - Remove avocado deb package install. - Set default runner. Changes since v9: - Rebase on next. - Fix avocado artifacts path. Changes since v8: - Remove duplication in logs. - Disable sysinfo plugin. - Show full errors in default mode. Changes since v7: - Remove code duplication. - Fix build_dir parsing. - Add missed protect files backup logic. Changes since v6: - Improve environment parsing. - Protect files backup logic. - Rebuild full log output option. - Protect ubuntu target with KFAIL. Changes since v5: - Rebase on current next. Changes since v4: - Add container tests. - Implement testcase skipping logic. - Separate and protect Bullseye tests. - Add checks for container dependencies. - Add Ilbers repo key. - Improve error handling. - Fix Ubuntu qemu image name. Changes since v3: - Add automatic bitbake init. - Fix line lengths. - Fix build without repro. - Fix build artifact paths. - Prepare for Gitlab build. - Change debug settings. - Move log splitting to start_vm. - Add deb package install. Changes since v2: - Make CI call Avocado tests. - Wait less on vm_start test. - Split Roman's vm_start commit. - Put logs to build folder. - Rearrange patches. - Split build test to classes. - Add test case tagging. - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. - Get rid of python-subprocess32. - Improve logging. Changes since v1: - Merge start_vm rebuild patches. - Fix patch comments. Anton Mikanovich (13): start_vm.py: Fix ubuntu image name vm_boot_test: Fix log file path in vm_boot_test vm_boot_test: Remove external varianter vm_boot_test: Improve QEMU images checking build_test: Refactoring build tests cases testsuite: Add Python generations for testsuite in gitignore testsuite: Fix test suite prepare guide gitlab-ci: Add Avocado build artifacts gitlab-ci: Add debug flag vm_boot_test: Add automatic bitbake init build_test: Protect ubuntu target with KFAIL ci_build: Migrate to Avocado vm_smoke_test: Migrate to Avocado Roman Pletnev (3): start_vm.py: Fix target name handling start_vm.py: Add output and PID file vm_start.py options start_vm.py: Add MIPS support .gitlab-ci.yml | 5 +- scripts/ci_build.sh | 208 ++++--------------------------- scripts/vm_smoke_test | 120 +++++------------- testsuite/.gitignore | 1 + testsuite/README.md | 21 +--- testsuite/build_test/build_test.py | 219 ++++++++++++++++++++++++++++++--- testsuite/build_test/cibase.py | 116 +++++++++++++++++ testsuite/build_test/cibuilder.py | 132 ++++++++++++++++++++ testsuite/build_test/run.sh | 3 - testsuite/build_test/run_fast.sh | 3 + testsuite/build_test/run_full.sh | 3 + testsuite/build_test/variant.yaml | 22 ---- testsuite/start_vm.py | 27 ++-- testsuite/vm_boot_test/run.sh | 3 - testsuite/vm_boot_test/run_fast.sh | 3 + testsuite/vm_boot_test/run_full.sh | 3 + testsuite/vm_boot_test/variant.yaml | 22 ---- testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++++++++--- 18 files changed, 641 insertions(+), 391 deletions(-) create mode 100644 testsuite/.gitignore create mode 100644 testsuite/build_test/cibase.py create mode 100644 testsuite/build_test/cibuilder.py delete mode 100755 testsuite/build_test/run.sh create mode 100755 testsuite/build_test/run_fast.sh create mode 100755 testsuite/build_test/run_full.sh delete mode 100644 testsuite/build_test/variant.yaml delete mode 100755 testsuite/vm_boot_test/run.sh create mode 100755 testsuite/vm_boot_test/run_fast.sh create mode 100755 testsuite/vm_boot_test/run_full.sh delete mode 100644 testsuite/vm_boot_test/variant.yaml -- 2.7.4