From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6953642757724307456 X-Received: by 2002:a2e:7804:: with SMTP id t4mr18895485ljc.99.1619021119290; Wed, 21 Apr 2021 09:05:19 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:6c2:: with SMTP id u2ls2075189lff.3.gmail; Wed, 21 Apr 2021 09:05:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxUESavJSGi0nBLbK192IB+8tqPvg9UpphZZdxj9u8KEiPHUi12uCGrWbua9B9cOsOGyJ0Q X-Received: by 2002:ac2:5149:: with SMTP id q9mr19512176lfd.361.1619021117992; Wed, 21 Apr 2021 09:05:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619021117; cv=none; d=google.com; s=arc-20160816; b=Gn6VcEHs79EHSlaHvv8tLyogJuHYE5QpPbCq6khDEU3p007vgd2l20lIZ9K7eP87sF 9DFFSE4Tl6tTLX7NOcMZXWwzYq2ewYrQYW+TjtsCcghJADqmUt/YNSIdfK6erdnO0+kx xZcBFiwd/d+zgwXb5zCVX4oEoN840i6/27RS9kmsOyBdrjeXL4cQlivxdH15f+FJDW0O dD5jAGIVFQNuAutyoVNqQa6m3ITxrE6lcDlGybf+DTBaiqDY4bm1A2Tkae6CrF2SQ5QK MqbYjst01p7vvseyjkI5J9ncL5PRCPfUoBxarMTpkdNXnid6ZtlynuHZz9yh6OdfrZqU QdHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=PJd2sLDOBmERKzfv0IEUIOssJpeyH6fykM+QffKJ928=; b=qe7U2HWvZFFLhIBtfbavkmibIz74WQAGZY8vKvq6xccAxAJm9ZqcSlJ1EbqFSZDFgV o1RFv9cfTRUNmKhtXN+UkBc9S4N84vdel/DqJxFrcBds1fzxtu3WBVJuzipjTunhYXww 0p4Y5brdSzsEayzlOd5wbXiMfhKwdrenbLEH7tcSA1ht8vjr9nFFvdfs0gLyaijW5l03 S/AL5xl9MrLoKQmcCiJRmxxmSylF5/xsDucXYpHFSv5Ye9MFocNQoDK6Cd09/uT1XTmg Ez/MVI8r61MU5F9Qew2uvy8TkIAAYFIIM2Q8+BXneGSnUtpbzs3F9ludPMjX+eev9C+R 6F6Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id u11si194616lji.3.2021.04.21.09.05.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 21 Apr 2021 09:05:17 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from localhost.localdomain (mm-153-39-214-37.mgts.dynamic.pppoe.byfly.by [37.214.39.153] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 13LG5C70027472 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 21 Apr 2021 18:05:15 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v4 00/14] Update Avocado testsuite Date: Wed, 21 Apr 2021 19:04:51 +0300 Message-Id: <20210421160505.67194-1-amikan@ilbers.de> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: TmFi2dS4B8zC This patchset provides implementation of build and vm_start tests functionality in Python with help of Avocado framework. Scripts ci_build and vm_smoke_test are rewritten to call Avocado. ci_build and vm_smoke_test are left for compatibility and should be replaced by direct avocado calls later. Original ci_build and vm_smoke_test are renamed to *_legacy. Build and run logs are hidden now until an error occurs. --- Changes since v3: - Add automatic bitbake init. - Fix line lengths. - Fix build without repro. - Fix build artifact paths. - Prepare for Gitlab build. - Change debug settings. - Move log splitting to start_vm. - Add deb package install. Changes since v2: - Make CI call Avocado tests. - Wait less on vm_start test. - Split Roman's vm_start commit. - Put logs to build folder. - Rearrange patches. - Split build test to classes. - Add test case tagging. - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. - Get rid of python-subprocess32. - Improve logging. Changes since v1: - Merge start_vm rebuild patches. - Fix patch comments. Anton Mikanovich (11): vm_boot_test: Fix log file path in vm_boot_test vm_boot_test: Remove external varianter vm_boot_test: Improve QEMU images checking build_test: Refactoring build tests cases testsuite: Add Python generations for testsuite in gitignore testsuite: Fix test suite prepare guide gitlab-ci: Add Avocado build artifacts vm_boot_test: Add automatic bitbake init ci_build: Migrate to Avocado vm_smoke_test: Migrate to Avocado ci_build: Install Avocado if needed Roman Pletnev (3): start_vm.py: Fix target name handling start_vm.py: Add output and PID file vm_start.py options start_vm.py: Add MIPS support .gitlab-ci.yml | 1 + scripts/ci_build.sh | 215 ++++--------------- scripts/ci_build_legacy.sh | 275 +++++++++++++++++++++++++ scripts/vm_smoke_test | 110 ++-------- scripts/vm_smoke_test_legacy | 168 +++++++++++++++ testsuite/.gitignore | 1 + testsuite/README.md | 21 +- testsuite/build_test/build_test.py | 137 ++++++++++-- testsuite/build_test/cibase.py | 122 +++++++++++ testsuite/build_test/cibuilder.py | 80 +++++++ testsuite/build_test/run.sh | 3 - testsuite/build_test/run_fast.sh | 3 + testsuite/build_test/run_full.sh | 3 + testsuite/build_test/variant.yaml | 22 -- testsuite/start_vm.py | 22 +- testsuite/vm_boot_test/run.sh | 3 - testsuite/vm_boot_test/run_fast.sh | 3 + testsuite/vm_boot_test/run_full.sh | 3 + testsuite/vm_boot_test/variant.yaml | 22 -- testsuite/vm_boot_test/vm_boot_test.py | 120 +++++++++-- 20 files changed, 955 insertions(+), 379 deletions(-) create mode 100755 scripts/ci_build_legacy.sh create mode 100755 scripts/vm_smoke_test_legacy create mode 100644 testsuite/.gitignore create mode 100644 testsuite/build_test/cibase.py create mode 100644 testsuite/build_test/cibuilder.py delete mode 100755 testsuite/build_test/run.sh create mode 100755 testsuite/build_test/run_fast.sh create mode 100755 testsuite/build_test/run_full.sh delete mode 100644 testsuite/build_test/variant.yaml delete mode 100755 testsuite/vm_boot_test/run.sh create mode 100755 testsuite/vm_boot_test/run_fast.sh create mode 100755 testsuite/vm_boot_test/run_full.sh delete mode 100644 testsuite/vm_boot_test/variant.yaml -- 2.25.1