From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6990633379674718208 X-Received: by 2002:a5d:518a:: with SMTP id k10mr4947473wrv.400.1628785185651; Thu, 12 Aug 2021 09:19:45 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:db90:: with SMTP id u16ls1727683wri.2.gmail; Thu, 12 Aug 2021 09:19:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzYgAvQv7OdQSsKDv5PIyJ5gq/8Y92GC/vQInL5j4RUGxyjn/g+FswhaDeU2nvdnQRnZrZa X-Received: by 2002:adf:cd8e:: with SMTP id q14mr2158709wrj.410.1628785184616; Thu, 12 Aug 2021 09:19:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628785184; cv=none; d=google.com; s=arc-20160816; b=qCadJmxB1UfwVkZOfYoKgHvtdVq+rf7Il/0enHJG4Zt7/3lWtpkyAG6Cul/58cgD4I D4MVLdjnFZ3FSZ6ynDmtBeC8EGtd/culBveGHaLhpT9z0/tnZU1Sbr7cXP2nE/yrytU3 zgy1meMr8Jh8SPdwp6YqhRKGa0tLscMPlObTR/09dhQcju8icizTr7VyvjSj2B8k4L8h +xsWHvcbn0ZRqMYOKZnnF85FFFhKWZH5Hev+QSGjYylu7YHJe8++VSB3mXIy3gR5b12Y MkCR4T6Q3F9s4t3fp5/owkMYyTmuoCHnA23TiupQQd3FfbpnUa0YlEplV8nBrd4nU1S7 QKvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=D++VihVkc/I/xLQmcov5WXWjcL2QeNO1EpMbpGx+SNo=; b=zyIJtOZ/rlJd0x9njci6urgu8P2bkz8RSWCEPemd5Ofr1mplBjHeeFA9++RPxuSOm9 9gOWpo7Stl4KbIohYpfXcKNNY5r3GrNVyCFrCNoUozE+u1KgPdBqe2zgXc7JC2NMqrpo cq0dlJm7QZmOR9MpQGstSyn66MHaoR38k0DSXvbgpRvpFMDXxfeC/CwVtBCHcaYzDAFT /ShAVZVPddYe39USoCIwX4TTzZfjcDgh+lJVUWZIam2LAc756mMLR3fncNQ2ZTyETnmb gnQXrgey2EjM3TyV45Jvk3sy1gO9jPM0Z0AKR3wwmPR4taRK01Z2UCIXNsc6yhtoMwLn /A2A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id 15si550812wma.1.2021.08.12.09.19.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Aug 2021 09:19:44 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 17CGJiCk020545 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 12 Aug 2021 18:19:44 +0200 Received: from md1za8fc.ad001.siemens.net ([139.25.0.59]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 17CGJh6M017107; Thu, 12 Aug 2021 18:19:43 +0200 Date: Thu, 12 Aug 2021 18:19:42 +0200 From: Henning Schild To: Anton Mikanovich Cc: isar-users@googlegroups.com Subject: Re: [PATCH v10 00/17] Update Avocado testsuite Message-ID: <20210812181942.25ae9204@md1za8fc.ad001.siemens.net> In-Reply-To: <20210730082739.15042-1-amikan@ilbers.de> References: <20210730082739.15042-1-amikan@ilbers.de> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: GKSjGXnIUAsL I was only playing with what we currently have in next. But that seems to serialize the builds listed in that yaml. While before bitbake did run them all at once. I think the tests from that yaml should run in parallel to cause the same kind of stress ... just parallel in the testsuite instead of bitbake. And to hopefully speed things up a bit. Henning Am Fri, 30 Jul 2021 11:27:22 +0300 schrieb Anton Mikanovich : > This patchset provides implementation of build and vm_start tests > functionality in Python with help of Avocado framework. > Scripts ci_build and vm_smoke_test are rewritten to call Avocado. > ci_build and vm_smoke_test are left for compatibility and > should be replaced by direct avocado calls later. > Original ci_build and vm_smoke_test are renamed to *_legacy. > Build and run logs are hidden now until an error occurs. > > --- > Changes since v9: > - Rebase on next. > - Fix avocado artifacts path. > Changes since v8: > - Remove duplication in logs. > - Disable sysinfo plugin. > - Show full errors in default mode. > Changes since v7: > - Remove code duplication. > - Fix build_dir parsing. > - Add missed protect files backup logic. > Changes since v6: > - Improve environment parsing. > - Protect files backup logic. > - Rebuild full log output option. > - Protect ubuntu target with KFAIL. > Changes since v5: > - Rebase on current next. > Changes since v4: > - Add container tests. > - Implement testcase skipping logic. > - Separate and protect Bullseye tests. > - Add checks for container dependencies. > - Add Ilbers repo key. > - Improve error handling. > - Fix Ubuntu qemu image name. > Changes since v3: > - Add automatic bitbake init. > - Fix line lengths. > - Fix build without repro. > - Fix build artifact paths. > - Prepare for Gitlab build. > - Change debug settings. > - Move log splitting to start_vm. > - Add deb package install. > Changes since v2: > - Make CI call Avocado tests. > - Wait less on vm_start test. > - Split Roman's vm_start commit. > - Put logs to build folder. > - Rearrange patches. > - Split build test to classes. > - Add test case tagging. > - Get rid of avocado-framework-plugin-varianter-yaml-to-mux. > - Get rid of python-subprocess32. > - Improve logging. > Changes since v1: > - Merge start_vm rebuild patches. > - Fix patch comments. > > Anton Mikanovich (14): > start_vm.py: Fix ubuntu image name > vm_boot_test: Fix log file path in vm_boot_test > vm_boot_test: Remove external varianter > vm_boot_test: Improve QEMU images checking > build_test: Refactoring build tests cases > testsuite: Add Python generations for testsuite in gitignore > testsuite: Fix test suite prepare guide > gitlab-ci: Add Avocado build artifacts > gitlab-ci: Add debug flag > vm_boot_test: Add automatic bitbake init > build_test: Protect ubuntu target with KFAIL > ci_build: Migrate to Avocado > vm_smoke_test: Migrate to Avocado > ci_build: Install Avocado if needed > > Roman Pletnev (3): > start_vm.py: Fix target name handling > start_vm.py: Add output and PID file vm_start.py options > start_vm.py: Add MIPS support > > .gitlab-ci.yml | 5 +- > scripts/ci_build.sh | 213 ++++---------------- > scripts/ci_build_legacy.sh | 267 > +++++++++++++++++++++++++ scripts/vm_smoke_test | > 110 ++-------- scripts/vm_smoke_test_legacy | 168 > ++++++++++++++++ testsuite/.gitignore | 1 + > testsuite/README.md | 21 +- > testsuite/build_test/build_test.py | 217 ++++++++++++++++++-- > testsuite/build_test/cibase.py | 110 ++++++++++ > testsuite/build_test/cibuilder.py | 130 ++++++++++++ > testsuite/build_test/run.sh | 3 - > testsuite/build_test/run_fast.sh | 3 + > testsuite/build_test/run_full.sh | 3 + > testsuite/build_test/variant.yaml | 22 -- > testsuite/start_vm.py | 27 ++- > testsuite/vm_boot_test/run.sh | 3 - > testsuite/vm_boot_test/run_fast.sh | 3 + > testsuite/vm_boot_test/run_full.sh | 3 + > testsuite/vm_boot_test/variant.yaml | 22 -- > testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++-- > 20 files changed, 1070 insertions(+), 382 deletions(-) > create mode 100755 scripts/ci_build_legacy.sh > create mode 100755 scripts/vm_smoke_test_legacy > create mode 100644 testsuite/.gitignore > create mode 100644 testsuite/build_test/cibase.py > create mode 100644 testsuite/build_test/cibuilder.py > delete mode 100755 testsuite/build_test/run.sh > create mode 100755 testsuite/build_test/run_fast.sh > create mode 100755 testsuite/build_test/run_full.sh > delete mode 100644 testsuite/build_test/variant.yaml > delete mode 100755 testsuite/vm_boot_test/run.sh > create mode 100755 testsuite/vm_boot_test/run_fast.sh > create mode 100755 testsuite/vm_boot_test/run_full.sh > delete mode 100644 testsuite/vm_boot_test/variant.yaml >