From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 23 Jan 2026 09:25:10 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-ed1-f63.google.com (mail-ed1-f63.google.com [209.85.208.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 60N8PAsT015025 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 23 Jan 2026 09:25:10 +0100 Received: by mail-ed1-f63.google.com with SMTP id 4fb4d7f45d1cf-64d1982d980sf2098209a12.2 for ; Fri, 23 Jan 2026 00:25:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1769156705; cv=pass; d=google.com; s=arc-20240605; b=QjLZ6FhQBPdrieuKma91LEboFDU0O6bi1tRX9SU8NUY3ewxWPM7tQnCYboc71GNRaf AgRy7Dy4EsAiU/KFYDgEZwyHFPQIgREWmFvSwzKCQBbenntl7JD9HMLjp7amy/WUHQxw 32I1IlprviphcfnTwqJqlTLYDGesABaV0p76jNDp2LArc13AuXIEQqHj7lIKzeMXIbVp RQuYX2y111nNu3xwtJesokhYqFOLdm2rlimtrNlQ5/dRzWwh9bSkDpkWOuBc0HQDmvK/ XlTQvXuUbgAunoDjB7sKyr2SA8lsbcd+eupi6QSECxdMKm+gqWyBy0o5YZWethcrNZdW ItBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=S5Dwq33GqBKe9xxsFlDOevCoo4cuEyntzeizPHTm85M=; fh=QboiLjVj2PKb3H6HhZkwjx45q+96EiGHHrL0nkCstlA=; b=ANU+uDdkVQUvz8O2ceL7h6zGJMgV2J+pQ7WR89TAfx8/NDIMh2+14gB3M/cW3IMT4e 7hdtw/qaA+oPsil2pXiCxhTjjR1qn0UeA4H9mu8LY6ok77Qi4aWuiwostipcsr+gg0Bn jHufGPe7EfTzMG+cnZI/8umKSS3ngnAdLRL8A37WXg4IiYUO7AdMnD5wzBBq/nzkul9/ 5iyUbxXxPwZqh8lwEhAivOfpjpNg8HcfAkLJGX9WwWwI1I44/XSNGr02T7BUYiOODAMf lPIHu17mJRz9HtbGQEPcGFA7vdf3gEfZwc4Fl/a8NcL3MwLmaCax1IaUGZ4mYw/hwgGO 3/Mw==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1769156705; x=1769761505; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=S5Dwq33GqBKe9xxsFlDOevCoo4cuEyntzeizPHTm85M=; b=hjUqj1R4dDUofh+XWy5XOimfGIzIA2uAl/DxbT7m8RQV58CiFUv2HovKLouIW0ZQVO K47bZeolBwJYtgqcJi3T8ipDri8BeMXUfsDEMuZhHO1YXm9kZeOCdn9tFHUyLkM8Fx3W 1Yx2iZ8+pUmeTmj+Mg1c86ZGMkxPVdqi1GtAJingEsnyd4aiFhoanphzqnR3R17mo8lN YE5VLDBLeflU7SVFN3VL769yxfoTqMzSsiYxXYfX1MIWUMkq2RkA0XcAJJEu3h/NFBsD u/RwLYN7glT5OHGOe3sKViWIf//OxZRMUKIZEpX6vdidzRxaQdzClmMZFkjrQHSqXlqC YrJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156705; x=1769761505; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=S5Dwq33GqBKe9xxsFlDOevCoo4cuEyntzeizPHTm85M=; b=gIImekUcm9fxJ8wG729jOnOsEksyNbmG42lA8TOBxzQ7z5LKtknFKjme4i1qthDuqz Zrq86KGfg7HdH4uXhTMaluc1rdYEmTwzAhKN05G2CRcdDe7m23ZvcDExwilr+BTCqNvS dCgIMN2rd92C+Go5tERT9l2AvRn7vgXJKivdE3nO1mNZy1DuxePzsVfePD2QzrtdJLf/ 70oMRayMncmsXNczRVHu611/k77N+O0RF3bzjnhPfwxMdw/UhbCkZ1WIArdTOApXIEHc SSxqhdWckuj5+UNi60sVUZAqZUs+BmVEa04hiGmeA9JkcppF4ZzoVbCE7MNEWrIKEIdt eJTA== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUFkm+g3ECJLY/+UgQYEpvAjFSfeH91RSqqVn0zx6bVTxA949Ya+0iGAAXLpEJ4dVlRJaAM@ilbers.de X-Gm-Message-State: AOJu0YxT1PFnlxYikmOCBciqeWR3Irk2ztl4MpaYV/IOfHLKM2pc2N/p YkwsV24GDT0o6qIAyvsJHCJbVzs9WPqsc3VEfekiekoLoqJIHFafU9/W X-Received: by 2002:a05:6402:1448:b0:64b:73d5:e2b4 with SMTP id 4fb4d7f45d1cf-6584874be52mr1563696a12.5.1769156704459; Fri, 23 Jan 2026 00:25:04 -0800 (PST) X-BeenThere: isar-users@googlegroups.com; h="AV1CL+F97IBhmENDALP2xePZS/HqHrG9ws969yDEDUb9KGdq0A==" Received: by 2002:aa7:d14b:0:b0:64b:9695:8dac with SMTP id 4fb4d7f45d1cf-65832d93590ls1321952a12.2.-pod-prod-05-eu; Fri, 23 Jan 2026 00:25:02 -0800 (PST) X-Received: by 2002:a17:907:7284:b0:b76:6020:ed2b with SMTP id a640c23a62f3a-b885ae0bb1amr130681166b.45.1769156702190; Fri, 23 Jan 2026 00:25:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1769156702; cv=none; d=google.com; s=arc-20240605; b=KyKup+XOtwoVOzhertnkaF98IFPoHTPKZ6yhWMphEtO2ijVoBOa40/CCrT6yYODne+ zyP6CYVW/27gf/Is2c29cCWhJdKAYSVMMMk0j1WJ5r9naqMJksVqQWEDUprc2306XKVh yA261ja+sVZqquBq/SiR2VyEMQn1An5ZswQLowLdicB2DkZgm5hyqRS9sRwPLPozrHwT SJnXcLKVzV0jAt1sV+5VRWtMWC2JAOp2JejANST1B3vjqgWVJLw8RcQVKUWtc6ddW33Z tBe/p/7pzpip8cRJnH52NT7z76DB7QwmVYDiCCXlQG0orHtDkn4Kg5sVOcOTGhNtvfoe i5xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=aUnHsv8L+y4zUusV0+1gFMChveMmNXsgK5OERmbuoAA=; fh=V42d4Y/lNDOgZ/9/qsphFBeZCU4sz3YugfOOjBc+1H8=; b=Lre6L7VTEvztsqdUSnAodJNZzZL9WXd+3KjL12vHtD/wP1CquJaUln6FNZs19tbHtU iTn5TUIl821SdJUF2NIuMQmZWak5hXkjMYt2Q49sd2U1PKtUQEt/WyaBAi07qnlXzY9U JYYWRb3E4QNqf5cVnc76yF7SPG+eY8WWklODhdL+xtuSfTqBfNzC083XH2n9UkcAiS1x 0iV6yPob3oqlN3gTx4mZvb2rLiZswm5xcN5885HWnIGIo517+SXqFIb3ARRrPlcskYkQ QsgAPcW16hu6ezydBhSzVQ0NgjCQV9U7V+by3grpUDpV6FfqpofxZQyuUT6dgoo4PkX8 UFOw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id a640c23a62f3a-b885b67f781si4041066b.2.2026.01.23.00.25.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Jan 2026 00:25:02 -0800 (PST) Received-SPF: pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from debian-zwei.m.ilbers.de ([88.130.203.42]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPA id 60N8P1Xf014977; Fri, 23 Jan 2026 09:25:01 +0100 From: Zhihang Wei To: isar-users@googlegroups.com, felix.moessbauer@siemens.com, amikan@ilbers.de Cc: wzh@ilbers.de, cedric.hombourger@siemens.com Subject: [PATCH v3 00/20] Various improvements to the testsuite Date: Fri, 23 Jan 2026 09:24:41 +0100 Message-Id: <20260123082501.240751-1-wzh@ilbers.de> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: wzh@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: Nug6jIgpuYHj This series aims at simplifying and speeding up the testsuite execution. This should enable contributors to actually run it and test their patches. As pointed out in [1], the testsuite currently tests various features over and over again, while others are not tested at all. We try to improve this by focusing on "feature" tests, whereby each test case checks a set of features instead of a whole image. We further identified long running tasks (like the kernel builds) and move them to a dedicated test. The execution time reduction varies greatly depending on the hardware, network and sstate usage. When applying these changes on recently added patches, the time was often decreased by more than 50% on "fast". Note, that more refactorings of the testsuite are needed - and planned, but this series should already address the biggest pain points. [1] https://groups.google.com/g/isar-users/c/uZBTIHzLp8Q/m/8k0L1vfMDAAJ Changes since v2: - move startvm tests after each build to avoid inter-testsuite dependency - remove SstateTest from full Changes since v1: - continued work on testsuite refactoring, this supersedes: - testsuite: fix SignatureTest by avoiding absolute path in bblayers - testsuite: add rootfs target for rootfs only tests - Enable SState cache in GitLab CI - testsuite: fix typo in log message in perform_signature_lint - completed exclusion / refactoring of db2m sbuild chroot - GitLab CI: increased after script timeout to give sstate upload more time - GitLab CI: do not upload sstate artifacts of images (as almost no reuse) - do not add SignatureTest to fast tag (as otherwise VM tests break...) - rebuild container test images in VM test to not depend on test order - rebased onto next Anton Mikanovich (1): testsuite: Group prebuilt_containers related test cases together MOESSBAUER, Felix (17): testsuite: move targets with custom kernel to separate test testsuite: enable ccache on kernel tests testsuite: make prebuilt container a feature test testsuite: make compat test standalone test testsuite: handle IMAGE_INSTALL solely in cibuilder.py testsuite: limit cross_debsrc test to subset of packages testsuite: forward SSTATE_MIRRORS into CI env on sstate ci: add support for sstate cache testsuite: make test_cross_deps more specific testsuite: fix typo in log message in perform_signature_lint testsuite: fix SignatureTest by avoiding absolute path in bblayers testsuite: use more recent distros in SignatureTest testsuite: make SignatureTest idempotent testsuite: add rootfs target for rootfs only tests testsuite: refactor sbom tests to avoid overhead testsuite: make sbuild-flavor test standalone testsuite: skip VM tests if images are not available Zhihang Wei (2): testsuite: Execute startvm testcases after building testsuite: Remove SstateTest from full testsuite .gitlab-ci.yml | 36 +- meta-test/conf/bblayers.conf.sample | 8 +- meta-test/conf/local.conf.sample | 10 - .../hello-isar/hello-isar.bbappend | 3 + .../recipes-app/libhello/libhello.bbappend | 5 + .../recipes-core/images/isar-image-ci.bb | 2 - .../recipes-core/images/isar-rootfs-ci.bb | 17 + testsuite/cibase.py | 34 +- testsuite/cibuilder.py | 49 +- testsuite/citest.py | 740 +++++++++++------- 10 files changed, 602 insertions(+), 302 deletions(-) create mode 100644 meta-test/recipes-app/libhello/libhello.bbappend create mode 100644 meta-test/recipes-core/images/isar-rootfs-ci.bb -- 2.39.5 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260123082501.240751-1-wzh%40ilbers.de.