From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 28 Jan 2026 14:01:28 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lf1-f57.google.com (mail-lf1-f57.google.com [209.85.167.57]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 60SD1Omu010801 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 28 Jan 2026 14:01:25 +0100 Received: by mail-lf1-f57.google.com with SMTP id 2adb3069b0e04-59b78adfc09sf3489079e87.0 for ; Wed, 28 Jan 2026 05:01:25 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1769605279; cv=pass; d=google.com; s=arc-20240605; b=dQSssK9GdiaE+9JQFJ4giG86CyqVL9LL5QnxkomQvYWGGu4j0RjGrXZYmNbltGCFOM TpCAmdlRPjolJOuCjzvz+Te4vYhfFeTP8OngpRWhieIzkY2slTI9KHsS61eAoJPhAH88 ZBKJnUYsdabNoieOHtu95ezLsipX9lVqtd3nhpZFkjcQHSsYwfMLgevNyj9FEezziITu om8y5SUXCjlruMJsE8pVAPgo/IncSbRlGMufkQVi9N1KEQN7HiCWWhEdWhA8T47uAALA GOes9p2UtdnVmKqCmLQe8+6o0MdRw/5b0K5Tz96vzZ41WwPDxqu95yFAyn0VH0s//7HN Tibw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :sender:dkim-signature; bh=KAz3Ik8OIhDVTcQZHxk94eOl0X6nq+qRN6U+qLDe+vQ=; fh=ujH/kUCb5rdgRgGbRaOKlnH1Ks8RSaJhfPhPck1rtKs=; b=bewpPkZ9hJSYzrE+ARnMuhB3dobG9DLfd+jaJWBHTZZgHc+MdyLc9Ai1HWyxN0xWh8 L9bxIhxtE0czscszeeJ1ak2Mj/cbuysh8pprkvxZ/BwoJzvUp8z0HuoGr1XEFVcK1QTc nwFGzOXhlwRK/zQ/Oyo4RqtzzBoB4yR2ZguywzQc7CI85YPdN14y0Xr39OQ1pl88cKVv rJ13F1Pt5uu+2/TmiRKwA2Sjm4LdnOBfaS+Rku3WYdHzv6CkPY46DYDGHBEFv8J6hvnT Yv4GpiLtWNNKMFdWIM80FEgoeKyzzUSMrzhuBv1a33KNXM/EHrgdIiPNnVGA4JcCYLNc BYXQ==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1769605279; x=1770210079; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:in-reply-to:from:content-language:references:cc :to:subject:user-agent:mime-version:date:message-id:sender:from:to :cc:subject:date:message-id:reply-to; bh=KAz3Ik8OIhDVTcQZHxk94eOl0X6nq+qRN6U+qLDe+vQ=; b=dXdj2xg7Eotl5HW7JmbAJGfiFR5GjnplyAKBPYR+0nm+Ez/N+8MIg+zMxyzX2G+YMa ywTw5PmJZspxJU7TBfBq/DpUbMLIK71Hm5g0JzPRIZDrFZx6snKwwtC+MYRAFe958Brs 95mTj0mFVfNrr/9NRnj9KVODsFu/Iqk4PalNe893c8OL7FbAekMrnpw2ug2pxiMr35iH aYRJFWrypQiYB7LGkYgqGat4qy/M9FsnH0lM/DiqlywP+8b0tdACWIm8J+35LeVUDaDl JmbawRkb8SoYAxf/VKqlxK4SsvZa36SC1PUm+A+4rVipE8H52hLruH/0gsjidRCVd15c 7J0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769605279; x=1770210079; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:in-reply-to :from:content-language:references:cc:to:subject:user-agent :mime-version:date:message-id:x-beenthere:x-gm-message-state:sender :from:to:cc:subject:date:message-id:reply-to; bh=KAz3Ik8OIhDVTcQZHxk94eOl0X6nq+qRN6U+qLDe+vQ=; b=XLy3n293h4X5joyQQz7NrbexDOJlrZCMjDb2DQezjKP+2//suWsPZOO87Gvveu9pGK 0lkZqdLIZGVftBJyRrtS+u4lTOc53GJrf1dJlyz3qaGfhDoeeotmku4jXRqF738PHfiD ZdYVaYCTUrLRLSAxm4KION0eTNVM8uma943vyFNRXqJXZGlxzrdFbocMAIOiQAwNx7rR Y+3glrZURaiTjcBDoCYcPB5nzDOKnnSr6/F3xDDXPi2MDtS+R587StLPmG++0LZGGNV9 UOCzG2s1SVgVu5j/lPHdBX4EE4/EGs0owyzgHJ5iAJvRcKc2pRKgEuf3vt6Swdhb5Vmm jyzw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCVyqeOVVYcZZ8WgHRIY0lZnZ+1c0AD63LEMVAUSy/SaC8yx8kHcTIbUNP27HmQGj5l5z6Xs@ilbers.de X-Gm-Message-State: AOJu0Yx6D2tOuiDJ8+3L/IupsEdS8DywYmXyF3T4/cjccol/zX0I7v6y +SduHsTGYohD1XXlC6G0ENGPM7vbmUkSZ6nyAPH9iw483GiaL3avvvfQ X-Received: by 2002:a05:6512:12ce:b0:59d:f546:a90a with SMTP id 2adb3069b0e04-59e04012d9cmr1927404e87.6.1769605278905; Wed, 28 Jan 2026 05:01:18 -0800 (PST) X-BeenThere: isar-users@googlegroups.com; h="AV1CL+FmTAEHOP4wWXgyHz4ohHzTReJ1OSNRd2+f69wc3RbnHw==" Received: by 2002:a05:6512:2313:b0:59d:d722:902 with SMTP id 2adb3069b0e04-59dd798229bls3140579e87.2.-pod-prod-04-eu; Wed, 28 Jan 2026 05:01:16 -0800 (PST) X-Received: by 2002:a05:6512:8006:10b0:59e:708:cf56 with SMTP id 2adb3069b0e04-59e0708cfb6mr817768e87.26.1769605275629; Wed, 28 Jan 2026 05:01:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1769605275; cv=none; d=google.com; s=arc-20240605; b=jSPgMnYnydQ/pdluG3UO+XYOZ25k2Ya/94ibS2MayS8dKhBrXX7TSiSNVzIfeN7opT 5F4XCYJVXrflBtNal09B22cLVBahzTTSTwpAB5do0Xj9WTvE51CA0Vx8DQDNUiZgOPy+ 0+E/n73e2uJkuahKjW6aCbW22SRdV+5Tsi9UcFNcy7XBI6pE9Yniddqc13a9XBcrs0zN WJrwj16ULbjtJL/+hEnUV9S6ZEfHCu/ZZKXreFsIAnJgB2ogltQzWY9V4cVj8NmE+dGO AUqLZaVO0nQE5oghHPjUoIQ9KYLojRycUwVZ7FQVucCGZOoO1Jghb2p58EG70dvnzeNt 2fUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id; bh=Sjj70PyEVXUXIfKromEbYTBE9R4znverlmH/DEj1Fng=; fh=C37l5vGm7U2eZ/l7+t9CMsA78Do2Rg4OXXyzPhO3+tA=; b=Sz375hWtVEMmkPrqwcpXWjAL0AtBYhXb05EgjUdtoBWKh+Htode6LLzD+B0YKjJufs tDwK7xU+yV8MYsIjz5ruQeNhz4T8D4wgb1d1Jr1wFcIqbc35heboL5qLrvSKY2H7h1gE FuzUxnYg2riVvRsUmVh4TGS3TGP13voqBORvXpIuA3wG6QuiYVfbHzhxiVtVEKPx75kM vOJJcbrQWQ4WWcYdk8GDCt+SOnZ+Yn6Y9kZXknPy15h1AYV1HHKPjZ6tEOXhm9a/y2vj bNwnFFL5EySmjHs9ffsjkNcSWAJmRz39yVJH2X8nOpl2A6JCyHFhMqUQTZbXANKzrQLc Lodw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-59e0a43e9f2si26337e87.7.2026.01.28.05.01.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Jan 2026 05:01:14 -0800 (PST) Received-SPF: pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from [192.168.178.117] ([88.130.203.42]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 60SD1CvS010792 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jan 2026 14:01:12 +0100 Message-ID: Date: Wed, 28 Jan 2026 14:01:12 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 00/20] Various improvements to the testsuite To: isar-users@googlegroups.com, felix.moessbauer@siemens.com, amikan@ilbers.de Cc: cedric.hombourger@siemens.com References: <20260123082501.240751-1-wzh@ilbers.de> Content-Language: en-US From: Zhihang Wei In-Reply-To: <20260123082501.240751-1-wzh@ilbers.de> Content-Type: text/plain; charset="UTF-8"; format=flowed X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: wzh@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: vROAqZkS9Gwy p1-16 and p18-20 were applied to next. Thanks. Zhihang On 1/23/26 09:24, Zhihang Wei wrote: > This series aims at simplifying and speeding up the testsuite execution. > This should enable contributors to actually run it and test their patches. > > As pointed out in [1], the testsuite currently tests various features over > and over again, while others are not tested at all. We try to improve this > by focusing on "feature" tests, whereby each test case checks a set of > features instead of a whole image. We further identified long running tasks > (like the kernel builds) and move them to a dedicated test. > > The execution time reduction varies greatly depending on the hardware, > network and sstate usage. When applying these changes on recently added > patches, the time was often decreased by more than 50% on "fast". > > Note, that more refactorings of the testsuite are needed - and planned, > but this series should already address the biggest pain points. > > [1] https://groups.google.com/g/isar-users/c/uZBTIHzLp8Q/m/8k0L1vfMDAAJ > > Changes since v2: > > - move startvm tests after each build to avoid inter-testsuite dependency > - remove SstateTest from full > > Changes since v1: > > - continued work on testsuite refactoring, this supersedes: > - testsuite: fix SignatureTest by avoiding absolute path in bblayers > - testsuite: add rootfs target for rootfs only tests > - Enable SState cache in GitLab CI > - testsuite: fix typo in log message in perform_signature_lint > - completed exclusion / refactoring of db2m sbuild chroot > - GitLab CI: increased after script timeout to give sstate upload more time > - GitLab CI: do not upload sstate artifacts of images (as almost no reuse) > - do not add SignatureTest to fast tag (as otherwise VM tests break...) > - rebuild container test images in VM test to not depend on test order > - rebased onto next > > Anton Mikanovich (1): > testsuite: Group prebuilt_containers related test cases together > > MOESSBAUER, Felix (17): > testsuite: move targets with custom kernel to separate test > testsuite: enable ccache on kernel tests > testsuite: make prebuilt container a feature test > testsuite: make compat test standalone test > testsuite: handle IMAGE_INSTALL solely in cibuilder.py > testsuite: limit cross_debsrc test to subset of packages > testsuite: forward SSTATE_MIRRORS into CI env on sstate > ci: add support for sstate cache > testsuite: make test_cross_deps more specific > testsuite: fix typo in log message in perform_signature_lint > testsuite: fix SignatureTest by avoiding absolute path in bblayers > testsuite: use more recent distros in SignatureTest > testsuite: make SignatureTest idempotent > testsuite: add rootfs target for rootfs only tests > testsuite: refactor sbom tests to avoid overhead > testsuite: make sbuild-flavor test standalone > testsuite: skip VM tests if images are not available > > Zhihang Wei (2): > testsuite: Execute startvm testcases after building > testsuite: Remove SstateTest from full testsuite > > .gitlab-ci.yml | 36 +- > meta-test/conf/bblayers.conf.sample | 8 +- > meta-test/conf/local.conf.sample | 10 - > .../hello-isar/hello-isar.bbappend | 3 + > .../recipes-app/libhello/libhello.bbappend | 5 + > .../recipes-core/images/isar-image-ci.bb | 2 - > .../recipes-core/images/isar-rootfs-ci.bb | 17 + > testsuite/cibase.py | 34 +- > testsuite/cibuilder.py | 49 +- > testsuite/citest.py | 740 +++++++++++------- > 10 files changed, 602 insertions(+), 302 deletions(-) > create mode 100644 meta-test/recipes-app/libhello/libhello.bbappend > create mode 100644 meta-test/recipes-core/images/isar-rootfs-ci.bb > -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/ced9d412-3c6a-46c4-ba1d-088baf8cebf1%40ilbers.de.