From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 23 Jan 2026 09:25:18 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f183.google.com (mail-lj1-f183.google.com [209.85.208.183]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 60N8PHcs015352 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 23 Jan 2026 09:25:17 +0100 Received: by mail-lj1-f183.google.com with SMTP id 38308e7fff4ca-38305f09475sf10183811fa.3 for ; Fri, 23 Jan 2026 00:25:17 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1769156711; cv=pass; d=google.com; s=arc-20240605; b=fGpR0lpm4Lij6I//07vFdt9Ln6h0OegWZcSLNLgzyZReL1TaqTrMOBIB5j4ZvXJzgA sSFRkv079Adl2k7eM+SP0AyeYpnbwH45RxQnPTPTCfRi5SX2YLcov9xFJ0qHT3sF8z+6 oOzs73PDcDXsU3pztfqvlgf9FoSu63ZaFZBcKzhH1VnCzHHV9ssauXI3hyexQ9uuIVE0 jTqC1yrA3n0eUInTP2RK1AUk4rz9eXwJNT/s8epMKLskL7haBUH/I81xgvUOCue1nIze EevQuYVRaFVzNDKOnk/+BJU5wE0ksAcaTyswq+ayYLmxjmS4ukvD2CESjmXwd2DCp2Ff nLiw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=FsWTelvCTgbf+S8C4dNF6azbNrTDS9QAqRFyikHdqwU=; fh=C5uiyL7Z181x0gz6rl3mFPpzFhoOAnotbRmsRKrGvlQ=; b=ehSGL10hM+UpveD4yD/vp47u5INDKe/LgkSqWTfD5dvVW42tJKEEo04IGXlLNuHeXW JW0n6qiKMEM9SRFz7wUZQogf6Q6ajReEJnv3qu0Knx6azAuxxdzHzHxH+p457kqCIQuy 0F5+mk3zf3S9oNmwlakeoJMC33GC76mXMUdeKVB+8nJZhzWd71kDlMsMnvT9uUt8kbFb MXd89eBhLqa+mL5JOCPtABPMPupevbsi/stMs5DuuKL9HDiGJuE+XQs8t7fY9urBiN04 +cF4+1DLRTiwyJgZ7/j9nKcdP8uxrDzE9hn5k7/ySr1vXezFbpGOxs8sSjIhN85gpV+g ng5w==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1769156711; x=1769761511; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=FsWTelvCTgbf+S8C4dNF6azbNrTDS9QAqRFyikHdqwU=; b=K5S1w4E134We00vUsSUPCJBe2Vyo5nhE/oIwtbuG6eRPhZ5XOl2trguzaM3Zw1k4je Yh30VWyUJCp0OfmewJf6pKwTspKq8Crs5/9X73klF2w1OFcupU3e9L9KcK3xyqeE8Csc YC0ef11R0L47vam/WtZji4UxvpObJi7xOjz6YGil5/ZA5yQJjcm3kw/PyQwL99kzUYTY UtrOpoUm+xhtsmZ7ol8b7n1La1Ezgtoco++2BzkBYIYa5CohNZ9iy63CaFz/L+FhE1tT AjEG50V6a8S9PeT3d6wQUNW2bRBdIX2ZwrDYxRMUsz3S7VZ0EKw7HoVOidqOL3oGEuGu pWSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769156711; x=1769761511; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=FsWTelvCTgbf+S8C4dNF6azbNrTDS9QAqRFyikHdqwU=; b=qngucZjChmwqEGZUIPft1a95bW7RNDW87zYvj/MgqWlbSrz0wb5oUaleDHosRiDVsU 4Img+xesmByjDQAaIom6e7E386qy6JV/Jw7zntAlKq+wzgKIEKw+l1X2ZqcH9e+x/8+X MJJ+P8kHvE++VWCG4sod87sJZkBRLGzRgSvkHeTBgrhROZ8FSBtxrxabviCKiCmMEDxo vFqFii6ULyeGG4EKaClZ2MDsXQMSpHALAqnVQhpWhQZmpHr6zdZhZsUxbPlBIKvkSN1i RLNe8a3Y27nRqH5fqWHCDTAqf02/Ch2VSFQQB5g34ZX+EXhiy/e85wOO4tHCHlYSSIrn TZ4Q== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCV4AJcY9LUt6KOlyOFbynphaQPAf5bQNAn38VTfofOQvdAeCpSqzBEie63wGr6QUEHdgj9v@ilbers.de X-Gm-Message-State: AOJu0YwPZ3+GxmOJ9fH8uNNv/F2QqU6nWU7z3ri3mEgRhf9c0ew7DjKG Yol8pPtGO4aovPIAgCElbLXCHnxPcMczfDcimeg+qOpLwkHP7Ik0A/c0 X-Received: by 2002:a05:651c:b25:b0:37b:afdc:bb0d with SMTP id 38308e7fff4ca-385d9ed5f54mr6384961fa.6.1769156710837; Fri, 23 Jan 2026 00:25:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com; h="AV1CL+HbrdhS8zetyiihHJJmHpqLcxkt989tGHVv5NOryCiV7w==" Received: by 2002:a2e:871a:0:b0:385:ba7e:10fb with SMTP id 38308e7fff4ca-385c269049fls2847631fa.2.-pod-prod-02-eu; Fri, 23 Jan 2026 00:25:08 -0800 (PST) X-Received: by 2002:a05:651c:1992:b0:380:989:f5fb with SMTP id 38308e7fff4ca-385d9e1bff4mr7276531fa.0.1769156708130; Fri, 23 Jan 2026 00:25:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1769156708; cv=none; d=google.com; s=arc-20240605; b=BOXffKRfg/JfUJ4LQYH987fidR5RU1b4dRw/vd46PnoCy/vYB4kZ6IxYq8Rb5mq1+4 v9HTqzyGHjTfp3+T+YcWeokFxze6c1AIumUWYoNNJB8kE2LIri017DtIDIF5JQpebp6O SwjejGLJUJkYKBjK6HKY2WZATNe+LsHUQPIWR577B2V9xZ3sjHc68ZQ5ayJYZG/eJgPT XuhrUyJOxkK5ZnY5uK7LE2MUERunRAm2oVm491i2NXl3XEXIV9CRsDhHk7ND4yj1QHEV +9HHlQkIfTpL0FDsTGivg78KhuhgZ3yJ/FTKr2hte5KQJC6tuvRPFazSAw56B17yW/u0 7HAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=bmrc9pIcykHA72MNymnLGtR+iP/RVsJueCR+h+ydxCA=; fh=V42d4Y/lNDOgZ/9/qsphFBeZCU4sz3YugfOOjBc+1H8=; b=hGwCkWKaMtz+wkDPGgjGlb/xQbE0MkvvieM5SDZYp6Go9i8e85KuVsMrZ2EPYooMLy c0ntgNfO6tiErzCL4FcH48FQBYRlA6gs72GSAIvRE21ayZIH9nr08LRy3IPfuRoDFr5g g5M6PByXUTtVqJINq4KAEfDabfzDi02jRlA/n5029bWEMg6JTCvbxV3ObhhVtDqrmIn/ RjpUZf0c5oiThfOMrJZHoCWTb35C0BsvV8uCuMWtbGB+GYHTCUYSm8DC1zRqRGCPgIEq 4ryajThXmGLFwC/FX59I9tE3BbPISmmPi3lWGP6TVKShyDzW/XGUL3yv9IMTlbqmCef2 2VeQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-385d9f8daf2si458931fa.2.2026.01.23.00.25.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Jan 2026 00:25:08 -0800 (PST) Received-SPF: pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from debian-zwei.m.ilbers.de ([88.130.203.42]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPA id 60N8P1Xu014977; Fri, 23 Jan 2026 09:25:07 +0100 From: Zhihang Wei To: isar-users@googlegroups.com, felix.moessbauer@siemens.com, amikan@ilbers.de Cc: wzh@ilbers.de, cedric.hombourger@siemens.com Subject: [PATCH v3 15/20] testsuite: refactor sbom tests to avoid overhead Date: Fri, 23 Jan 2026 09:24:56 +0100 Message-Id: <20260123082501.240751-16-wzh@ilbers.de> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20260123082501.240751-1-wzh@ilbers.de> References: <20260123082501.240751-1-wzh@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: wzh@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: Fnd3yPrtVhFq From: "MOESSBAUER, Felix" We currently test the SBOM infrastructure in all image builds, which adds a significant overhead. We now change this to not generate SBOMs in general (and by that avoid building the dependencies). To not have a testing gap, we add a dedicated SBOM test that checks the SBOM creation for various targets. In addition, we now also check the content of the SBOM for plausibility. In the future, the SBOM test can be extended without slowing down the overall test execution. Signed-off-by: Felix Moessbauer --- testsuite/cibase.py | 26 ++++++++++++++++++++++++++ testsuite/cibuilder.py | 4 ++++ testsuite/citest.py | 33 +++++++++++++++++++++++++++++++++ 3 files changed, 63 insertions(+) diff --git a/testsuite/cibase.py b/testsuite/cibase.py index 5ef1a5b5..fd6a3df9 100755 --- a/testsuite/cibase.py +++ b/testsuite/cibase.py @@ -140,6 +140,32 @@ class CIBaseTest(CIBuilder): self.delete_from_build_dir('ccache') self.unconfigure() + def perform_sbom_test(self, targets, **kwargs): + """ + Build a rootfs containing a needle package and check if that package + is added to the sbom. + """ + import json + + needle_pkg = 'cowsay' + self.perform_build_test( + targets, image_install=needle_pkg, + generate_sbom=True + ) + + for t in targets: + ds, pn, distro, machine = \ + CIUtils.getVars('DEPLOY_DIR_SBOM', 'PN', 'DISTRO', 'MACHINE', + target=t) + for t in ["cdx", "spdx"]: + sbom_path = os.path.join(ds, f'{pn}-{distro}-{machine}.{t}.json') + self.log.info(f"Check {t} SBOM in {sbom_path}") + with open(sbom_path) as f: + sbom = json.load(f) + pkg_key = 'components' if t == 'cdx' else 'packages' + if not any(c for c in sbom[pkg_key] if c['name'] == needle_pkg): + self.fail(f'{needle_pkg} package not found in SBOM {sbom_path}') + def perform_sstate_populate(self, image_target, **kwargs): # Use a different isar root for populating sstate cache isar_sstate = f"{isar_root}/isar-sstate" diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 9c97115b..7538ade2 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -126,6 +126,7 @@ class CIBuilder(Test): installer_distro=None, installer_device=None, customizations=None, + generate_sbom=False, lines=None, **kwargs, ): @@ -176,6 +177,7 @@ class CIBuilder(Test): f" image_install = {image_install}\n" f" installer_image = {installer_image}\n" f" customizations = {customizations}\n" + f" generate_sbom = {generate_sbom}\n" f" lines = {strlines}\n" f"===================================================" ) @@ -275,6 +277,8 @@ class CIBuilder(Test): 'CUSTOMIZATION_FOR_IMAGES:append = " isar-image-ci"\n' 'HOSTNAME:isar-image-ci = "isar-ci"\n' ) + if generate_sbom is False: + f.write('ROOTFS_FEATURES:remove = "generate-sbom"\n') if lines is not None: f.writelines((line + '\n' if not line.endswith('\n') else line) for line in lines) diff --git a/testsuite/citest.py b/testsuite/citest.py index eaa4c440..d908f9bc 100755 --- a/testsuite/citest.py +++ b/testsuite/citest.py @@ -694,6 +694,39 @@ class CustomizationsTest(CIBaseTest): ) +class SbomTest(CIBaseTest): + """ + Test to check if sbom is generated and contains expected packages. + Most tests are rootfs tests to avoid costly initrd build and imaging. + + :avocado: tags=sbom,fast + """ + + def test_sbom_rootfs_generate(self): + targets = [ + 'mc:qemuamd64-bookworm:isar-rootfs-ci', + 'mc:qemuarm64-bookworm:isar-rootfs-ci', + 'mc:qemuamd64-trixie:isar-rootfs-ci', + 'mc:qemuarm64-trixie:isar-rootfs-ci', + 'mc:qemuamd64-noble:isar-rootfs-ci', + ] + + self.init() + self.perform_sbom_test(targets) + + def test_sbom_unsupported(self): + targets = [ + 'mc:qemuamd64-bullseye:isar-rootfs-ci', + 'mc:qemuamd64-focal:isar-rootfs-ci', + ] + + self.init() + self.perform_build_test( + targets, bitbake_cmd='do_rootfs', image_install='cowsay', + generate_sbom=True + ) + + class SignatureTest(CIBaseTest): """ -- 2.39.5 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260123082501.240751-16-wzh%40ilbers.de.