From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7353323043876241408 X-Received: by 2002:a05:6a20:6a14:b0:1a7:4962:6fad with SMTP id p20-20020a056a206a1400b001a749626fadmr642992pzk.10.1712334745724; Fri, 05 Apr 2024 09:32:25 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6a00:a08:b0:6ec:d942:1606 with SMTP id p8-20020a056a000a0800b006ecd9421606ls1672018pfh.2.-pod-prod-07-us; Fri, 05 Apr 2024 09:32:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGFkpfBFzTd5ivVEL0r3tYR0gfh/gnxrNic1faOV52n2v4qQdMxBzZnzIMR5nvZrJT9lZvn X-Received: by 2002:a17:902:f611:b0:1e2:2587:d43f with SMTP id n17-20020a170902f61100b001e22587d43fmr2733857plg.36.1712334742957; Fri, 05 Apr 2024 09:32:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712334742; cv=none; d=google.com; s=arc-20160816; b=eEKZvsZIymMm83Kk/ktnHvHcmQq35u2EBXcNJuIroMkAZdZzCzaYdJKbqBz9hHgZSe 1n8pAqQSx8PFl/xxy0mr3rlSbOvdgc78OD8ylCUQey7fb3tV29rbGSGs60VBBQbF5cV1 19ErgB7KGUx2WXT6d3+d80dWd9fjeRbtnsHO2FjOZ5g/ilhfXPSi2fy2L2wG3dq19S8E iT9XhG7NW6MILsvXvDLByVHrlIfxVFZSvZEApLMH0HDydiBICKEZYj86wybjKbsUx97P XP9xO/S3yFOuYtHgcB8cCgf4GNpO1iylSSJ45xDK8tXxS2RcIC0hEnNA16ckqDgPIV5U IM7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=+fV1RXziguR83yrysLAByW6r/CHw+cnS8GN4MxXn+Sk=; fh=3bh9z2BWAzGAc1byRhMi0K+arMZmX5BQHE0sOxINdyU=; b=AaHRq6JlWUw6DLOnINvwBp7ZWTgLoAXbg9qJcMq42v/IFzlODP2ZhMZms1mfKHe8Oh /LYxYklmCTF24Xs0S2Yz0YBFRk5MFcFgVsYn+O1obqKfroPjbv1nYyXFS0YXVkOevm4n XxWa5fEsK58g3NVbz0H80163Y8iQ05FTIdQzYjkK1ivorVj06fIeLBteb4TlJc+gytUF hsH08xlQBkJHNj7w98GjoDB9qL/qI/6wp+N4cI8A+d6rk4J0splnJ8U2q7S4XJWwiROL XKiQoVzsWLr4WetsngaKzuCLgYBSjGii4zvTxAkD4KUsqSX3/rJtvUGVWQsZD98jCWwv 5e9g==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=iTMiieUv; spf=pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163220b4bfc19647cb4060b4-dJlTBs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Return-Path: Received: from mta-64-228.siemens.flowmailer.net (mta-64-228.siemens.flowmailer.net. [185.136.64.228]) by gmr-mx.google.com with ESMTPS id mp6-20020a170902fd0600b001e2c4ecc402si110212plb.8.2024.04.05.09.32.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Apr 2024 09:32:22 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) client-ip=185.136.64.228; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=iTMiieUv; spf=pass (google.com: domain of fm-1325773-20240405163220b4bfc19647cb4060b4-djltbs@rts-flowmailer.siemens.com designates 185.136.64.228 as permitted sender) smtp.mailfrom=fm-1325773-20240405163220b4bfc19647cb4060b4-dJlTBs@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: by mta-64-228.siemens.flowmailer.net with ESMTPSA id 20240405163220b4bfc19647cb4060b4 for ; Fri, 05 Apr 2024 18:32:20 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=chris.larson@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc; bh=+fV1RXziguR83yrysLAByW6r/CHw+cnS8GN4MxXn+Sk=; b=iTMiieUvfNcJbEF9L8Yul8hy9wS69Y0cviNgoMMhJnfb0rb58C+FDoYrMtu0Y6s+MRl9q3 8bxS5tRPxGOBA2SQW4mT/uDubP7IMw+GfOaQ674k3vEjM8sp83S2ynY/zG8AzmTJ5DCUzUPF 3AIfroGVxRsP9pDrXYFuCF8gGVnIw=; From: chris.larson@siemens.com To: isar-users@googlegroups.com Cc: Christopher Larson Subject: [PATCHv2 0/9] Add more signature cachability tests to the testsuite Date: Fri, 5 Apr 2024 16:31:26 +0000 Message-Id: <20240405163135.2987489-1-chris.larson@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1325773:519-21489:flowmailer X-TUID: 1ZgO9cBXmKgv From: Christopher Larson This series improves isar-sstate's lint command to support checking sigdata in tmp/stamps and to check for absolute paths in SRC_URI, and adds an additional test to the testsuite to exercise this capability. The new test checks for signature cachability issues much more quickly, as it does so without a full build, so it checks more target configurations. This uses bitbake's `-S none` option to avoid building anything, and then runs isar-sstate lint on the resulting sigdata in tmp/stamps. This test will accept a verbose parameter on the command-line to pass --verbose to isar-sstate lint, which will show the "other" absolute paths found, not just source and build directory. This series also fixes two of the failures which were identified by the new test. Here we can see the issues identified by the new test, which are fixed in this series: ``` $ avocado run testsuite/citest.py -t signatures --max-parallel-tasks=1 JOB ID : 3a3308967946663d9b239f638b030502fa80ef0a JOB LOG : /builder/avocado/job-results/job-2024-03-29T19.24-3a33089/job.log (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: STARTED (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: FAIL: Detected cachability issues (42.93 s) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 44.63 s Test summary: 1-testsuite/citest.py:SignatureTest.test_signature_lint: FAIL ``` Applicable section of the avocado full.log: ``` 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| found cachability issues (scanned 602 signatures) 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` If we strip off the prefix for legibility: ``` ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" ERROR| found cachability issues (scanned 602 signatures) ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` The sstate/signatures tests were run for each commit in this series using `git rebase -x` to ensure it does not break a bisect. I welcome any and all feedback on this. v2 updates: - Explain the use of --dump-signatures in the docs on the lint command in isar-sstate - Explain the stamps regex the way the sstate regex is explained, for consistency - Support wildcards for the --excluded-tasks argument and default exclusions - Drop the additional default task exclusions, as this should be revisited - Fix a lint traceback if a variable's value is only whitespacxe Christopher Larson (9): isar-bootstrap: avoid forced early expansion of key vars isar-ci-ssh-setup: avoid abs path in signatures isar-sstate: lint: check for absolute paths in SRC_URI isar-sstate: lint: add support for checking stamps isar-sstate: add --excluded-tasks argument isar-sstate: fix failures if a variable is set to just whitespace cibuilder.py: add -S support to the bitbake method testsuite: add perform_signature_lint method testsuite: add signature cachability checks .../isar-ci-ssh-setup_0.1.bb | 3 + .../isar-bootstrap/isar-bootstrap.inc | 8 +- scripts/isar-sstate | 85 +++++++++++++++---- testsuite/cibase.py | 19 +++++ testsuite/cibuilder.py | 5 +- testsuite/citest.py | 22 ++++- 6 files changed, 121 insertions(+), 21 deletions(-) -- 2.39.2