From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7353323043876241408 X-Received: by 2002:a17:90a:de94:b0:2a1:f586:d203 with SMTP id n20-20020a17090ade9400b002a1f586d203mr294425pjv.41.1712078937505; Tue, 02 Apr 2024 10:28:57 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90b:4f88:b0:2a1:f3c8:8df4 with SMTP id qe8-20020a17090b4f8800b002a1f3c88df4ls3362139pjb.2.-pod-prod-03-us; Tue, 02 Apr 2024 10:28:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGeXHIgaTKj8nrzhbdc20ClvZ/CxixlIYJdS7Dd3TxDMNVuYaiY8vBz23YCzVUPhdu+oWkC X-Received: by 2002:a17:90a:468f:b0:2a2:11f1:8630 with SMTP id z15-20020a17090a468f00b002a211f18630mr399557pjf.5.1712078935850; Tue, 02 Apr 2024 10:28:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712078935; cv=none; d=google.com; s=arc-20160816; b=qDT+QmPSDrIjWAYxUJGYcyzkgxf6XXxWmoVZdgUxgXuVVxzfmcCByvzuK7lpOuvFpx wjGhOYsj6ArDElQc8SmK0XrgP69QpWZIXEEMbUilnACXk4AOTrBLjeJKvGhJCsMCTDUi nOhSjIBw11W5vHp3ZmgPirgxj0UDcLgYT+y6gcEc2mJ5vuQwha8bgVzHCnpmKf0hC4eC WA7cA4THjbAayiF1mYb1EAzRAbGo9iiKMjJzF7McTB9vwid/rYdj7CmbbE0jCovRnCbf 0XPWx/DDZOVrdSu7Upw3ODmVgGuzjXqj19k/Cz0/kMyuYeAFTj17u9occFQSCyc7s2FM 1LaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=Blp8Hggosrp44tSJ9ehomIsGFj3w9hdwq7i90GkGkQ4=; fh=3bh9z2BWAzGAc1byRhMi0K+arMZmX5BQHE0sOxINdyU=; b=h9QKV3/VdA3u7Dt2BKkDNY4n2ZKgA9FphwHfq1XD4kcuyYkMDDsVBJAWQQjE1I32/d Iv1kAZAzzdKMwxwewz8rbkoqpQ1Pl/XhGyrHQuRIb0rI+W/qI/ySbLqiiyDVyANCCUGG GMrahy0rsZbL+B4/GAfvZ4du9iE4qPEs/LAnjXxPISS+jR7oGbJON6R6MzPdVAZI/nQS /d0d/onSneJUpOQu/AoMUJWOG1sreCtSxWp5dCKQDhGB/wfCNtG1O1wayYOPivT8+LiG pO24G//o8CiaudUEVQ09rBYt4WZGP8cZrwyu8u5TdDyqNYMqNDHkyKt4B8hCqM7XroTH MqNw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=iM5auxIV; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com. [2607:f8b0:4864:20::529]) by gmr-mx.google.com with ESMTPS id np17-20020a17090b4c5100b002a28309e16bsi11798pjb.0.2024.04.02.10.28.55 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 02 Apr 2024 10:28:55 -0700 (PDT) Received-SPF: pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) client-ip=2607:f8b0:4864:20::529; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=iM5auxIV; spf=pass (google.com: domain of kergoth@gmail.com designates 2607:f8b0:4864:20::529 as permitted sender) smtp.mailfrom=kergoth@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-53fbf2c42bfso4237342a12.3 for ; Tue, 02 Apr 2024 10:28:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712078935; x=1712683735; dara=google.com; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Blp8Hggosrp44tSJ9ehomIsGFj3w9hdwq7i90GkGkQ4=; b=iM5auxIV8bEFyf/PvSUVPXiJ8goZHXOjlOpXUlr6LBZn/qPD4aYae2WG5eggaQTMoX 9SR4e9Wrc9fdtGTlWOmAQGmPxdLuPJf8Iixr1vdQFeGmXGk0cJ2yzKiDesf2MyjtzZLe hjDuZKa3rnLT/jRYqDwtavUcrR8CsATypvBvxVLH4R8E5uaWHIsReUjDhwa2I3G+nr7C rfGoGW9xKAiq8GSvJIBnakcjkIxgcqrBFd45yHCzU5NGz+rMmzITU+4R81OB2uBga0yM hKCAsDhuWISIcnA6ui6NmAHGaq/F/xqnBOt3L6GQeRk/xiDyibMR1JfxUA+JdhoF8SCg x8UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712078935; x=1712683735; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Blp8Hggosrp44tSJ9ehomIsGFj3w9hdwq7i90GkGkQ4=; b=riYr1HwCAMtxVekdP17ZpU7m2a9ulhcJg0riPBKnFywN1iP8+glyYZTO//+DkN0fch J7xlxTy5IIDXQEnkOl7v4hnqDtefdxlJcA89bRxwBLPlpw1SIYhLtAvECAFuDFuMS35s evfhDs1rDo2VDaPDr4VYtpjn4fesTwTkCakIRGXkfVcMuZkx2atgWJk0kub76a43dpT3 PU50i/ObOcdfmVEtUq/QcCWilJXtT4pCYX/GoxEN25kO3R9mclYLGoXFCtv9hAzv0TOE LZLx+0FKp4DRruCyVilmbPHRnzG9YwzWt7waEALa8F8H0RmfpHdQ547OEbf1SlqrptyT KWPg== X-Gm-Message-State: AOJu0Yx9cegK9+2nG7sLDJuLmgCmy9pUlHTcDh1P7V2wNbUMbAcjiwmd 9riMuwOAo/XSl5yyhmqCLEsk7CO++DrEfT8p+hoRmp3zvYdkjLxZB8AU2bSeKKI= X-Received: by 2002:a17:90a:ea0b:b0:2a1:2506:b937 with SMTP id w11-20020a17090aea0b00b002a12506b937mr375371pjy.23.1712078935365; Tue, 02 Apr 2024 10:28:55 -0700 (PDT) Return-Path: Received: from dc4670097af0.alm.mentorg.com (nat-mam.mentorg.com. [139.181.8.34]) by smtp.googlemail.com with ESMTPSA id bt19-20020a17090af01300b002a2313f62dcsm66705pjb.0.2024.04.02.10.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 10:28:55 -0700 (PDT) From: kergoth@gmail.com X-Google-Original-From: chris.larson@siemens.com To: isar-users@googlegroups.com Cc: Christopher Larson Subject: [PATCH 0/9] Add more signature cachability tests to the testsuite Date: Tue, 2 Apr 2024 17:28:04 +0000 Message-Id: <20240402172813.418770-1-chris.larson@siemens.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: KtgckzIBmoGn From: Christopher Larson This series improves isar-sstate's lint command to support checking sigdata in tmp/stamps and to check for absolute paths in SRC_URI, and adds an additional test to the testsuite to exercise this capability. The new test checks for signature cachability issues much more quickly, as it does so without a full build, so it checks more target configurations. This uses bitbake's `-S none` option to avoid building anything, and then runs isar-sstate lint on the resulting sigdata in tmp/stamps. This test will accept a verbose parameter on the command-line to pass --verbose to isar-sstate lint, which will show the "other" absolute paths found, not just source and build directory. This series also fixes two of the failures which were identified by the new test. Here we can see the issues identified by the new test, which are fixed in this series: ``` $ avocado run testsuite/citest.py -t signatures --max-parallel-tasks=1 JOB ID : 3a3308967946663d9b239f638b030502fa80ef0a JOB LOG : /builder/avocado/job-results/job-2024-03-29T19.24-3a33089/job.log (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: STARTED (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: FAIL: Detected cachability issues (42.93 s) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 44.63 s Test summary: 1-testsuite/citest.py:SignatureTest.test_signature_lint: FAIL ``` Applicable section of the avocado full.log: ``` 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| found cachability issues (scanned 602 signatures) 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:10,818 avocado.test cibase L0142 ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` If we strip off the prefix for legibility: ``` ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) ==== ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum=36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| ==== issues found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) ==== ERROR| -> path in sources-dir: TESTSUITEDIR = "/home/kergoth/Code/industrial/signatures/isar/testsuite" ERROR| found cachability issues (scanned 602 signatures) ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` The sstate/signatures tests were run for each commit in this series using `git rebase -x` to ensure it does not break a bisect. I welcome any and all feedback on this. In particular I don't love extending the hardcoded list of image tasks being ignored in isar-sstate lint, but it's a start. I'm open to suggestions. Christopher Larson (9): isar-bootstrap: avoid forced early expansion of key vars isar-ci-ssh-setup: avoid abs path in signatures isar-sstate: lint: check for absolute paths in SRC_URI isar-sstate: lint: add support for checking stamps isar-sstate: lint: ignore more image tasks isar-sstate: add --excluded-tasks argument cibuilder.py: add -S support to the bitbake method testsuite: add perform_signature_lint method testsuite: add signature cachability checks .../isar-ci-ssh-setup_0.1.bb | 3 + .../isar-bootstrap/isar-bootstrap.inc | 8 ++- scripts/isar-sstate | 60 +++++++++++++++---- testsuite/cibase.py | 19 ++++++ testsuite/cibuilder.py | 5 +- testsuite/citest.py | 22 ++++++- 6 files changed, 100 insertions(+), 17 deletions(-) -- 2.39.2