From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7353323043876241408 X-Forwarded-Encrypted: i=2; AJvYcCXIiWxE8qs0lYAuwe2C8axomysCYSiE/h96WuCXAmMo5aOd/fXQhlf/N1J6FPI5Tvzet3c58L2/2jUK+lSqJNEaQUieM7l80pemyzk= X-Received: by 2002:a05:600c:444d:b0:416:6a8a:d84c with SMTP id v13-20020a05600c444d00b004166a8ad84cmr7766848wmn.38.1713173270809; Mon, 15 Apr 2024 02:27:50 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:3b94:b0:416:41a8:86ba with SMTP id n20-20020a05600c3b9400b0041641a886bals1499736wms.0.-pod-prod-07-eu; Mon, 15 Apr 2024 02:27:48 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWQzswr69Thp7uLVdoNvsCVj1+CRecscMUrMM9rOYsydzSV/Ma12PNwo8lKHtui4PTOrOakM2idEUhaa3ef8p0StISrm4HjYUKcHbg= X-Google-Smtp-Source: AGHT+IHJnP+pI5W2Idhiq8kzgTEbzTXe5whwsG+eEGkJiPcx3i/h3Z16yPjp1aX2J0fi0xCwYfLU X-Received: by 2002:adf:e25d:0:b0:343:e824:d798 with SMTP id bl29-20020adfe25d000000b00343e824d798mr5402635wrb.62.1713173268509; Mon, 15 Apr 2024 02:27:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1713173268; cv=none; d=google.com; s=arc-20160816; b=nCm3kCeDmPeHQt25IVQH2gQ7xAEfo7bQqMPMwu6Y6LUplujyEVH3V0rmlKKnTRBGdG P5KOUsNPN5S3UZoA5xdOobr5pVuxlqB0yvK9O/7ivcdD9QWvGIPprOuWRQQ0fKTOYEV6 YCGmK6MaNhzgBj0OIj2yc+Qm+fIN+Buwhr82PTgBtYOwIyjRCwb3fTXuPeCOSCZakMZS GB3L1r/HxvL3tnkle9JD6DIWwuQC7Y1n+5VFfwAQA7ThvNxBw/9ys/cyDs5D8PbgvDfu VKPrIoYZm4EstsskaFkeej2Q2p99qhwD5/ubMFu6ng9lL3Yh9ttYPlsURg/CkjkBq+FR WMrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id; bh=vh06YUFie1E393dbHOyIJSElQTrxWpa6X5klp8DJeN4=; fh=jbYBnisNYSJvRlbHg/ujTMhygJ4x63rwn8SYi3Sx8Qw=; b=Fp/bWyfMBSy3jhsvNDihCJ6rzPysh1bIaNdHgX1qzrfG4Ts1th59u4DzMAQCMB2cAX LnoDo0UuJo17Y7VZkwy7eAc7Sbrbmv7lX3AUqXPe2cPjGkkoGWXje+Txq2uT2pV4vT5t Pd8cxXVXL3n3rGnnvA5Wvrp7rYLBny7zU+x/uFrb9U95ntUArZS0z9O4H/CEQnTDq0+u lFDunIEccvmFxPLxJMyuWJDFvQXtZY/yH8Ab94NxZsyrEB99LHeLltAreCcQ0Tj6oOWd zTHPzrSFsynhSEsDnN6ZwnU5Je3aGa1/+luSfkHZIUzEjyGI1A10tIIdtMRtnfnk1ul0 MJMA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id c9-20020a056000184900b00341c9bc6836si231391wri.3.2024.04.15.02.27.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 15 Apr 2024 02:27:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 43F9RjhX017304 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 15 Apr 2024 11:27:47 +0200 Message-ID: <40ba6a7a77eb180cd8d056955cd09d36562e65e6.camel@ilbers.de> Subject: Re: [PATCHv2 0/9] Add more signature cachability tests to the testsuite From: Uladzimir Bely To: chris.larson@siemens.com, isar-users@googlegroups.com Date: Mon, 15 Apr 2024 12:27:45 +0300 In-Reply-To: <20240405163135.2987489-1-chris.larson@siemens.com> References: <20240405163135.2987489-1-chris.larson@siemens.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.52.0 (by Flathub.org) MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: MoaAPYuzEz0f On Fri, 2024-04-05 at 16:31 +0000, chris.larson via isar-users wrote: > From: Christopher Larson >=20 > This series improves isar-sstate's lint command to support checking > sigdata in tmp/stamps and to check for absolute paths in SRC_URI, and > adds an additional test to the testsuite to exercise this capability. > The new test checks for signature cachability issues much more > quickly, as it does so without a full build, so it checks more target > configurations. This uses bitbake's `-S none` option to avoid > building anything, and then runs isar-sstate lint on the resulting > sigdata in tmp/stamps. >=20 > This test will accept a verbose parameter on the command-line to pass > --verbose to isar-sstate lint, which will show the "other" absolute > paths found, not just source and build directory. This series also > fixes two of the failures which were identified by the new test. >=20 > Here we can see the issues identified by the new test, which are > fixed in this series: > ``` > $ avocado run testsuite/citest.py -t signatures --max-parallel- > tasks=3D1 > JOB ID=C2=A0=C2=A0=C2=A0=C2=A0 : 3a3308967946663d9b239f638b030502fa80ef0a > JOB LOG=C2=A0=C2=A0=C2=A0 : /builder/avocado/job-results/job-2024-03-29T1= 9.24- > 3a33089/job.log > =C2=A0(1/1) testsuite/citest.py:SignatureTest.test_signature_lint: STARTE= D > =C2=A0(1/1) testsuite/citest.py:SignatureTest.test_signature_lint: FAIL: > Detected cachability issues (42.93 s) > RESULTS=C2=A0=C2=A0=C2=A0 : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 |= INTERRUPT > 0 | CANCEL 0 > JOB TIME=C2=A0=C2=A0 : 44.63 s >=20 > Test summary: > 1-testsuite/citest.py:SignatureTest.test_signature_lint: FAIL > ``` >=20 > Applicable section of the avocado full.log: > ``` > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| =3D=3D=3D=3D issues > found in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) > =3D=3D=3D=3D > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| -> path in > sources-dir: SRC_URI entry > "file:///home/kergoth/Code/industrial/signatures/isar/meta- > isar/conf/distro/ubuntu.public.key;sha256sum=3D36a38199a4bf4eae1e7f5748 > 91f7dfcb79b91b87a33a499383265e1224b5e989" > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| =3D=3D=3D=3D issues > found in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) > =3D=3D=3D=3D > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| -> path in > sources-dir: SRC_URI entry > "file:///home/kergoth/Code/industrial/signatures/isar/meta- > isar/conf/distro/ubuntu.public.key;sha256sum=3D36a38199a4bf4eae1e7f5748 > 91f7dfcb79b91b87a33a499383265e1224b5e989" > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| =3D=3D=3D=3D issues > found in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) > =3D=3D=3D=3D > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| -> path in > sources-dir: TESTSUITEDIR =3D > "/home/kergoth/Code/industrial/signatures/isar/testsuite" > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| found > cachability issues (scanned 602 signatures) > 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 > 19:25:10,818 avocado.test cibase=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 L0142 ERROR| -> absolute > paths: sources-dir 3, build-dir 0, other 184 > ``` >=20 > If we strip off the prefix for legibility: > ``` > ERROR| =3D=3D=3D=3D issues found in ubuntu-focal-amd64:isar-bootstrap- > target:unpack (95af4581) =3D=3D=3D=3D > ERROR| -> path in sources-dir: SRC_URI entry > "file:///home/kergoth/Code/industrial/signatures/isar/meta- > isar/conf/distro/ubuntu.public.key;sha256sum=3D36a38199a4bf4eae1e7f5748 > 91f7dfcb79b91b87a33a499383265e1224b5e989" > ERROR| =3D=3D=3D=3D issues found in ubuntu-focal-amd64:isar-bootstrap- > target:fetch (e8249cb2) =3D=3D=3D=3D > ERROR| -> path in sources-dir: SRC_URI entry > "file:///home/kergoth/Code/industrial/signatures/isar/meta- > isar/conf/distro/ubuntu.public.key;sha256sum=3D36a38199a4bf4eae1e7f5748 > 91f7dfcb79b91b87a33a499383265e1224b5e989" > ERROR| =3D=3D=3D=3D issues found in debian-bullseye-amd64:isar-ci-ssh- > setup:install (1c0a8b21) =3D=3D=3D=3D > ERROR| -> path in sources-dir: TESTSUITEDIR =3D > "/home/kergoth/Code/industrial/signatures/isar/testsuite" > ERROR| found cachability issues (scanned 602 signatures) > ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 > ``` >=20 > The sstate/signatures tests were run for each commit in this series > using `git rebase -x` to ensure it does not break a bisect. >=20 > I welcome any and all feedback on this. >=20 > v2 updates: > - Explain the use of --dump-signatures in the docs on the lint > command in isar-sstate > - Explain the stamps regex the way the sstate regex is explained, for > consistency > - Support wildcards for the --excluded-tasks argument and default > exclusions > - Drop the additional default task exclusions, as this should be > revisited > - Fix a lint traceback if a variable's value is only whitespacxe >=20 > Christopher Larson (9): > =C2=A0 isar-bootstrap: avoid forced early expansion of key vars > =C2=A0 isar-ci-ssh-setup: avoid abs path in signatures > =C2=A0 isar-sstate: lint: check for absolute paths in SRC_URI > =C2=A0 isar-sstate: lint: add support for checking stamps > =C2=A0 isar-sstate: add --excluded-tasks argument > =C2=A0 isar-sstate: fix failures if a variable is set to just whitespace > =C2=A0 cibuilder.py: add -S support to the bitbake method > =C2=A0 testsuite: add perform_signature_lint method > =C2=A0 testsuite: add signature cachability checks >=20 > =C2=A0.../isar-ci-ssh-setup_0.1.bb=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 3 + > =C2=A0.../isar-bootstrap/isar-bootstrap.inc=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 8 +- > =C2=A0scripts/isar-sstate=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 85 +++++++++++++++-- > -- > =C2=A0testsuite/cibase.py=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 19 +++++ > =C2=A0testsuite/cibuilder.py=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 5 +- > =C2=A0testsuite/citest.py=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 22 ++++- > =C2=A06 files changed, 121 insertions(+), 21 deletions(-) >=20 > --=20 > 2.39.2 >=20 Applied to next, thanks.