From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7353323043876241408 X-Received: by 2002:a05:6a00:cd3:b0:6ea:bbff:7561 with SMTP id b19-20020a056a000cd300b006eabbff7561mr135998pfv.11.1712851832943; Thu, 11 Apr 2024 09:10:32 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6a00:2d97:b0:6ed:41e2:9c9e with SMTP id fb23-20020a056a002d9700b006ed41e29c9els18820pfb.2.-pod-prod-05-us; Thu, 11 Apr 2024 09:10:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEJ9btok/0UwkT6oj3WwY7Op//zEOr6AoAikym1z6hW3J6A9Kf+2mUJc3ABnnKkjyq+flNj X-Received: by 2002:a05:6a00:391c:b0:6ec:da6c:fd42 with SMTP id fh28-20020a056a00391c00b006ecda6cfd42mr189799pfb.5.1712851831159; Thu, 11 Apr 2024 09:10:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712851831; cv=none; d=google.com; s=arc-20160816; b=LoHqNe5MK6XonZH1qP0slzD2FW7R92NWp/88sIpXZ/czUXYTfaQ8vHO2fA8hUtdda4 q0mVqIR1TK2UxwZOLsHrANl8LH5t5ihW2gPN8LHP/ogigPcsW4sc+ECPzDCHJBx66vti 3VAWTJUjEfDTWOzue0T3MX4vOC6NVZ/Bvqrxk7HiGPW9xspyEwEB9o6/QUt5eg4oVZy1 5XzgwZz9kAXklKwx1oE/wjfAkVU9xKm3RJfbwx9jFikd4faSz7N2Rubg1sThLRiQA8Ee A9XQyAt4u0Fz/mQ+IJ4zqT0Z0OgtiEdjMHcQmerCvCeOLh62C7H5+dXjZvMGdaahbsWZ Qpbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:msip_labels:content-language :accept-language:in-reply-to:references:message-id:date:thread-index :thread-topic:subject:to:from:dkim-signature; bh=p/2TkYmOG9nOEVY2HSR63MVY5x+ROuSiPqIwROduH9g=; fh=OCLipNJhBAjuYZVWe88RgIbTd+C/BiXjPppf4/h0OyA=; b=ohM9KHkMIMQ8yres472rMQVFU6Oxlq97OPqj3Bcpw/1RtRGMa1d6aPduweuyZR3++b +oSKdorySPanT+A9J9isc9sV7q/5Iq2QGe+WF0mLoNvdkHJw5ms+Xale3S47piUWVbk3 YUpqRUVqQDayI1N2uxHtFcGxKexPG1lB5zzjyxsyQVXYEv2T0ctfUGQ+C0AFAH8x5cDq LVtx6TdttYQ5r5OKqB1gacywBFUui6El+sJxaS4ckRCkQGuAS1Mcdsmb9nU4XZvlSdx+ TmvsgbBkqrtay6YPeh+Mmf9X7iShZXlIq7evbjQCXORe7SjMJf+5JjImZz/wqyu0UDoQ +sWA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=kSxN6Z5S; spf=pass (google.com: domain of chris.larson@siemens.com designates 2a01:111:f400:7e88::617 as permitted sender) smtp.mailfrom=chris.larson@siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Return-Path: Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on20617.outbound.protection.outlook.com. [2a01:111:f400:7e88::617]) by gmr-mx.google.com with ESMTPS id x27-20020a056a00189b00b006ed2f54b0b5si101787pfh.4.2024.04.11.09.10.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Apr 2024 09:10:31 -0700 (PDT) Received-SPF: pass (google.com: domain of chris.larson@siemens.com designates 2a01:111:f400:7e88::617 as permitted sender) client-ip=2a01:111:f400:7e88::617; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=kSxN6Z5S; spf=pass (google.com: domain of chris.larson@siemens.com designates 2a01:111:f400:7e88::617 as permitted sender) smtp.mailfrom=chris.larson@siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p/2TkYmOG9nOEVY2HSR63MVY5x+ROuSiPqIwROduH9g=; b=kSxN6Z5SIK475RhKLwj0Ndlziz4wwW4rmH+ssRCRSGR7eUf5ZrnpB9hjieXTmLMdl4bhmFeeWXbFbn2ZCNmM9Bk20IcBq2ETbjERnAal6vBENSH21/T/F0JHJaCJW6IvzzrxYh0icfTKG/LZaBBo0nkI7TLKghwZC+dMASx0+wYsHYHucHOXsxBgg9o2gNUSaIpgR6Gmt+hFS0hUK3UzPRmxmfAuRBxZy794RIeBjDHFVP8t/G+070gK55uHlszZT+xpfL8m/drrzR617fRJCUeDYYD5BrkYYB443mFHUpOGagM511id66phu1PZyWAjQa/jH4VRHbKhQMf5RPMn7w== Received: from BN0PR07MB8375.namprd07.prod.outlook.com (2603:10b6:408:12e::24) by MN2PR07MB6990.namprd07.prod.outlook.com (2603:10b6:208:1a8::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.55; Thu, 11 Apr 2024 16:10:29 +0000 Received: from BN0PR07MB8375.namprd07.prod.outlook.com ([fe80::a429:2b28:36e4:89d0]) by BN0PR07MB8375.namprd07.prod.outlook.com ([fe80::a429:2b28:36e4:89d0%7]) with mapi id 15.20.7409.042; Thu, 11 Apr 2024 16:10:29 +0000 From: "Larson, Chris" To: "isar-users@googlegroups.com" Subject: RE: [PATCHv2 0/9] Add more signature cachability tests to the testsuite Thread-Topic: [PATCHv2 0/9] Add more signature cachability tests to the testsuite Thread-Index: AQHah3bYtgCwCs5XkE6QCGwZonxYurFjRoFA Date: Thu, 11 Apr 2024 16:10:29 +0000 Message-ID: References: <20240405163135.2987489-1-chris.larson@siemens.com> In-Reply-To: <20240405163135.2987489-1-chris.larson@siemens.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_ActionId=d2a7a152-f0cc-4079-aaf5-60ea6189761b;MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_ContentBits=0;MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_Enabled=true;MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_Method=Standard;MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_Name=restricted;MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_SetDate=2024-04-11T16:10:06Z;MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_SiteId=38ae3bcd-9579-4fd4-adda-b42e1495d55a; authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN0PR07MB8375:EE_|MN2PR07MB6990:EE_ x-ms-office365-filtering-correlation-id: 76479dec-8912-4608-972d-08dc5a41e7f8 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: zqQJ7GzvxU1rW0N/Kv2AUkYtNxYRjikoEL/mIDWAidr3dcwBY6eVyciKf9VuRUSVVqlAiiZ4LHIUzT7M/psLC2goDPT6fY13UfwOW6dnWj8vYbojEkfcx4biecCTz+bu0f7boHXQm3tYibfJTV/8P6p810DFrEOLtRDdDi/y7pYCjwYPbpThco3Ci5L+PQFMNwVAEBfOgqpELXdTcV1/Dhz/nt4LZYiaMFgZ2u6b7GgpFZ085JbXm/GBWB3SQbJqHGgxtp2h74kmvo416CUpq8nMFVOIVMvo82QAAbph1KUVsozTp2WSVadOi7hQ8G9qMb9YWQ4yrhpW2iAmIoul6E23X+om+nfGa2fc/ykrGwAnoNTp84DEcSxq/lCT72b+Nnnp76lxlLEk9ycyAWcEFeb9l6m06NVg2rnGK22ZuD4Va7doiReCX4mm+JKXJT6O3TZ2EByqtiRUJQraphgd0yaqlx3sEmVHeNi7HWf9KQUmhLw5tFhkj7mTkN6PcyYmKJAyFndFgTWUWKHoZTCdHOqp5UhOhEf94KDBPLvSJEGTVVAk1MUPp52lkE9teEzq8hMWweJbdxWWMwoLpaMcodM5LNRFzZCyWnuA6F9y7dNPYhiVNR2X4DFnPvneSYMbs436vWdRq3cPU2yJYR7gteTFd9AwnV5zGieszZ2oc1o= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN0PR07MB8375.namprd07.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(376005)(1800799015)(38070700009);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?903d7cOOCsZ66D5lQVuVwvQabx78nsWngEGqOTnPAtvPtNyjkkAFLAc+Gu2h?= =?us-ascii?Q?lXSVCHHLClVUWTHPh4rl9fc5TONndSJJQzzpaAQXxkkcSOKHz7YzzuqKA4f7?= =?us-ascii?Q?TH8KiBnOt+wmj7M5yUuwB/RJFDkaVUwp6kcI/UkmLWiOD+smpVtmZ9TzYpr5?= =?us-ascii?Q?sSh0jXf7LJtPTaK/qPfk94GDkZjhRWJwt8+XgUah67YWtrOZXIibp/+9wmpt?= =?us-ascii?Q?bJu1OmIXQ2rPYLPe0RBZwfQurjJK7sMBYzLb1bANi7EaoTdY8irZbLmK8YQb?= =?us-ascii?Q?Vd8K6wtYc1WVxHQUvq5+fC46bbMG8vKgOoGW9D+dHKnbaKX+ej0xdMQkH20R?= =?us-ascii?Q?HxZ2MaAl9tY/ZVc2fg7NPyXl90lhhmWXcDArNfvT9RKdzrRiEjlVr3ZZwK/n?= =?us-ascii?Q?KFc6i5rHEESW932P/A/o+0i/h1y2C+NlwP6F4IICpEgTyFnSHKw7Xv0VNluL?= =?us-ascii?Q?LL1bvNYOZ23GYpKIlpgfz9BthiyPZKr5OF1kDKYPq+jVqmZxm0ZoAlfFzaES?= =?us-ascii?Q?VOM/7cuGORBd/kPYdRzX3jQEu0Aue9FJTY6x5GWMwJAU5dLOXKCf50PQfWJ+?= =?us-ascii?Q?C2W1Jy8wZHiG5q9tQA2gxMPhqvFDisa8Osc7UOewgZpB/WHdIyokQt6RnyZJ?= =?us-ascii?Q?+3dXicjYt4VOn+wg3MLYUXOF2eXRlZ3/mAAe3rfgcTWDm9I/WH79qWcEcmQp?= =?us-ascii?Q?uPa+hNrLEV5rasu8464w+yQUkzHrcCYVJqlqLOEniKyAXB8OIFmhIsA/bD2g?= =?us-ascii?Q?Sr1xPcQiaFZtNW6xRI/fmo/JkTxSpv08CrJmykp1hZeOXZWX1l6dpfi0h4Xh?= =?us-ascii?Q?V1OZrC4DBOjMSkEzYAJIKsXjm4nPcYknTwLU+Y+9qoNLVDqnvmlZNzXk3OY3?= =?us-ascii?Q?Zv5sWT4fIgqEWX7bZVgUd+qtOmmgm2oTH6zugESKy2w3AQIi2tjKGg7Ch/tT?= =?us-ascii?Q?OlWhpppPZDJfai0/huHbyapXYZsnuO9o+Vk+B4w8UYcNvLS6YCuovdWGAsmT?= =?us-ascii?Q?VQn43DYMWuZUfJcYhtrStFnsJiNBAac2GjFP2Abj+GGR0/A49PeC7Vumx26R?= =?us-ascii?Q?SiC01NLRsoRrMYftK+Mm4DXxYwDXDwk/5oz5PXjp+ImjAdOSpUrPtwHvnLpo?= =?us-ascii?Q?qMjDuIfEOyJOu6JZxV9TZ+FmpiWk5/NB6aXYJ5FsXBEIl8e2mwh+ZKykbD5E?= =?us-ascii?Q?L+rABDcmR5x/wzjB8uR4mxEqPBuhZvSG5B2tZeYRLUr2VVINryFQ0aR/g1Jr?= =?us-ascii?Q?6n8rsHeRHRuFgF3vugf2Kl96YgOJw9KYIIHUkTUriCvkP9z8E4V8RgCc72w1?= =?us-ascii?Q?3isFU3gmG2n4RdH8uxgZzoN5AjXV8w81Coi05iiFPVGFoTJ/qRIhIiseVLfn?= =?us-ascii?Q?U0sWIp5t9GsPerKMNuqL+juuDGRlAS+c3XHo88k1DTsLjamJ6SAkaENOA69D?= =?us-ascii?Q?m+OvM1Z+qIht/39g1VwXzvAqpcP9Zd3Z4MF67nOx8PUNiIF3sPoI5V6A2q8e?= =?us-ascii?Q?C/uJNaOGc9XF04XN67TP8NTcNsMs/V3Xu+OwYwZ/UOvPsDU5tLY8J2pWmhMK?= =?us-ascii?Q?2gqUTTwNhKcp/+Kq7ooVTiYkelBjWh92m5amP0Ny?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0PR07MB8375.namprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 76479dec-8912-4608-972d-08dc5a41e7f8 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Apr 2024 16:10:29.0821 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 77fOcHqQfX01G4TOxVIoa8J88mPtPeU1IntCl0JdSbReHuS8i6ezYsI+c1pvJ3v2IbZkH+Mv8Nw95lKD7Ui0+yzm8n3uhz9s9tksbt461so= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR07MB6990 X-TUID: 8or+PiktE2AZ Were there any issues with the v2 updates? Thanks. -- Christopher Larson Siemens AG www.siemens.com -----Original Message----- From: Larson, Chris (DI CTO FDS CES LX MEL) =20 Sent: Friday, April 5, 2024 9:31 AM To: isar-users@googlegroups.com Cc: Larson, Chris (DI CTO FDS CES LX MEL) Subject: [PATCHv2 0/9] Add more signature cachability tests to the testsuit= e From: Christopher Larson This series improves isar-sstate's lint command to support checking sigdata= in tmp/stamps and to check for absolute paths in SRC_URI, and adds an addi= tional test to the testsuite to exercise this capability. The new test chec= ks for signature cachability issues much more quickly, as it does so withou= t a full build, so it checks more target configurations. This uses bitbake'= s `-S none` option to avoid building anything, and then runs isar-sstate li= nt on the resulting sigdata in tmp/stamps. This test will accept a verbose parameter on the command-line to pass --ver= bose to isar-sstate lint, which will show the "other" absolute paths found,= not just source and build directory. This series also fixes two of the fai= lures which were identified by the new test. Here we can see the issues identified by the new test, which are fixed in t= his series: ``` $ avocado run testsuite/citest.py -t signatures --max-parallel-tasks=3D1 JOB ID : 3a3308967946663d9b239f638b030502fa80ef0a JOB LOG : /builder/avocado/job-results/job-2024-03-29T19.24-3a33089/job.= log (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: STARTED (1/1) testsuite/citest.py:SignatureTest.test_signature_lint: FAIL: Detecte= d cachability issues (42.93 s) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CA= NCEL 0 JOB TIME : 44.63 s Test summary: 1-testsuite/citest.py:SignatureTest.test_signature_lint: FAIL ``` Applicable section of the avocado full.log: ``` 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| =3D=3D=3D=3D issues found = in ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) =3D=3D=3D=3D 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SR= C_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar= /conf/distro/ubuntu.public.key;sha256sum=3D36a38199a4bf4eae1e7f574891f7dfcb= 79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| =3D=3D=3D=3D issues found = in ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) =3D=3D=3D=3D 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: SR= C_URI entry "file:///home/kergoth/Code/industrial/signatures/isar/meta-isar= /conf/distro/ubuntu.public.key;sha256sum=3D36a38199a4bf4eae1e7f574891f7dfcb= 79b91b87a33a499383265e1224b5e989" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| =3D=3D=3D=3D issues found = in debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) =3D=3D=3D=3D 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| -> path in sources-dir: TE= STSUITEDIR =3D "/home/kergoth/Code/industrial/signatures/isar/testsuite" 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| found cachability issues (= scanned 602 signatures) 1-testsuite/citest.py:SignatureTest.test_signature_lint: 2024-03-29 19:25:1= 0,818 avocado.test cibase L0142 ERROR| -> absolute paths: sources= -dir 3, build-dir 0, other 184 ``` If we strip off the prefix for legibility: ``` ERROR| =3D=3D=3D=3D issues found in=20 ERROR| ubuntu-focal-amd64:isar-bootstrap-target:unpack (95af4581) =3D=3D=3D= =3D ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/ind= ustrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum= =3D36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| =3D=3D=3D=3D issues found in=20 ERROR| ubuntu-focal-amd64:isar-bootstrap-target:fetch (e8249cb2) =3D=3D=3D= =3D ERROR| -> path in sources-dir: SRC_URI entry "file:///home/kergoth/Code/ind= ustrial/signatures/isar/meta-isar/conf/distro/ubuntu.public.key;sha256sum= =3D36a38199a4bf4eae1e7f574891f7dfcb79b91b87a33a499383265e1224b5e989" ERROR| =3D=3D=3D=3D issues found in=20 ERROR| debian-bullseye-amd64:isar-ci-ssh-setup:install (1c0a8b21) =3D=3D=3D= =3D ERROR| -> path in sources-dir: TESTSUITEDIR =3D "/home/kergoth/Code/industr= ial/signatures/isar/testsuite" ERROR| found cachability issues (scanned 602 signatures) ERROR| -> absolute paths: sources-dir 3, build-dir 0, other 184 ``` The sstate/signatures tests were run for each commit in this series using `= git rebase -x` to ensure it does not break a bisect. I welcome any and all feedback on this. v2 updates: - Explain the use of --dump-signatures in the docs on the lint command in i= sar-sstate - Explain the stamps regex the way the sstate regex is explained, for consi= stency - Support wildcards for the --excluded-tasks argument and default exclusion= s - Drop the additional default task exclusions, as this should be revisited - Fix a lint traceback if a variable's value is only whitespacxe Christopher Larson (9): isar-bootstrap: avoid forced early expansion of key vars isar-ci-ssh-setup: avoid abs path in signatures isar-sstate: lint: check for absolute paths in SRC_URI isar-sstate: lint: add support for checking stamps isar-sstate: add --excluded-tasks argument isar-sstate: fix failures if a variable is set to just whitespace cibuilder.py: add -S support to the bitbake method testsuite: add perform_signature_lint method testsuite: add signature cachability checks .../isar-ci-ssh-setup_0.1.bb | 3 + .../isar-bootstrap/isar-bootstrap.inc | 8 +- scripts/isar-sstate | 85 +++++++++++++++---- testsuite/cibase.py | 19 +++++ testsuite/cibuilder.py | 5 +- testsuite/citest.py | 22 ++++- 6 files changed, 121 insertions(+), 21 deletions(-) -- 2.39.2