From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7095678487044816896 X-Received: by 2002:a17:907:9487:b0:6f8:747a:845e with SMTP id dm7-20020a170907948700b006f8747a845emr8055452ejc.299.1652091394959; Mon, 09 May 2022 03:16:34 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:907:3e11:b0:6f5:1321:37ec with SMTP id hp17-20020a1709073e1100b006f5132137ecls4119022ejc.11.gmail; Mon, 09 May 2022 03:16:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyjiK7vczEp6/RmS0VwP8HevZoV+C1d1XxKqWSYG4v0zc35G2qyeiu+8+Bq5bw2eZBxDwob X-Received: by 2002:a17:907:2394:b0:6f5:108c:a43 with SMTP id vf20-20020a170907239400b006f5108c0a43mr13828529ejb.28.1652091393958; Mon, 09 May 2022 03:16:33 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1652091393; cv=pass; d=google.com; s=arc-20160816; b=bSelfYUSIXUvja08nJBhy3xAfsceKTCvRswWNPsli69UmH+gpB1xysZhaYJRK0RXLb THLG0MKeB1B4gxiCPiiDBfyJgTJKApZTz6/wmmtoHfBaTTcm5a4uKYTk0Ejnm2N9ouJm k9n+058a5YvZl6S2Z9d6w7tQIFWTxbufaP7Q4bVZW12OKRFAmOy7zuAbclfAIqalFwjU YsdGu5BbITwunabgzVJnOvKBB/DTSKny5aJwtTfnI/NLewjBfyOPL2WAHmmXwSBBnyp8 ax47yzVS0vfcsMcMgefen77v1Z0kWkxjaJv75BonwqXjg1enFv8bg4OCYpmdBBAHd0Yt ngnQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=Ge02Nudm5zzdK+VXUw1/C6pTnZ+FXc1MQU53qtSZKvI=; b=S4hzkcSDviASXIVNSp5Qp2sNJ8+DphpXOTbFdYS8/O0HC/XfkH0I95GX9SoL2CJadc EKOjw65YW/JD2NnvoOJIn5m2Y9GoRG7p5A1QmUj3dDawo10Ny896gW5t+Ky2xxGpSVL3 EmeJAaUc3VXqA6Cj+9hLmKfKQQ+RxKdzwA+ZLXCMbcbc4qoCE3naDCpDySrU8Xzz4lq5 iVI7828+0lWMtmRP7R90HdurjEZ1dc+mzzaCj94N60VRSzwrPxIP7vRB1mHZnzBaip+O Nexkl8ZTz5UgFe+V+QeG20j/7OjhFYsLaoNgczQGqyS/EwLT3FVo4/snP/PtQ3d/aAmT ANLg== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=h0j+SgTv; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:7e1b::629 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on20629.outbound.protection.outlook.com. [2a01:111:f400:7e1b::629]) by gmr-mx.google.com with ESMTPS id y4-20020aa7ccc4000000b00425b0722545si722856edt.3.2022.05.09.03.16.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 May 2022 03:16:33 -0700 (PDT) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:7e1b::629 as permitted sender) client-ip=2a01:111:f400:7e1b::629; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=h0j+SgTv; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:7e1b::629 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XaPQUI5OGluXJiXnlFFj7sfF5+hiBz1id6FK3Nf9CKxSGKPHcm6ObFPB/MdxTf1VJ74lSnLOVs+SniCX0ZcFcun9BFl4WFlDiS4SZbM6omzqz+FipDsL2/Miv0yRynM5gDKsWAUCPk2fFfo5a9zM4BrQcN/WJqI+q0EnJv6epnah4T2rqAJBuZH/u30MZ0kyhFHovkoWtCG1ZHL/CL8eAG/0HF7oeiZ3IHTEUyz4rxLcUZ3yuerbFiQas78W/Iz16myNRRrDMA0LvjVVRBIk5yfbqTeipwp59rIlrZe6svcQxpoIqxz10wBwGB8vq0HR+rlTMPeW7arUli5S4Wa1iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ge02Nudm5zzdK+VXUw1/C6pTnZ+FXc1MQU53qtSZKvI=; b=QSrQNIxrPaDF2EFRDuGzsCIXQjpGJ2zeviXFpaVbetpeHhR1R5VOnKqweNcbLuYF3gA4lxI5jUY2Q0MTBc3/wnvGVP+74mk5BHNgRuUXcd4HC0QVpBVB8XLYZRmTDzMxhucvYAqgaOOxZUl7CeQozxlr4cK54X3G91QBIUzBKkbx5VciOBWJwIeIhqU79CtbMa0IOZ0ufxx8lXF9tAMLL/7sLd+oOpf7MKmnIQGYLbe2+gYL3Y4RsjGlxiiVd+wxktHkakzJeY03yzPXjFCSpyLl7cw8EZrfl06p6XVNVTdsJ39b+ZAe04XOSu4Tkg5IL2XoW/nF/SuuG2dLz7SBIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.70) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ge02Nudm5zzdK+VXUw1/C6pTnZ+FXc1MQU53qtSZKvI=; b=h0j+SgTvBNuVGe4O0udmZ7BrvI7uoTzklHs0mUjOiNV1qF/bOX8mqtPTwNg6shUgUJWneCasxg2/RGWQiPHWkx1MurY+I+qQK7YPwF41NygTNYOJ46DPbE+y5VP8Y/wPhs3Z6ncJ7i8P94I1WhhfLApzm3axcICrLKqb9VhQ6IlFimGHx1ux2UhQdUGM5CeaGA9YLbN3ZllQqff1Srkh5QCrG+kg5X4sB7Z66+WmSFIRL2g5LRJ7huhBPpQYh/kKCdvSPYxTRQdshqi8HB+kBR3h+iw5KQFqDBcemzLIav1DDCYJTPtTVoj7HWPqepH2ov7r8N+lhnL/Wslr33q9rw== Received: from AS8PR04CA0166.eurprd04.prod.outlook.com (2603:10a6:20b:331::21) by AS4PR10MB6136.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:587::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.20; Mon, 9 May 2022 10:16:33 +0000 Received: from VE1EUR01FT058.eop-EUR01.prod.protection.outlook.com (2603:10a6:20b:331:cafe::24) by AS8PR04CA0166.outlook.office365.com (2603:10a6:20b:331::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.23 via Frontend Transport; Mon, 9 May 2022 10:16:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.70) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.70 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.70; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.70) by VE1EUR01FT058.mail.protection.outlook.com (10.152.3.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5227.15 via Frontend Transport; Mon, 9 May 2022 10:16:32 +0000 Received: from DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) by DEMCHDC9SJA.ad011.siemens.net (194.138.21.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 9 May 2022 12:16:32 +0200 Received: from random.ppmd.siemens.net (139.25.68.25) by DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 9 May 2022 12:16:32 +0200 From: Adriaan Schmidt To: CC: Adriaan Schmidt Subject: [PATCH v2 2/7] bitbake-diffsigs: make finding of changed signatures more robust Date: Mon, 9 May 2022 12:15:59 +0200 Message-ID: <20220509101604.3249558-3-adriaan.schmidt@siemens.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220509101604.3249558-1-adriaan.schmidt@siemens.com> References: <20220509101604.3249558-1-adriaan.schmidt@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain Return-Path: adriaan.schmidt@siemens.com X-Originating-IP: [139.25.68.25] X-ClientProxiedBy: DEMCHDC89XA.ad011.siemens.net (139.25.226.103) To DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1389eb25-7dae-4bd6-76ed-08da31a4fdce X-MS-TrafficTypeDiagnostic: AS4PR10MB6136:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d+h2dV7VSuNm7m8sFi6FDW6nve/FKK0d66F6ifEu5LJhU5Y5SkmSYLEa1OXgJeXILDTC74pBGsKC1j5b/1KSkAo7ctiNaI6QeR0pXesVPsn/bQ1MILxHNQQs1HxwWHGo5oMJ1l2QW6jjEEtVxF0wrN2gXiNcmeVPOYQfdWow9J4/p1Sq33MrI5aDqI4F41WYSQrNkge37zGqjr5WnXprpIe0+/Wvx+fO66UWnQV/yt7s9t7qkQ6Xn5dPJD8BC1CHrYXoOBt4SAkiSVuuxoZ3PtMUa/uokyUMRxsMS2TaKspyIdLMVxY3SpQ+oEr/xPXFa7H9Wm8FBaTNEliQ7WpbfN8kBioMmzDCbkjZkO099weKYIR2SOnYUZVtWqupn2bWS44UL7Px0uHf3DaEr2fOL/g+M4OTr0aeQxTXQycCKKwbjQ5xUcUaD2vfc40I7haK5VKE+Kelem47+PpzoEn4tcVi10SjSavW98lvZ3FLEAvw0MAMYIO9QI0a9Dyxb1EHrHCj6QNzbgBM1LVfyBP5twlSH5+sIU43E13osiupWTjdhSUnck0EQfvhu3eHdfBLGxNaEAZ+TSAd2LXSeaXW9Qm6/O5dkvFZ9vMAuvx3DmhxY938vy7Fh9sfMPHuzD2Wsyxb5DrrvsoKJGEuwYeMA+kGOWJYQlfSqryh+SQdoXv682k2a0TKgGYr1Eb7Igce1LJVRm8jevDt3olQhzKReK1ALZ4TlzwJVK00OaBp+hFvTgWu8o6sieyEIXzRtewz37jIxOhL1i3K2Pqg9YKQxw== X-Forefront-Antispam-Report: CIP:194.138.21.70;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:hybrid.siemens.com;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(47076005)(2616005)(186003)(508600001)(1076003)(966005)(26005)(336012)(40460700003)(956004)(86362001)(16526019)(36756003)(6666004)(36860700001)(70206006)(6916009)(82960400001)(44832011)(316002)(70586007)(7596003)(8676002)(4326008)(107886003)(5660300002)(2906002)(7636003)(83380400001)(356005)(8936002)(82310400005);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2022 10:16:32.8285 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1389eb25-7dae-4bd6-76ed-08da31a4fdce X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.70];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR01FT058.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR10MB6136 X-TUID: WYguPZiwkjq7 In `runtaskhashes`, the keys contain the absolute paths to the recipe. When working with shared sstate caches (where these absolute paths can be different) we see that compare_sigfiles does not identifiy a changed hash of a dependent task as "changed", but instead as "removed"&"added", preventing the function from recursing and continuing the comparison. By calling `clean_basepaths` before comparing the `runtaskhashes` dicts, we avoid this. Backported from upstream: https://git.openembedded.org/bitbake/commit/?id=7358378b90b68111779e6ae72948e5e7a3de00a9 Signed-off-by: Adriaan Schmidt --- bitbake/lib/bb/siggen.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py index 0d88c6ec..8b23fd04 100644 --- a/bitbake/lib/bb/siggen.py +++ b/bitbake/lib/bb/siggen.py @@ -944,8 +944,8 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False): if 'runtaskhashes' in a_data and 'runtaskhashes' in b_data: - a = a_data['runtaskhashes'] - b = b_data['runtaskhashes'] + a = clean_basepaths(a_data['runtaskhashes']) + b = clean_basepaths(b_data['runtaskhashes']) changed, added, removed = dict_diff(a, b) if added: for dep in added: @@ -956,7 +956,7 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False): #output.append("Dependency on task %s was replaced by %s with same hash" % (dep, bdep)) bdep_found = True if not bdep_found: - output.append(color_format("{color_title}Dependency on task %s was added{color_default} with hash %s") % (clean_basepath(dep), b[dep])) + output.append(color_format("{color_title}Dependency on task %s was added{color_default} with hash %s") % (dep, b[dep])) if removed: for dep in removed: adep_found = False @@ -966,11 +966,11 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False): #output.append("Dependency on task %s was replaced by %s with same hash" % (adep, dep)) adep_found = True if not adep_found: - output.append(color_format("{color_title}Dependency on task %s was removed{color_default} with hash %s") % (clean_basepath(dep), a[dep])) + output.append(color_format("{color_title}Dependency on task %s was removed{color_default} with hash %s") % (dep, a[dep])) if changed: for dep in changed: if not collapsed: - output.append(color_format("{color_title}Hash for dependent task %s changed{color_default} from %s to %s") % (clean_basepath(dep), a[dep], b[dep])) + output.append(color_format("{color_title}Hash for dependent task %s changed{color_default} from %s to %s") % (dep, a[dep], b[dep])) if callable(recursecb): recout = recursecb(dep, a[dep], b[dep]) if recout: -- 2.30.2