From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7085973421677346816 X-Received: by 2002:a05:622a:256:b0:2e1:a01b:a538 with SMTP id c22-20020a05622a025600b002e1a01ba538mr6242279qtx.167.1649831753271; Tue, 12 Apr 2022 23:35:53 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ad4:4022:0:b0:444:3572:f542 with SMTP id q2-20020ad44022000000b004443572f542ls700547qvp.11.gmail; Tue, 12 Apr 2022 23:35:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzC8UGGqTTLXAnysLdRXaRGOi3+QkdjDb1V2kdIq4SaGyhi3FzNqNQ3/lMtVab3w/yLQon2 X-Received: by 2002:a05:6214:2309:b0:435:374d:4bbb with SMTP id gc9-20020a056214230900b00435374d4bbbmr6833764qvb.105.1649831752760; Tue, 12 Apr 2022 23:35:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1649831752; cv=pass; d=google.com; s=arc-20160816; b=PT5gyyXLguoZ7MoSHTZa6rebCpJpo6UK3uBqIdVgYy3FP/zWflW9NnaIciVDGdAFZZ dvmlXSEW+JrOYNlySWGzUmfx2WiVcwllbrDZxTwBSzdtpVSDLQJFyiP+VgBKCf6wz+2h O520CHoNUTIklfM5ysi3+PjvjeSVsmP30Ru2BVur217OT6RCjLoBwXK1bMMUww2h0O8C mHCV1ntZ+gg0BbQ7v8AatBZRIkMqzpoUaLwxVs8G+Ek6GKxsZ3ip4Gfu9TOPyF74Hgcz XWoIYemr2m8wpP0sKYr/S7zCOQ70qP/zI2v+lZbU8cZTg96nsA3vMC+gh5KbexBwPesW bv4w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=DU1xdQZhFG+HiPEuVH1eVIPR/1z15AwD5uKHOPyXqKY=; b=EoPbg7UfJKLjQcZ1Lm6tGe1QisINzrfwjIpoOeNA32A1ur1qbK0DQe8xi/4wfJ0/l+ dU3rB2zxwBuNHxiPu/ErTkwBzu+F/cASAtoMt9lziwoOVrqC2BvHsets6qjF7IU4pCTY 8EaCXpMHSlsD3ShWOiWHhAPf26xTYHpkgqQap1DdgDQClXsdrtvmTvFMh7revpktl+ZC XNKw+8J+mSWFVq6K88xibsH9PjzMdy5qLE0YpmqQkxJ2liOPK2PjG7mXjSvaDgxeIY+K 4bV1httJ1okpObACuWYzLPcBgt2rv6bnM1A9GkUmRZP52ZNzpM9+jlaF1VRqL1su8HKp HLMw== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=agi6VCwX; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:fe0d::617 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on0617.outbound.protection.outlook.com. [2a01:111:f400:fe0d::617]) by gmr-mx.google.com with ESMTPS id j21-20020ac85515000000b002e2dc3fa927si1782608qtq.1.2022.04.12.23.35.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Apr 2022 23:35:52 -0700 (PDT) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:fe0d::617 as permitted sender) client-ip=2a01:111:f400:fe0d::617; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=agi6VCwX; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:fe0d::617 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MAJfum6PY2BXxlsuPU6M903pZRdZmJqaZC17BUb2BUZBFcJfMspnaF06P91agT4AWCHXFedBpmzUApgCIJn4MVyxsPnJRekYKf1LcfSZR+pRDX8oZ8eLGfwpK+byeNX95aba4IpjvZlOsL5zoMcGan9oyDG5/4t2eOpVDO+wd9C4VhpYVk3XrnYZ9Sy03E21rJxavJ12xmX0U6joR8/Iip3V79oEH6Hke6sLV5oVhZYCUalfGXlMZFEnp4AJ/9ZsdvzYzv73fXyBrJeU0Pf9lx9Zhy7ZLG6oDk54ZlnwGz9KMVdnY8GhDhYrBw4cjB9w268zhqlaSjk0gT5A3WCQfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DU1xdQZhFG+HiPEuVH1eVIPR/1z15AwD5uKHOPyXqKY=; b=Z3YUaf6oaBN7g2c8Q0pVhUMfwCVaAekyCbY2EZKjX79DghHveyxQ8Cj/2ImmV3F3vtPgY+wt4lMyz7pHUsmRiWQtYImTTwyO/k+8IDgpxv2ZLY2eVpH5Ip5fS65jxE1QMx9vzrfU4wsQ6FGWKQl4qTzXnNYZvNruXQlGZOu+lc+YF2WhwW7t+8M1FLJQ7ffgrbQg4HxdZqotvct+sZ/BnigvLbgewkvb73EsNSYvIibOox4EHVcYBOSk5KUrStZNHcyn1JcAnq00V8VGg0X+7stElQTUF0yWfbQz0yJUbmPqEKCKKW6BHrP0SLRYB6xEIGcbyhe96ScQdmytx7XzOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.72) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DU1xdQZhFG+HiPEuVH1eVIPR/1z15AwD5uKHOPyXqKY=; b=agi6VCwXzUrquOFDZzTIN1cqQvK4Qa/qnr8vUP2CrrXwAyRgdX6EtylIdWTwL2XjtToL6KLdKmdd+vzCqDYfmwFh2frO1V5yeYUPHaCkDObVD8gTuuE7fzxwEMfsVnZjMMPo43QohpsZvBmES9zMBMQanCgFknPW4ntlpqmIXtk+0U9SFaOo8ZsLGXdOYq3KFg6+C/2C1cV3z64wGVlM7nI7TFGgnroQ55sQpwywllEqPl3hIKaOPYkSucoBk79rsR1EsNA3L4bfmOOXAd8fcapf/fywn3SpiCEz/5O+YMnnSkIMrO70tlzCDXovAc5RIfXQPNqu3vLjK/IJh3vhXQ== Received: from DB8PR06CA0004.eurprd06.prod.outlook.com (2603:10a6:10:100::17) by AS8PR10MB4935.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:401::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Wed, 13 Apr 2022 06:35:49 +0000 Received: from DB5EUR01FT057.eop-EUR01.prod.protection.outlook.com (2603:10a6:10:100:cafe::4a) by DB8PR06CA0004.outlook.office365.com (2603:10a6:10:100::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20 via Frontend Transport; Wed, 13 Apr 2022 06:35:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.72) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.72 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.72; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.72) by DB5EUR01FT057.mail.protection.outlook.com (10.152.5.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5164.19 via Frontend Transport; Wed, 13 Apr 2022 06:35:49 +0000 Received: from DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) by DEMCHDC9SMA.ad011.siemens.net (194.138.21.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Wed, 13 Apr 2022 08:35:48 +0200 Received: from random.ppmd.siemens.net (139.25.68.25) by DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Wed, 13 Apr 2022 08:35:48 +0200 From: Adriaan Schmidt To: CC: Adriaan Schmidt Subject: [PATCH 2/2] bitbake-diffsigs: make finding of changed signatures more robust Date: Wed, 13 Apr 2022 08:35:34 +0200 Message-ID: <20220413063534.799526-3-adriaan.schmidt@siemens.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220413063534.799526-1-adriaan.schmidt@siemens.com> References: <20220413063534.799526-1-adriaan.schmidt@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain Return-Path: adriaan.schmidt@siemens.com X-Originating-IP: [139.25.68.25] X-ClientProxiedBy: DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) To DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ccfc444a-358a-40dd-e713-08da1d17d947 X-MS-TrafficTypeDiagnostic: AS8PR10MB4935:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TxAj8EMqtIjhpuSqY0o/QJKJuTkLfU9a8bDrdJI/trf5MZjzERmazZ7ylTwxCPM+TPecdPTlXrXDwFmF7l/0WxWlyyZNxH0VxPqZNd80w8gS0h07HWvSz+HTj4p1+6R1gbK2niRDeX+0YdzniA4CeTAvhoPr+wRpiDICGsFmDMscnzRAgHNqs3WAjt9wVCC48WgNoN+GbGmy5r9jk470EFBRA+lU8/IkgX6qYccpMEiwSjlSiIjyFZoBoyinWaR/PEBLVv2orCoBOhuPCDb2czfyCnO7BlKHZMwo4md5LBlkrlYdJNpLRB7xHOewGhCswBT0jj2NuKkua66HiY6At9u5HlhQeQtC7XFr7hXVWeEg1JSF44hwN5uxTer3cGHTOa/YmDZqVjyUhpq72j2QCWMfat2ISHcM+Fw5PPdOKWOd9lbHSRQuiwb9AjUzJc56m6hVn8YL6HDewmDI/1zIwzE5T4idRX2o2EudpVeFv5pYce7xQ63pJbkuEJM852+J2oZNUppjRnJCR9PS5oIAHQh1GPQFS+vJzotkTFilvawPnuAc+NRhcplMpkAqgPi0KLaFVusBOt2i2s03FXPTanNOcGjYdrjIn0vldnpaY+b1FhKQxv613ilcuG/rjml0PTa/KJtuRx1argexZps/UsXF2NphMUIy+KlhbGxD9xt3r7RnlzYJBu2+U4oBjCxARqLo3Zwjk87AEylxsEWgb0IeTWnecmIB6fICISJPMfOD9gXCLEM+p3MKuW8XvIBr0BRIJiYPALzbTScM4XiRzUkbWnawMlGnpGuuZgnFpJs= X-Forefront-Antispam-Report: CIP:194.138.21.72;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(2906002)(82960400001)(1076003)(82310400005)(36860700001)(336012)(498600001)(966005)(107886003)(956004)(44832011)(81166007)(5660300002)(40460700003)(70586007)(36756003)(47076005)(2616005)(8676002)(4326008)(70206006)(8936002)(356005)(83380400001)(26005)(16526019)(186003)(86362001)(6666004)(6916009)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2022 06:35:49.2386 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ccfc444a-358a-40dd-e713-08da1d17d947 X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.72];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR01FT057.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR10MB4935 X-TUID: qEpe0foPsb8d In `runtaskhashes`, the keys contain the absolute paths to the recipe. When working with shared sstate caches (where these absolute paths can be different) we see that compare_sigfiles does not identifiy a changed hash of a dependent task as "changed", but instead as "removed"&"added", preventing the function from recursing and continuing the comparison. By calling `clean_basepaths` before comparing the `runtaskhashes` dicts, we avoid this. Submitted upstream: https://lists.openembedded.org/g/bitbake-devel/message/13603 Signed-off-by: Adriaan Schmidt --- bitbake/lib/bb/siggen.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py index 0d88c6ec..8b23fd04 100644 --- a/bitbake/lib/bb/siggen.py +++ b/bitbake/lib/bb/siggen.py @@ -944,8 +944,8 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False): if 'runtaskhashes' in a_data and 'runtaskhashes' in b_data: - a = a_data['runtaskhashes'] - b = b_data['runtaskhashes'] + a = clean_basepaths(a_data['runtaskhashes']) + b = clean_basepaths(b_data['runtaskhashes']) changed, added, removed = dict_diff(a, b) if added: for dep in added: @@ -956,7 +956,7 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False): #output.append("Dependency on task %s was replaced by %s with same hash" % (dep, bdep)) bdep_found = True if not bdep_found: - output.append(color_format("{color_title}Dependency on task %s was added{color_default} with hash %s") % (clean_basepath(dep), b[dep])) + output.append(color_format("{color_title}Dependency on task %s was added{color_default} with hash %s") % (dep, b[dep])) if removed: for dep in removed: adep_found = False @@ -966,11 +966,11 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False): #output.append("Dependency on task %s was replaced by %s with same hash" % (adep, dep)) adep_found = True if not adep_found: - output.append(color_format("{color_title}Dependency on task %s was removed{color_default} with hash %s") % (clean_basepath(dep), a[dep])) + output.append(color_format("{color_title}Dependency on task %s was removed{color_default} with hash %s") % (dep, a[dep])) if changed: for dep in changed: if not collapsed: - output.append(color_format("{color_title}Hash for dependent task %s changed{color_default} from %s to %s") % (clean_basepath(dep), a[dep], b[dep])) + output.append(color_format("{color_title}Hash for dependent task %s changed{color_default} from %s to %s") % (dep, a[dep], b[dep])) if callable(recursecb): recout = recursecb(dep, a[dep], b[dep]) if recout: -- 2.30.2