From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7085973421677346816 X-Received: by 2002:a02:91c5:0:b0:324:23be:fc26 with SMTP id s5-20020a0291c5000000b0032423befc26mr15026368jag.129.1649838000780; Wed, 13 Apr 2022 01:20:00 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a6b:3b82:0:b0:645:be7f:ac03 with SMTP id i124-20020a6b3b82000000b00645be7fac03ls266252ioa.1.gmail; Wed, 13 Apr 2022 01:20:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwZUNBFvz5ZDow0X7CnnkJxCIKTGtxoUNoxaIW+AM16kw/RT5emj1qpnjANBRxocEQDnYuo X-Received: by 2002:a6b:d10c:0:b0:641:63b2:9ef8 with SMTP id l12-20020a6bd10c000000b0064163b29ef8mr17389925iob.135.1649838000256; Wed, 13 Apr 2022 01:20:00 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1649838000; cv=pass; d=google.com; s=arc-20160816; b=dsG6cR8lF+hUSLxy1ccpOlwqdQV2420FagT7mX+bC8WJJbF4weVL28PuoJ/DpQ7opx A0P0SUhn/DQyiw9SALymoUSF+9Yy4fjhJ7OtTfXB9o02+ib/hau0nXtfORUA5K1RX42X mKrlLyBTpeo4whLa4cfCG6Fi9p8VTUGtEP0BoyBhc33q/GcJ1DfFJP7ZFgwgT88oBaiS Rfcr/pUhnvDY85NWlMO8G+fs/lQcgwy6ujhNnohD5z4L8hZY2JNmrfSzzxLZzh8yD2zu cOCeYu8/yDqOWvRoGqUn0bDEvxioGfROa16QwIij2ikN/P0UFPoJk/MerfjVMe+b4MqQ MSXA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=nB7p4s4DM+3AzO20DHkdcbHXOWt+N/T6XU0ZqBJdVDA=; b=KMDwSwU1rec9I1wgXSX0McdAUhNVi8sexFfkV5TisVECSu2wBEFyxv+NyJE8nG8A8X Md9Mv4BvefXs8ysRHEDtWwfJyyhW19BzrZZtLajKXe23nMTi+n5S+cjOhzIp++V+OKNj vKJRhe51byLxiwSpXcQpnP20Q7wzMyHwQRPNIQ30TxyG3V3Ct7VwMKooszUeas5UJd1J rwJnIgrp7KJynv4ygZLVt8zPSzRkkdPCtZFO+oK3y9d2Z7MDnO18X9cPujkLdVIR73pw uX1P67gNj6lCayUpyA6FEF1oSTaxpr6lm/JcptT3X3nIRA6Ynbaknl3By0nEUbvnWi97 C+PA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=qV8omg1i; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe06::631 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-ve1eur02on0631.outbound.protection.outlook.com. [2a01:111:f400:fe06::631]) by gmr-mx.google.com with ESMTPS id c15-20020a056e020ccf00b002caa1789135si1147820ilj.4.2022.04.13.01.19.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Apr 2022 01:20:00 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe06::631 as permitted sender) client-ip=2a01:111:f400:fe06::631; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=qV8omg1i; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe06::631 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W2Jp9fY9fX3fOAmmveEKobzn6curOvEedPG0Lpkmy4EvACtr1u4aMG8nOwM9a0wcaE7kOD8a7OcNdAjnHcHNYpQU597aXRHmQee50N8QYPKfqq5+usSkTrvL2fP0m5aHdFnAE1lzACfBRl1fNodmgLJjGTEUPf2Fb0l3iHvhVJZeIdGSaQ5OEeNztCu+5mjg1q1IDjiOyCmedCp0IYCxjB9wlrzG2jYFMglNfZ5hs9EbVkr7ak1PWOYj5hzifYS5eGyKJFAZwgSJ4JIR4pztnC5WTaEF8XJqoQudfCM1CCyoz90N00/pN+zd5c7NTJNQsgUQabOf70yO/Rg2n0vu6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nB7p4s4DM+3AzO20DHkdcbHXOWt+N/T6XU0ZqBJdVDA=; b=NdxXACN3k9rv5GGAmKwqzKlFjiiABi3XBIernSB+pzssdqz0369F5+t+Dkr1V5PAtP1g6H7dcN/B7kgmEpq7JaBIP5rWc+3yY3wGRA4TQx/zoTAXLS+G3mQmzmSr7kDfkXb0t0IadRidZJcAWXB8Mgpjle1F6kwR6p8ZSJvs1ddtl8GXjxcQn6nVdX5AbrHjgeFukpcwkYP2kW2xcCe2xSyOvL3YQ6UXfyquFrWeycEpEa8FX2NbHC0nk6jjhWc50Moh272l0HUz1+Svu+BN8e5bzit2kcMJPjhekL9Hb8ihZWHhVK8mIxkB+MrIw8xGMsBkBRSu8BA2O6yWYUOk1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nB7p4s4DM+3AzO20DHkdcbHXOWt+N/T6XU0ZqBJdVDA=; b=qV8omg1iwc0S44pL0F5ZvqUAhw86B0Krt5EqI2/0vKNKhJI7prMxyYr5CIGOxuv9NlrKx8TP27wjb+AJxgoPu6uh1iMMK5S8IQ54s2W/Fg+aZiwZljfVzPpr+Ap+2UJK/ju47lFPy+UzKN7eXn0UUhJZjd7S7SAFYw4S1Hxpr1VipY8dSSf/mKJ7FyJBqfoPmSOhnBosJuYW29uS6lp9U06YI4VMyRtSuJQw5pVz76j+Y1c+Gbl4A15wI8ZCPWQq7wrjB/yCeHKX6/zuAyHiD1VziLxGcNu32aHuksDkKJKY955DGA06oiqJefv0tH4J3hD3D4iqcDmaziQ/biwmFQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by AM9PR10MB4943.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:41a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Wed, 13 Apr 2022 08:19:57 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::f4be:8b5d:4314:c2d8]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::f4be:8b5d:4314:c2d8%4]) with mapi id 15.20.5144.028; Wed, 13 Apr 2022 08:19:57 +0000 Date: Wed, 13 Apr 2022 10:19:54 +0200 From: Henning Schild To: Adriaan Schmidt Cc: Subject: Re: [PATCH 2/2] bitbake-diffsigs: make finding of changed signatures more robust Message-ID: <20220413101954.67aca2e0@md1za8fc.ad001.siemens.net> In-Reply-To: <20220413063534.799526-3-adriaan.schmidt@siemens.com> References: <20220413063534.799526-1-adriaan.schmidt@siemens.com> <20220413063534.799526-3-adriaan.schmidt@siemens.com> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: AS9PR06CA0185.eurprd06.prod.outlook.com (2603:10a6:20b:45d::6) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a9226948-f73d-41db-22e3-08da1d266514 X-MS-TrafficTypeDiagnostic: AM9PR10MB4943:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: f/GGOe5zRvmW/xiDvcMYhQ9LqeaYMEAYEKmQehWR4GWYbmJ1werley+klzT4Ow7gYjkNAn5NQOswq2qlgdJrhp/qaioJv6ak0TY+xP7RAUwUu2pIWacSlaD1F+nCbqqUeyiutojcJ1qOyVH6sO6VAgCH2PTEERNdMiZGeWS4wQqHxjj/6F9MEmXL4mW6D1FSjNURh+YXrIz7qmgffaHxm4AZOviRmKsg99mlpaDzmE1KWLu5RjYDjpmHUTDbAKrQG8MtoSvFC0S3WiBphuR4B6LOIpudupVIvCy0GsTRLY8348u+RGqlYQDHq+Q9ONFpGnhurgbJ2UExawXusrSdjzY6Ui4UuwkF7557DVSwKBpiZeGnNXQIj4Z98QPuuCWR9cPfggTM1MqFx6R+Ca3qeKQLoAZOiAzsx/wyUnegALu5wi9fBbrwRn/P4xR6bnivFdaqKbPT4Jc7jnc9fxvISDFfHrnSqXFSEYhM6D1aV4lJr8vjOeHNs24HLzRz8UTsmuHPYN7n0seNgzLWn6fS8FeMbXH6uja1fLLIvmIYfWYGflhVp4dd3c8395syXItWdqh1ZrWWC8/rDtDS9vprjINxqHYDTeb2GTY+W0j0RhddWZtxlKW10rpz8JVNj1kdS82+fufc8vJAlAxaX4TEykPDF7Ui/NmGbnQyKYlaZaYK3n0rCpPp/hmRNieD5bhBV2HGSDkbDYrrv47FntoLzhmVtwaAzvahtvdj8AMlQWw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(6636002)(186003)(6666004)(26005)(8676002)(66476007)(38100700002)(82960400001)(66946007)(6862004)(66556008)(83380400001)(86362001)(2906002)(6506007)(8936002)(5660300002)(6486002)(498600001)(966005)(44832011)(9686003)(6512007)(1076003)(4326008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?yfqndAKj5pK4HIFWG/g6jtht8psPZmQh8ZlyVdPkPRSSa+tNUZzqHsqWzgBR?= =?us-ascii?Q?rtVuDZxKV+jNdTz57OcOJG33/Xq7ei/fobJu/OF33z7Bk8RamgdFc7A/K6HW?= =?us-ascii?Q?U4IZz8HbFRI9JmXJxMpMNCVlJzop+K6Ygm9M9h7YywYqQsukH+gn7row8Gb7?= =?us-ascii?Q?v1zyr9jX13ygTelEMPSosfjPDNTHGJVeylmr00Np4GbVzzg60GSpSR0G7jlC?= =?us-ascii?Q?d3+2jzzpjmzPSk+mueU7+CFdDxS8dnhCkvHn20bGf0fU0odo2Kag4LF4gD70?= =?us-ascii?Q?Qa9Z60Sf3BhIjGXc68X63zsT3ptGlOS00fKY12EqbSNZbukPUbgCwDIJ5fvg?= =?us-ascii?Q?H1xwi3mtLNkX6UANedMf5WcxphUXfireaxufjk/AknB5a65/nBDBbmrQeS1q?= =?us-ascii?Q?rri/lqTdr5cf7Fjqr8fwpB8gPCab6TeDwCTZjH/UVzjxEWf6XD+/QARd1oTw?= =?us-ascii?Q?m6Y6KNmfbw2VnwWKXe6iz7woDR+MHduHvVKbUQWxhP1tXq6gYra1o3Vli5Z8?= =?us-ascii?Q?3ddig3xQJpJmvEzq7MoVa6TugkpErVB4t3bW8jF3RhhTmRf/Ffax6SbrfJTF?= =?us-ascii?Q?o0PAjQeIM5RXd9G2jip/PnWZGBTdQhvgxSvcS1n1vesVeGw7Q/FCbjZj6z4d?= =?us-ascii?Q?kxXqUTiCOGwT4iocLkw0eMb6kkMWioWhZCOHNjbcu9UzZSjKcpjAVWVe0dpB?= =?us-ascii?Q?orq3Q564mMQ+g5EMyED09uMNJJjJRuz+m8nZzkCvKImdQz89wmDBY8TsLX2C?= =?us-ascii?Q?RTV7gzG6Yh/J96S54UjV7MOpBO0zPk3dVA1BKTwJugKO5sWXixqfiFWj0GpA?= =?us-ascii?Q?HZqB6ot41JQsSBYwoj0earQaJ0SAOF+0d5OInt67gBPJBwArZKlLYjMt6uEA?= =?us-ascii?Q?k0qwTpjDZsJG5dTtj1ZXDWES63jceEdUPhqZK4s9DzaacU6g2p1DJZrWywaW?= =?us-ascii?Q?U4HbNJES+YFT4JsapdKgzndPqSomlXxO83rJ6dp/rsrMHjjEMnScI1xWXBgf?= =?us-ascii?Q?Qigsf/10GMsLgUSiQn4glnigJE8eTqqW2FJ6QhcCsbipQQKvWxi9yDgSuHc6?= =?us-ascii?Q?alG+YrceqoMC8tCRFfXFv/+jl4A9buXBRMl2y5Zg8gek2d+1BPpz2b2rh0P9?= =?us-ascii?Q?B5CPkEvaNXGxLUY/QZ1+1JiacBp753hPmP2folw2WZfLghFPYmt4zs9SfJhr?= =?us-ascii?Q?QuWDN9CfXdVv37A8yAiCH+NaB/zCPDP9MYL9Kzc+e6Yv+1L0ip5EBz3GAkyM?= =?us-ascii?Q?Pdzzl3K+HxQdoPagVGxO7wjM2/eLKO2UVrpPGNipmGuyID9wu8zYFUkrM4pJ?= =?us-ascii?Q?de7ByhF33eBuJ27V3hTEh9dfOGjhSkQ2uGdzB+JKJ6X72NPHkHDo/WGdCCPw?= =?us-ascii?Q?DlKvK5r4vopwp7eWLAJKhmBYZ8IAHRT4QTtZRG5eHSOvIWvJaHL/quZdn+mU?= =?us-ascii?Q?y3Vn9ZtHV9ssRGJaMY6MB/8+nGbzLUwqHTo68pjE3Upzf6JO1c8MaWmmVFA4?= =?us-ascii?Q?Ws3jk93Q/yqR+l0XpHes0YERqwpNp2o5d0DOQcuetK7PK+1y5fWfn7KzaST4?= =?us-ascii?Q?nqvdjfQqJw2JcDdOMhPK20sQwQ82wZYte+b38hjaaeALl979gENbFRvr4sFe?= =?us-ascii?Q?9iGs0jE4dPselmPS/wbNbYWlndGGplQ3fRwRiLL5LZjfaDVVQm/qHzUrq3qU?= =?us-ascii?Q?MlQI3hRY6usVyznF95PVxFh8ZFHEWeIcg64GmZtWjOJFGufnRvS/xmP1UkjW?= =?us-ascii?Q?fhV3/KM4X5bJGtb8ydkwJNyI+tY2ozs=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: a9226948-f73d-41db-22e3-08da1d266514 X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2022 08:19:56.9221 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tgqVX6GkzftEyloP0sPxGwnTa9pgmeWzYSlLdrrV5GpP1rqQU3xka+DSx2cOvUX2snFZWTOCt4cqV068CaAFeiDnTx8NbqN/G83L05A+Yas= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR10MB4943 X-TUID: JZIrW+hUzV8W Am Wed, 13 Apr 2022 08:35:34 +0200 schrieb Adriaan Schmidt : > In `runtaskhashes`, the keys contain the absolute paths to the > recipe. When working with shared sstate caches (where these absolute > paths can be different) we see that compare_sigfiles does not > identifiy a changed hash of a dependent task as "changed", but > instead as "removed"&"added", preventing the function from recursing > and continuing the comparison. > > By calling `clean_basepaths` before comparing the `runtaskhashes` > dicts, we avoid this. > > Submitted upstream: > https://lists.openembedded.org/g/bitbake-devel/message/13603 Submitted does not count, we will have to wait till it is merged and then we can think about a backport. How important is that? If it is just "nice to have" i think we should wait and not even do the backport once merged into bitbake. Henning > Signed-off-by: Adriaan Schmidt > --- > bitbake/lib/bb/siggen.py | 10 +++++----- > 1 file changed, 5 insertions(+), 5 deletions(-) > > diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py > index 0d88c6ec..8b23fd04 100644 > --- a/bitbake/lib/bb/siggen.py > +++ b/bitbake/lib/bb/siggen.py > @@ -944,8 +944,8 @@ def compare_sigfiles(a, b, recursecb=None, > color=False, collapsed=False): > > if 'runtaskhashes' in a_data and 'runtaskhashes' in b_data: > - a = a_data['runtaskhashes'] > - b = b_data['runtaskhashes'] > + a = clean_basepaths(a_data['runtaskhashes']) > + b = clean_basepaths(b_data['runtaskhashes']) > changed, added, removed = dict_diff(a, b) > if added: > for dep in added: > @@ -956,7 +956,7 @@ def compare_sigfiles(a, b, recursecb=None, > color=False, collapsed=False): #output.append("Dependency on task %s > was replaced by %s with same hash" % (dep, bdep)) bdep_found = True > if not bdep_found: > - > output.append(color_format("{color_title}Dependency on task %s was > added{color_default} with hash %s") % (clean_basepath(dep), b[dep])) > + > output.append(color_format("{color_title}Dependency on task %s was > added{color_default} with hash %s") % (dep, b[dep])) if removed: for > dep in removed: adep_found = False > @@ -966,11 +966,11 @@ def compare_sigfiles(a, b, recursecb=None, > color=False, collapsed=False): #output.append("Dependency on task %s > was replaced by %s with same hash" % (adep, dep)) adep_found = True > if not adep_found: > - > output.append(color_format("{color_title}Dependency on task %s was > removed{color_default} with hash %s") % (clean_basepath(dep), a[dep])) > + > output.append(color_format("{color_title}Dependency on task %s was > removed{color_default} with hash %s") % (dep, a[dep])) if changed: > for dep in changed: if not collapsed: > - output.append(color_format("{color_title}Hash > for dependent task %s changed{color_default} from %s to %s") % > (clean_basepath(dep), a[dep], b[dep])) > + output.append(color_format("{color_title}Hash > for dependent task %s changed{color_default} from %s to %s") % (dep, > a[dep], b[dep])) if callable(recursecb): recout = recursecb(dep, > a[dep], b[dep]) if recout: