From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7198551104298156032 X-Received: by 2002:adf:e9cc:0:b0:2c5:3ccf:e9a4 with SMTP id l12-20020adfe9cc000000b002c53ccfe9a4mr273935wrn.29.1676044129760; Fri, 10 Feb 2023 07:48:49 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ea50:0:b0:298:bd4a:4dd9 with SMTP id j16-20020adfea50000000b00298bd4a4dd9ls1140678wrn.1.-pod-prod-gmail; Fri, 10 Feb 2023 07:48:48 -0800 (PST) X-Google-Smtp-Source: AK7set8HK61esw5EK5KjUzyrQ36rSYKCbNmvhktrCFfOYjfOE06RsDezApsl9dYH5g1oMgEkIsco X-Received: by 2002:a5d:6108:0:b0:2c4:3d5:46dc with SMTP id v8-20020a5d6108000000b002c403d546dcmr8879204wrt.37.1676044128315; Fri, 10 Feb 2023 07:48:48 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1676044128; cv=pass; d=google.com; s=arc-20160816; b=fh/ZdkucosgbbCWeBKcAPlE6dLoYjo/r5elUFzPhSHBo2nDrs8Pz0t7Wz6sc13rFDy /0Yva6yOyjHakIuPi+RktP74mluN1575KCp22KI+lqs4ZJiZz8w654kTNUJb4H3xHZJi Jvy2rBKQoVDQJ181OICsQpFxPE93YVXnb/V6S/H0gdJLcuwWw7DSnxRe56m+EoVYeZbZ iyQKj3nuXham2BReetO+VxC/nk2U8a9KEwjgxe38FZSASfSBW88GTJDiR0LFFZk98/Dq l8vku21cMWo/mKAmyJJBoH2LDwQ+5QlYIrTbtx0+ZcogAY4pcfPZe7hMaV/jI9B2NVj8 oEFQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=3tSSZVtpLXpDjoLllcKjVrP9G5FceigkC2HymhL/evw=; b=mes0eF1fIqLRxBNNEdiZbYd22eN9LgvFBIwgY/IjlU2Jg601eB+B5jbRM/wwcAYe8b eTDYF+FfWELpYfVq5UAO8q3lAvNIVdnOSDqPl1Irau1TtIqgV18qP+oCi0H66t3AIUsP DlNi61x/TDlY4/IBUsCGg8vkIZU+cxlJASWJ64AnMeZxEwL2fb22Dc9q65BjI91KzEDD YITiSUeJlkpo0zcUiki0YjPPoDbhSa0aGjdo0MOxxs46Ov6D5w7/Yfay6zyqSLXIS7NA +TagrrgL027k5JnaX2Tc3AK8x5nF187pwiQGYEQdDz95YYj3R+EXllF+6ACVAxTR6koh Zr+w== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=wN2KcG0+; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::604 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on20604.outbound.protection.outlook.com. [2a01:111:f400:7e1b::604]) by gmr-mx.google.com with ESMTPS id ba27-20020a0560001c1b00b002c3df588e35si205799wrb.7.2023.02.10.07.48.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Feb 2023 07:48:48 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::604 as permitted sender) client-ip=2a01:111:f400:7e1b::604; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=wN2KcG0+; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::604 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FmgKSvZ/lLDdt6ip2ekzutSIixcvjjzL4fnfD6CL64rm+PXtZWJeVqKhYWCJX2XjAUS7JuN0oUPCt3Uomfs9kGdXbIGmZhxQueVIp7xppiimCCU48rt/RHPxWtcyME1BEvbJHrA8iRPVDo2C6bTTz0b+A1Elu9kLY/AP5RlCLp10cZ4Vt//1/5567HZ/thLxyRCPYILCOyR4b2C8Rigye+CLfIGm+fNHoyOk+6BC70hnBrcSSpfMHxVTY74k+4r2GKIuzTwuBmjiNTU/qULfL3YacuLRV3DzwQUKEtguHZpOCjc90XsXFqM46Gz4isrwhHP+/xPAqV4ZndwQZS81Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3tSSZVtpLXpDjoLllcKjVrP9G5FceigkC2HymhL/evw=; b=DoVV7W3sArOBrfb8WK4HKXfmHfNwlpJDP37TNvIQNclZQNc2KlzkSvjzEjwNyCCs7XskGC5qReAAv3LZRPtQW1OXlHa37TPET73wcgtFnFUv6d1e3j/DtxNIj14gT+2OVWpoJMAonRiJx68TdEmpMKLRPs3VxOLqN82qZMmEXjfjqHLBcx+wFn4hCTWH1YjdGifvfFtORR2dEqNunOghhviLfpf69jWu+OXeQDL8ks57tS0QTn5/ZpL5epTvxDaeqWfCHkoxW+UViejfpt2+XXNcjfpLoSEBbyyvCC0Uk9JM5aHBEC91yaTnFCeYwMf0o3ENH4ZLkP9a87FPi6toeQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3tSSZVtpLXpDjoLllcKjVrP9G5FceigkC2HymhL/evw=; b=wN2KcG0+GeznhaCVJ+JW1YwnEJdrQIZk6ApLPxcJrLskeMe3MsEbYAmsbrUeNYyx8mWLx+Ia6MH+YioHfiIk/l+a+AARiogfURQ6LfDbojvXYXRWBcUrnBUtqFEAtw1hyvjJWU/CLUKTUBpt/B0GUUl9EUZiZ/uelYOsYfXAfQGQIjpYgYUdSIu6MHB51AQi9Zci5YLFGcnP8H5L2dp+D2sb7+zfQYji3rem6PKkS3QkND99wFmy09bi+HYfBtQ3ROpmd0V3Ey9W86JDFgkzepK/49sCA8MW5bXdP1gi2ouom6pp5up864BxC8a7jWVGyZixFVmnfpZFivFJrsp6iQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by PAWPR10MB7792.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:369::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6086.19; Fri, 10 Feb 2023 15:48:47 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::ee44:171b:4c40:d42c]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::ee44:171b:4c40:d42c%6]) with mapi id 15.20.6086.020; Fri, 10 Feb 2023 15:48:47 +0000 Date: Fri, 10 Feb 2023 16:48:44 +0100 From: Henning Schild To: Adriaan Schmidt Cc: isar-users@googlegroups.com Subject: Re: [PATCH] fix(isar-sstate): also handle zst files Message-ID: <20230210164844.440757d7@md1za8fc.ad001.siemens.net> In-Reply-To: <20230210153434.1024604-1-adriaan.schmidt@siemens.com> References: <20230210153434.1024604-1-adriaan.schmidt@siemens.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR3P281CA0168.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a0::20) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|PAWPR10MB7792:EE_ X-MS-Office365-Filtering-Correlation-Id: ef099de7-3e07-4632-17b7-08db0b7e4bec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0hCAsV1va07At7lRqXca5MPbX5Ziz6eE4AqSh9VsTE8ZbfHUgN///fkF2PQbfAtzS3MD5bC1z1NyKL38sMU4vPW520FsHETCrc3+RLLNpQH+HFixGwVLPyiwPFCQbt27dXqInfncIe4Wc+DL02XiZbuCi7QjG4OMADzq49eJtYptQr95zHFkjs2PH1X+WTbqHeAe4HezvwVtOUO9PapDjFrnUsXBP2vdmLbJiqKmpl8a7jLxpox0M4LetrWALc+xRExukVr871en8t1VV1uzE5JClEIXsCfBC2lLLW8DE16G/axWuZ16RrmAc9wwuW4NM636RJx4E/F2OTMjbrRP8bfGD4IfL6CoiPWXZ94FWJjCIC1ZlQT81PHanqKCsbljf54i9vO2FGEk0HyW6BnhYusJgm6bthgVWnV8qYbhM9uPitTi00b/q266VBiracStGDtoNPQKA098Oc4Bu/Hrhspnpj6qP6yxQexU3WOEQumgs8Ea4sl1/L7TYFEBBupbvx7pgN7uI8k3G9FKwbgNrWt5ILdlmx2UeCR16aFaGUD0HGCWR8Jf4kY69bxYP7dUtThVcdcYm1OQBbxdM6wPNkwfiyomrORiOmBLrRbm9PrLsBqL0/KAe3NhLwAf1wxpWWWUbNjLi7laVD6Rgr5ImA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230025)(4636009)(136003)(346002)(396003)(366004)(39860400002)(376002)(451199018)(82960400001)(38100700002)(86362001)(6512007)(186003)(26005)(83380400001)(6666004)(9686003)(6506007)(1076003)(2906002)(8676002)(66946007)(4326008)(66556008)(66476007)(6486002)(478600001)(44832011)(6862004)(5660300002)(8936002)(41300700001)(6636002)(316002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?etq7K1jxqz0FB2UVJq7UXeYiwkqbcqshZBbzfkWCXrDoU0rgxpMdzxKkwCXP?= =?us-ascii?Q?9fj3m6iGyFGGNLN+PElNlMg0Hv2i2RLC3+0UYxe3S8z5prxgQtITn8ZNHmPo?= =?us-ascii?Q?Iq/Lp5Jb3EAA+FL1QeQXhYZrzky9zdg5O/2rGqO8lXziPlwg1qkQK1XfJ5At?= =?us-ascii?Q?YXvzskTwVuy42ssODt/lGVusEIgElFOCbf2C51L5orVVZh589xDf6huFmePh?= =?us-ascii?Q?h07AlhpdAbnf6kCJj/ZptZ92+uzW38qAgLRtx6nwYsLwaAZZEnADhgyrk/XF?= =?us-ascii?Q?PZDn0IhTwq5DfIPszcj2xsL0rouOJVUxKIwAsIK7URyz5O9F7vJRGZEN0YJv?= =?us-ascii?Q?hxcGUouDSA5I6t30fleHKXgetc/NPyHB9zkQ6Lg7zM3xuCMPom/8tVlydnrG?= =?us-ascii?Q?IkHPMOB0tME6YvT6QfufvKfP0a6tK4JKfec1E18bmPbs5c38YGCMvQB8eSkI?= =?us-ascii?Q?vUB5KLeGi9EwOLVmwxyVG2Stw//pgAM4ofhNchZcqCcQpi5PCYdMR9VVTjmZ?= =?us-ascii?Q?8rrCkYckBeUNRAYmOf4AEPK4BO+S6vbOn0oq9MxIu4Mh66kLolGit5BYW5GQ?= =?us-ascii?Q?cfTJfuueT/nIw91WLs+VKpPUuKqtbxbEMKxEVmT+NGYOgac7Hk51e9bJpfh5?= =?us-ascii?Q?gF3LnlB+cWFCF9uZl3PysMXjyVOiuoXnf00RBSVNScG4RGVS/Yl7usrds1or?= =?us-ascii?Q?GvfszffAeRhfqq9LYyU6+PK7sLiYaVSRe9NFkw+LQQY+/4vTsuw6pq1yc2Iz?= =?us-ascii?Q?bhqsmwscHUpLy8eAAali+vQpfEN59SYSac6/1QRGE0zllgjNFOlte12sW9uX?= =?us-ascii?Q?KRfOzWscWrFuXStLD2n+7oqvzvQAcU5ucqc9z+CrQQUo4hLCv7r4GPyUT0Ou?= =?us-ascii?Q?RscvsvcnNPvGig3Iti5Cb5QwQGPdmVKzdfhGwIctiUDUb00/0KhWTKyic+gw?= =?us-ascii?Q?pShI+XnWZb6gvX5sjBg8HwNuUSt2QoElfo47k6D+az0UOhKQLyEWhXbmKRac?= =?us-ascii?Q?hDTPHf784l4AWni4BRVjCjIinyOZ/QPuH6yp8GOOpuvGD1KF8M0owdTiL0xP?= =?us-ascii?Q?wtr2OL7jvwFrRVigfcdfcsYikUr0TdIoZni1O81Sbnubv2Tl2Os4rlfQKYkp?= =?us-ascii?Q?2eniBleeTfXeAU0iw/2OHYdKWiCPS0OILi9okJgDbNpSeGrUwvdJiU2lEYGc?= =?us-ascii?Q?AM9WeNsgE3jnJdnzqj6mNtpEWGjdxUyWiHlHHOkXTHKYXPZna2OfkSJ8sN0a?= =?us-ascii?Q?QTljSx+HrE+eeTuo+1SNJOspxWGwOVrWBnkX6aGV60UVvuDZ/WYd/Vl/zjLX?= =?us-ascii?Q?Qr/Ufm2LHsRQPhfJHeiA/2QPfpfNFKS2MfHSV+FZ9OI617p414gNxVomehSZ?= =?us-ascii?Q?k37MWViyDlPDOpnCIUVJJs8mNJU+6mMbi+cjwZhxL+4x1wuaBtamWnupcQej?= =?us-ascii?Q?6VMjVMXINSrei1nSjH0dYrw6qiWHs8SmnA08PEVLPngyjdWE6I3qhHFzM0Ei?= =?us-ascii?Q?xFm52tDRryT/u4Z31Y9j0Ho3oVdB3sjUdZOuRve+hkgWraC34jnXNKM2hfrT?= =?us-ascii?Q?g/03VOmXs1WU+BPgybyGkZ5UueHuKH5d4aKyIfmQeYXfs3Aq2bz0rSEBoJ0j?= =?us-ascii?Q?jaJQqPsWhGjX69j8sxlh5bA=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: ef099de7-3e07-4632-17b7-08db0b7e4bec X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2023 15:48:47.1543 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EgLvsUaxHF1MwaBLSFNePRH4tZseo0YB2EMU6xJL+4GktAY3DSjRduk08eVMdalhLhhkLNS5zcMp3Xut7pOinqMAQkoxNPAo6+bR2/Nfu60= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR10MB7792 X-TUID: afs11KxIE5MX Am Fri, 10 Feb 2023 16:34:34 +0100 schrieb Adriaan Schmidt : > With bitbake 2.0, sstate artifacts have changed from tgz to tar.zst. > Our isar-sstate script needs to scan for those as well. The > implementation is backwards-compatible. I came here just wanting to make sure it will work for both, and yes it does! Good catch! Henning > Signed-off-by: Adriaan Schmidt > --- > scripts/isar-sstate | 24 ++++++++++++------------ > 1 file changed, 12 insertions(+), 12 deletions(-) > > diff --git a/scripts/isar-sstate b/scripts/isar-sstate > index 53d0541f..c14c2843 100755 > --- a/scripts/isar-sstate > +++ b/scripts/isar-sstate > @@ -40,7 +40,7 @@ followed by one of `w`, `d`, `h`, `m`, or `s` (for > weeks, days, hours, minutes, seconds, respectively). > > `--max-age` specifies up to which age artifacts should be kept in > the cache. -Anything older will be removed. Note that this only > applies to the `.tgz` files +Anything older will be removed. Note > that this only applies to the archive files containing the actual > cached items, not the `.siginfo` files containing the cache metadata > (signatures and hashes). To permit analysis of caching details using > the `analyze` command, the siginfo @@ -576,7 +576,7 @@ def > arguments(): '-v', '--verbose', default=False, action='store_true') > parser.add_argument( > '--max-age', type=str, default='1d', > - help="clean: remove tgz files older than MAX_AGE (a number > followed by w|d|h|m|s)") > + help="clean: remove archive files older than MAX_AGE (a > number followed by w|d|h|m|s)") parser.add_argument( > '--max-sig-age', type=str, default=None, > help="clean: remove siginfo files older than MAX_SIG_AGE > (defaults to MAX_AGE)") @@ -664,21 +664,21 @@ def > sstate_clean(target, max_age, max_sig_age, verbose, **kwargs): links > = [f for f in all_files if f.islink] if links: > print(f"NOTE: we have links: {links}") > - tgz_files = [f for f in all_files if f.suffix == 'tgz'] > - siginfo_files = [f for f in all_files if f.suffix == > 'tgz.siginfo'] > - del_tgz_files = [f for f in tgz_files if f.age >= > max_age_seconds] > - del_tgz_hashes = [f.hash for f in del_tgz_files] > + archive_files = [f for f in all_files if f.suffix in ['tgz', > 'tar.zst']] > + siginfo_files = [f for f in all_files if f.suffix in > ['tgz.siginfo', 'tar.zst.siginfo']] > + del_archive_files = [f for f in archive_files if f.age >= > max_age_seconds] > + del_archive_hashes = [f.hash for f in del_archive_files] > del_siginfo_files = [f for f in siginfo_files if > - f.age >= max_sig_age_seconds or f.hash in > del_tgz_hashes] > - print(f"INFO: found {len(tgz_files)} tgz files, > {len(del_tgz_files)} of which are older than {max_age}") > + f.age >= max_sig_age_seconds or f.hash in > del_archive_hashes] > + print(f"INFO: found {len(archive_files)} archive files, > {len(del_archive_files)} of which are older than {max_age}") > print(f"INFO: found {len(siginfo_files)} siginfo files, > {len(del_siginfo_files)} of which " > - f"correspond to old tgz files or are older than > {max_sig_age}") > + f"correspond to old archive files or are older than > {max_sig_age}") > - for f in del_tgz_files + del_siginfo_files: > + for f in del_archive_files + del_siginfo_files: > if verbose: > print(f"[DELETE] {f.path}") > target.delete(f.path) > - freed_gb = sum([x.size for x in del_tgz_files + > del_siginfo_files]) / 1024.0 / 1024.0 / 1024.0 > + freed_gb = sum([x.size for x in del_archive_files + > del_siginfo_files]) / 1024.0 / 1024.0 / 1024.0 print(f"INFO: freed > {freed_gb:.02f} GB") return 0 > > @@ -716,7 +716,7 @@ def sstate_info(target, verbose, **kwargs): > for k, entries in recipes.items(): > print(f"Cache entries for {k}:") > for pn in entries: > - artifacts = [f for f in all_files if f.pn == pn and > f.task == key_task[k] and f.suffix == 'tgz'] > + artifacts = [f for f in all_files if f.pn == pn and > f.task == key_task[k] and f.suffix in ['tgz', 'tar.zst']] print(f" - > {pn}: {len(artifacts)} entries") print("Other cache entries:") > for pn in others: