From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7198551104298156032 X-Received: by 2002:ac2:4e6a:0:b0:4b5:8f6f:baf0 with SMTP id y10-20020ac24e6a000000b004b58f6fbaf0mr3390337lfs.217.1676043286253; Fri, 10 Feb 2023 07:34:46 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:2526:b0:4db:2bf0:d4b5 with SMTP id be38-20020a056512252600b004db2bf0d4b5ls929464lfb.0.-pod-prod-gmail; Fri, 10 Feb 2023 07:34:44 -0800 (PST) X-Google-Smtp-Source: AK7set81KepVdoZAAifzIeUgSKNZSEISVuTOpDXxvABJMYFk4so9gmeqhEaaStLFAVZzpWu+nU5q X-Received: by 2002:a05:6512:21c8:b0:4d8:6540:a72c with SMTP id d8-20020a05651221c800b004d86540a72cmr5283191lft.46.1676043284718; Fri, 10 Feb 2023 07:34:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676043284; cv=none; d=google.com; s=arc-20160816; b=IcBnDPtqstTdv+3+9uyjdLmbN5osy5ktMSRCmW131PrVN2bWqV9rVlo0u0zrzlN7oc /HLXiO8zdxb4GPdjyt235NqY/WrB0MtbA+89qzdsmHWtZV8WJEnVhc46/YfMI3spKhMb txEbp7eQj+JxEJLp+lN4sN7fPN2rkO5jsVZi1+gJPQ3HzCFixV7f2UXlM7ttQ2IQEQeq haijT0xHWQx12VsMWYO4uXRk7ZgMEddx8zYyjmACCsAbsMYmVjVqunLJo5zvriInmfCZ mjBaBBUcRaZ3pMuVQpiaJrXuG1p7fppUVBCQBbH1vZR0wWx5or2Tw5fkufUaDhDJpCJs 8RqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=XlqzTjJBED2bihT07eNPe22HlTwnidJqD0sLtwHnLrk=; b=bRr6uzSe2DpBFM6TGKO/+b+dG3QV4/Do16s+wbwjnwZTu+fVvjklnhhCiBDertiSIo J1v1A4M+Rnhwg+Qbxc2mb/O2yLfEYviYMRAHaKFWLBB6C+QvgZNx/+t5QZrqPSmxO++x WL3r9o++X76qLpLYBU/RxOYoyk6fO8dvudHuyIyBjPuzHbfFSmRf/PDzmYvnd8V/lc0o INRL6XpwElYbjd2Eh1T3Sroua0jjZrgJCFqlJMiA1UjpZvVoWHLiMvU9rjBgBGJOmir7 0l8tYTrR4Ua+7HaktFRb1rkk2LQ4isswnHQh+NjtwXwYC3TOwCwy6/a9U3qoPn9qY/gT lz7w== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=lpeKwdfs; spf=pass (google.com: domain of fm-526363-20230210153444b39bd6e0cc7877d3a5-_b_ctu@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-526363-20230210153444b39bd6e0cc7877d3a5-_b_ctU@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net. [185.136.65.227]) by gmr-mx.google.com with ESMTPS id p8-20020a05651212c800b004d579451cc2si264980lfg.12.2023.02.10.07.34.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Feb 2023 07:34:44 -0800 (PST) Received-SPF: pass (google.com: domain of fm-526363-20230210153444b39bd6e0cc7877d3a5-_b_ctu@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) client-ip=185.136.65.227; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=lpeKwdfs; spf=pass (google.com: domain of fm-526363-20230210153444b39bd6e0cc7877d3a5-_b_ctu@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-526363-20230210153444b39bd6e0cc7877d3a5-_b_ctU@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20230210153444b39bd6e0cc7877d3a5 for ; Fri, 10 Feb 2023 16:34:44 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adriaan.schmidt@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc; bh=XlqzTjJBED2bihT07eNPe22HlTwnidJqD0sLtwHnLrk=; b=lpeKwdfsxAjVkjsydEtdFNoMoBDrotUkbqejdRvREA9nB6noVsX0JsTG+yIJ2XNbvdcIVo SCtuKBuBnlMm3qlOcR6aLpbL78WUpm7o4nzAgoW7xbnBUNdhiHYVEY7mCXecaoDmtj0KWxSF faH7K8roB82EfM4B7Fftf7lvhoxHg=; From: Adriaan Schmidt To: isar-users@googlegroups.com Cc: Adriaan Schmidt Subject: [PATCH] fix(isar-sstate): also handle zst files Date: Fri, 10 Feb 2023 16:34:34 +0100 Message-Id: <20230210153434.1024604-1-adriaan.schmidt@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-526363:519-21489:flowmailer X-TUID: sxwgZWtApAQP With bitbake 2.0, sstate artifacts have changed from tgz to tar.zst. Our isar-sstate script needs to scan for those as well. The implementation is backwards-compatible. Signed-off-by: Adriaan Schmidt --- scripts/isar-sstate | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/scripts/isar-sstate b/scripts/isar-sstate index 53d0541f..c14c2843 100755 --- a/scripts/isar-sstate +++ b/scripts/isar-sstate @@ -40,7 +40,7 @@ followed by one of `w`, `d`, `h`, `m`, or `s` (for weeks, days, hours, minutes, seconds, respectively). `--max-age` specifies up to which age artifacts should be kept in the cache. -Anything older will be removed. Note that this only applies to the `.tgz` files +Anything older will be removed. Note that this only applies to the archive files containing the actual cached items, not the `.siginfo` files containing the cache metadata (signatures and hashes). To permit analysis of caching details using the `analyze` command, the siginfo @@ -576,7 +576,7 @@ def arguments(): '-v', '--verbose', default=False, action='store_true') parser.add_argument( '--max-age', type=str, default='1d', - help="clean: remove tgz files older than MAX_AGE (a number followed by w|d|h|m|s)") + help="clean: remove archive files older than MAX_AGE (a number followed by w|d|h|m|s)") parser.add_argument( '--max-sig-age', type=str, default=None, help="clean: remove siginfo files older than MAX_SIG_AGE (defaults to MAX_AGE)") @@ -664,21 +664,21 @@ def sstate_clean(target, max_age, max_sig_age, verbose, **kwargs): links = [f for f in all_files if f.islink] if links: print(f"NOTE: we have links: {links}") - tgz_files = [f for f in all_files if f.suffix == 'tgz'] - siginfo_files = [f for f in all_files if f.suffix == 'tgz.siginfo'] - del_tgz_files = [f for f in tgz_files if f.age >= max_age_seconds] - del_tgz_hashes = [f.hash for f in del_tgz_files] + archive_files = [f for f in all_files if f.suffix in ['tgz', 'tar.zst']] + siginfo_files = [f for f in all_files if f.suffix in ['tgz.siginfo', 'tar.zst.siginfo']] + del_archive_files = [f for f in archive_files if f.age >= max_age_seconds] + del_archive_hashes = [f.hash for f in del_archive_files] del_siginfo_files = [f for f in siginfo_files if - f.age >= max_sig_age_seconds or f.hash in del_tgz_hashes] - print(f"INFO: found {len(tgz_files)} tgz files, {len(del_tgz_files)} of which are older than {max_age}") + f.age >= max_sig_age_seconds or f.hash in del_archive_hashes] + print(f"INFO: found {len(archive_files)} archive files, {len(del_archive_files)} of which are older than {max_age}") print(f"INFO: found {len(siginfo_files)} siginfo files, {len(del_siginfo_files)} of which " - f"correspond to old tgz files or are older than {max_sig_age}") + f"correspond to old archive files or are older than {max_sig_age}") - for f in del_tgz_files + del_siginfo_files: + for f in del_archive_files + del_siginfo_files: if verbose: print(f"[DELETE] {f.path}") target.delete(f.path) - freed_gb = sum([x.size for x in del_tgz_files + del_siginfo_files]) / 1024.0 / 1024.0 / 1024.0 + freed_gb = sum([x.size for x in del_archive_files + del_siginfo_files]) / 1024.0 / 1024.0 / 1024.0 print(f"INFO: freed {freed_gb:.02f} GB") return 0 @@ -716,7 +716,7 @@ def sstate_info(target, verbose, **kwargs): for k, entries in recipes.items(): print(f"Cache entries for {k}:") for pn in entries: - artifacts = [f for f in all_files if f.pn == pn and f.task == key_task[k] and f.suffix == 'tgz'] + artifacts = [f for f in all_files if f.pn == pn and f.task == key_task[k] and f.suffix in ['tgz', 'tar.zst']] print(f" - {pn}: {len(artifacts)} entries") print("Other cache entries:") for pn in others: -- 2.30.2