From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7198551104298156032 X-Received: by 2002:a5e:d907:0:b0:716:9e92:a018 with SMTP id n7-20020a5ed907000000b007169e92a018mr1627819iop.43.1676521974953; Wed, 15 Feb 2023 20:32:54 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6602:394a:b0:71e:5d17:7fe5 with SMTP id bt10-20020a056602394a00b0071e5d177fe5ls174000iob.1.-pod-prod-gmail; Wed, 15 Feb 2023 20:32:54 -0800 (PST) X-Google-Smtp-Source: AK7set+e4nqK4tbwLDJEkwwYHquFjBMM/5gk5e3fhjc/XTRCk8W3GAOWp9Jv7qfMQkHYcRx8PP3v X-Received: by 2002:a5d:91cc:0:b0:6e4:80fc:4885 with SMTP id k12-20020a5d91cc000000b006e480fc4885mr3712142ior.4.1676521974384; Wed, 15 Feb 2023 20:32:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676521974; cv=none; d=google.com; s=arc-20160816; b=SukGIcKy/cavMTo3bb2gCWKVYXwU0VVFHKUqXNEc+XvbS0qgfhm1UYmvMGekL6EIQ/ 8I1wYiJ9pTQzAuwIdJXM8gGkMdVBpnEEvROpEUotT4Vr/cnTVzrUw8XBxOe0owTRZF+Z mRzt2RDuB89WLjSkdnLOHjpKLpUgDqiC4uNijOoEYg+s4b/KrIvl6RQsKdt40eucm0FP DyYtfrobJCMWiuJgxdKOEWkuaOHOAgI+N3bWQU72i4xuj5e3JEQNXCz5Dc33H4dFX38+ zewFtjG4lmuMIL9QKcg5TdtVW8Sa6UyP4nz168NDQj/yCzNaqtwQ0zlUlIJUk7/7miQz fQEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=7BtDNJ62VCroJ1hEEB9/QkACl5VPHuEyEejHFR1zRhE=; b=yI9OiWCA45vUFG2ID510F3pOpMjBkjRUWTdhldtqmFN999QoSsMV7QhYNaBJckw7/k DOm6Np3/nIKbMlzxvNIOB0uv9xkT0OmFUezhqfhpoJjhRfYSHkckUlntodlr5C3DO/AV JtPJ59PVDyruFfIFiRx8pH2hOp0ZzrBF7MoT7NTi+NiA8Wja2vPszxApj5xj0IJlWdhQ yDUJah5KffCj2bb5txivsg/KMZYWLsaWaN4JF4kB4Oz9z/rAC4eZXMFMeY0kOh5pt/Sg R5MQouXs8mLpHazRAk1zwBkA7y6d9uKQSUTSefSTdpSlvgi58QvycxUyqqsY6x0M8umE +AIQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id w7-20020a0566022c0700b0073b120a6a65si41649iov.3.2023.02.15.20.32.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 15 Feb 2023 20:32:54 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 31G4WpV2016748 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 16 Feb 2023 05:32:52 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Cc: Adriaan Schmidt , Adriaan Schmidt Subject: Re: [PATCH] fix(isar-sstate): also handle zst files Date: Thu, 16 Feb 2023 07:32:51 +0300 Message-ID: <2626782.lGaqSPkdTl@home> In-Reply-To: <20230210153434.1024604-1-adriaan.schmidt@siemens.com> References: <20230210153434.1024604-1-adriaan.schmidt@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: aARr0WxQM885 In the email from Friday, 10 February 2023 18:34:34 +03 user Adriaan Schmidt wrote: > With bitbake 2.0, sstate artifacts have changed from tgz to tar.zst. > Our isar-sstate script needs to scan for those as well. The implementation > is backwards-compatible. > > Signed-off-by: Adriaan Schmidt > --- > scripts/isar-sstate | 24 ++++++++++++------------ > 1 file changed, 12 insertions(+), 12 deletions(-) > > diff --git a/scripts/isar-sstate b/scripts/isar-sstate > index 53d0541f..c14c2843 100755 > --- a/scripts/isar-sstate > +++ b/scripts/isar-sstate > @@ -40,7 +40,7 @@ followed by one of `w`, `d`, `h`, `m`, or `s` (for weeks, > days, hours, minutes, seconds, respectively). > > `--max-age` specifies up to which age artifacts should be kept in the > cache. -Anything older will be removed. Note that this only applies to the > `.tgz` files +Anything older will be removed. Note that this only applies > to the archive files containing the actual cached items, not the `.siginfo` > files containing the cache metadata (signatures and hashes). > To permit analysis of caching details using the `analyze` command, the > siginfo @@ -576,7 +576,7 @@ def arguments(): > '-v', '--verbose', default=False, action='store_true') > parser.add_argument( > '--max-age', type=str, default='1d', > - help="clean: remove tgz files older than MAX_AGE (a number followed > by w|d|h|m|s)") + help="clean: remove archive files older than > MAX_AGE (a number followed by w|d|h|m|s)") parser.add_argument( > '--max-sig-age', type=str, default=None, > help="clean: remove siginfo files older than MAX_SIG_AGE (defaults > to MAX_AGE)") @@ -664,21 +664,21 @@ def sstate_clean(target, max_age, > max_sig_age, verbose, **kwargs): links = [f for f in all_files if f.islink] > if links: > print(f"NOTE: we have links: {links}") > - tgz_files = [f for f in all_files if f.suffix == 'tgz'] > - siginfo_files = [f for f in all_files if f.suffix == 'tgz.siginfo'] > - del_tgz_files = [f for f in tgz_files if f.age >= max_age_seconds] > - del_tgz_hashes = [f.hash for f in del_tgz_files] > + archive_files = [f for f in all_files if f.suffix in ['tgz', > 'tar.zst']] + siginfo_files = [f for f in all_files if f.suffix in > ['tgz.siginfo', 'tar.zst.siginfo']] + del_archive_files = [f for f in > archive_files if f.age >= max_age_seconds] + del_archive_hashes = > [f.hash for f in del_archive_files] > del_siginfo_files = [f for f in siginfo_files if > - f.age >= max_sig_age_seconds or f.hash in > del_tgz_hashes] - print(f"INFO: found {len(tgz_files)} tgz files, > {len(del_tgz_files)} of which are older than {max_age}") + > f.age >= max_sig_age_seconds or f.hash in del_archive_hashes] + > print(f"INFO: found {len(archive_files)} archive files, > {len(del_archive_files)} of which are older than {max_age}") print(f"INFO: > found {len(siginfo_files)} siginfo files, {len(del_siginfo_files)} of which > " - f"correspond to old tgz files or are older than > {max_sig_age}") + f"correspond to old archive files or are older > than {max_sig_age}") > > - for f in del_tgz_files + del_siginfo_files: > + for f in del_archive_files + del_siginfo_files: > if verbose: > print(f"[DELETE] {f.path}") > target.delete(f.path) > - freed_gb = sum([x.size for x in del_tgz_files + del_siginfo_files]) / > 1024.0 / 1024.0 / 1024.0 + freed_gb = sum([x.size for x in > del_archive_files + del_siginfo_files]) / 1024.0 / 1024.0 / 1024.0 > print(f"INFO: freed {freed_gb:.02f} GB") > return 0 > > @@ -716,7 +716,7 @@ def sstate_info(target, verbose, **kwargs): > for k, entries in recipes.items(): > print(f"Cache entries for {k}:") > for pn in entries: > - artifacts = [f for f in all_files if f.pn == pn and f.task == > key_task[k] and f.suffix == 'tgz'] + artifacts = [f for f in > all_files if f.pn == pn and f.task == key_task[k] and f.suffix in ['tgz', > 'tar.zst']] print(f" - {pn}: {len(artifacts)} entries") > print("Other cache entries:") > for pn in others: Applied to next, thanks.