From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7192672744921628672 X-Received: by 2002:ac8:5417:0:b0:3a9:6b48:a132 with SMTP id b23-20020ac85417000000b003a96b48a132mr1726681qtq.148.1674674648516; Wed, 25 Jan 2023 11:24:08 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac8:4745:0:b0:3b6:3041:f96c with SMTP id k5-20020ac84745000000b003b63041f96cls15220317qtp.10.-pod-prod-gmail; Wed, 25 Jan 2023 11:24:08 -0800 (PST) X-Google-Smtp-Source: AMrXdXtTY4riSl8BS/kJhbzYVv6a1WyEcPbQ4RKqkcQFDZZrSrg2riiY3fTiNeQet374aCe46z8U X-Received: by 2002:ac8:6f09:0:b0:3a8:1789:73b1 with SMTP id bs9-20020ac86f09000000b003a8178973b1mr54316272qtb.2.1674674647974; Wed, 25 Jan 2023 11:24:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674674647; cv=none; d=google.com; s=arc-20160816; b=jt76ohxl+8RvWTl+o+6PEqfGriDvFpT9v/0DzZyHZhx7p65Jy+RyzkSkeepOUyROhs WP2Ll0Rur4blju6700vtDCrLXhL2T3uwIN9QvDTJB9ov2oH6OngiQAXD0bclKXuTfgxw 5k4keWILJJonf2wyyzcKwZowUvlb1FgsDHtQp7FbESblJqCdR82y9Da5Jn7bgI393h6Y 3MHxy0tR+rSWYSnqfqUOl0RCKwbgCF8feGuBHPnTzZBWW0J81soLMyI/cpwic15bBjlJ I+AXUgYbaDpYXdJ2R6Fdk7upqLn+UCs44bBnX/fZMKjzbX/Cq/e9vA+F00g3js/cw61Y 3thw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=tvD23R1WTohH/Mt856eVGCN6gwfxE87BlAVjqX9fyVs=; b=uYBa8u3W/+0JnzqStynhFoa4Ua8OyA+KzdOtKCHN5k8K4cn/CPFvACf3vmU+Pmq14c Uz5tQdrLpWBAvkDnbyx9pXzbtqN81b8M8VV9AUH+WkA4Ko6Bzvhtp5sjFXkTeU0fGq2h 8yCHJEzfBGgRYNHqK7lV0Mkrpkp/Akopd6YY5Pzms0bSBjfN2rXHCn6rusRXb3sTQpIw z2FcZ1sHyaQ1D+ybUntQfkQqLj4v2a9WOk/mHQnaFEmOY2eSiLyVKeSAkqVhIc4snqj8 fjHIO+W/nzlOT0FgiEgx+CGACet0hBCysedF1gXbDW5wNF77F/TkH2DlIKc3OEJi5uNX fADQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id f8-20020a05622a104800b003b63b24c87dsi664744qte.0.2023.01.25.11.24.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 25 Jan 2023 11:24:07 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from user-B660.. (IN-213-226-141-182.bitemobile.lv [213.226.141.182] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 30PJNdaW028378 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Jan 2023 20:24:04 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich , Adriaan Schmidt Subject: [PATCH v8 17/20] isar-sstate: adapt sstate maintenance script Date: Wed, 25 Jan 2023 21:23:34 +0200 Message-Id: <20230125192337.86869-18-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230125192337.86869-1-amikan@ilbers.de> References: <20230125192337.86869-1-amikan@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: CHffaXNMpU+S Fix sstate maintenance helper script to work with updated Bitbake siginfo data format. Signed-off-by: Anton Mikanovich Signed-off-by: Adriaan Schmidt --- scripts/isar-sstate | 70 ++++++++++++++++++++++----------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/scripts/isar-sstate b/scripts/isar-sstate index 8908b09..53d0541 100755 --- a/scripts/isar-sstate +++ b/scripts/isar-sstate @@ -126,7 +126,7 @@ import shutil import sys from tempfile import NamedTemporaryFile import time -import pickle +import json sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', 'bitbake', 'lib')) from bb.siggen import compare_sigfiles @@ -819,41 +819,41 @@ def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, ** continue sig_file = target.download(sig.path) - with open(sig_file, 'rb') as f: - try: - sigdata_raw = pickle.Unpickler(f) - sigdata = sigdata_raw.load() - except: - # invalid file or format... never mind - continue + try: + with bb.compress.zstd.open(sig_file, "rt", encoding="utf-8", num_threads=1) as f: + sigdata = json.load(f, object_hook=bb.siggen.SetDecoder) + bb.siggen.handle_renames(sigdata) + except: + # invalid file or format... never mind + continue - pn_issues = [] - for name, val in sigdata['varvals'].items(): - if not name[0].isupper(): - continue - if sigdata['basewhitelist'] and name in sigdata['basewhitelist'] or \ - sigdata['taskwhitelist'] and name in sigdata['taskwhitelist'] or \ - name in ADDITIONAL_IGNORED_VARNAMES: - continue - if not val: - continue - # remove leading whitespaces possibly added by appending - val = val.lstrip() - if not val[0] == '/': - continue - if val.startswith(build_dir): - pn_issues.append(f'\033[0;31m-> path in build-dir: {name} = "{val}"\033[0m') - hits_builddir += 1 - elif val.startswith(sources_dir): - pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m') - hits_srcdir += 1 - else: - hits_other += 1 - if verbose: - pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m') - if len(pn_issues) > 0: - print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m") - print('\n'.join(pn_issues)) + pn_issues = [] + for name, val in sigdata['varvals'].items(): + if not name[0].isupper(): + continue + if sigdata['basehash_ignore_vars'] and name in sigdata['basehash_ignore_vars'] or \ + sigdata['taskhash_ignore_tasks'] and name in sigdata['taskhash_ignore_tasks'] or \ + name in ADDITIONAL_IGNORED_VARNAMES: + continue + if not val: + continue + # remove leading whitespaces possibly added by appending + val = val.lstrip() + if not val[0] == '/': + continue + if val.startswith(build_dir): + pn_issues.append(f'\033[0;31m-> path in build-dir: {name} = "{val}"\033[0m') + hits_builddir += 1 + elif val.startswith(sources_dir): + pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m') + hits_srcdir += 1 + else: + hits_other += 1 + if verbose: + pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m') + if len(pn_issues) > 0: + print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m") + print('\n'.join(pn_issues)) target.release(sig_file) sum_hits = hits_srcdir + hits_builddir -- 2.34.1