From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7172560304523444224 X-Received: by 2002:a05:6808:24a:b0:359:a4de:1d3a with SMTP id m10-20020a056808024a00b00359a4de1d3amr127312oie.138.1670936064702; Tue, 13 Dec 2022 04:54:24 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6870:bb0a:b0:143:1d71:23f2 with SMTP id nw10-20020a056870bb0a00b001431d7123f2ls5377390oab.9.-pod-prod-gmail; Tue, 13 Dec 2022 04:54:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf7kdHPLTYXq0Kf2iQudJX+DN2ZsrsKsH3vlxGqPLhU8oa30jggymKatPkF907j/CqUrPfz4 X-Received: by 2002:a05:6870:b10:b0:144:dad7:80f3 with SMTP id lh16-20020a0568700b1000b00144dad780f3mr8025202oab.37.1670936064206; Tue, 13 Dec 2022 04:54:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670936064; cv=none; d=google.com; s=arc-20160816; b=H5LXNikqKwNlauHA+2mZzvk6CHjI21DPUk/ygP5XIADj4hQU1qc6Imc0q0iVHwLOex rUMZyov1MWk51MBdZgqcq/ujkNM5Nih9x8ptOVXgeOlCvKnt6sRQOnZOFXiTPty7tR0Y SJMJoAn6ZxmkNA0q071uk5vj/rfy3wnWXM/p54JMcDCbkdYtVnfPF/pB+hmsMXkQ+cMI pAMujQ3+sAbm/il2TB2rA5pIQIlEh01HHd+IhtZ3tl0H9OgZBh1Zsg8EX7RjuXs1XF1m QZK09hPmVQnST1TvfZrWXNcKYSUd5Q+t0JCfmy5BxbJXt+WRkEHG4BpXvVQD9jQtRMD7 ND2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=plydLOYNi4BJ9B+zKiUfw6vylzeeqy4jLcvJJoZwl5c=; b=h5yPvf1hfZSHoSsrO1JIOoyCM5X4xBVgE6W4rT4wZJBy6alYMQjQoTxAi8oRiD66KD wX6ftf8hohmyZqNlZofCRC8o1ZRHPjhtGZX7iUnWaH60rnNZvPHTmj92C/6liYPTK5f1 uAkH/HjuLGnprjz78H3gIfkuXh7tzOJQI+ur7a26cEpQ1cmp78sUjsKdzys66tLn5VMW abzCriTEUj71fkY8m91cJJIdGcNJjYCHu1RVrYjPDdsY/qZA7JaKbBEc/DnbXRVVjeF2 Us9+J2cFvraE9Bu1g4/wQJhpXNLnz916SPy1KPA1nyNMX0el7BKBl4E7wBpZG/KRn7yx bJEg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id e7-20020a056870238700b001371e49ab90si360349oap.3.2022.12.13.04.54.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 13 Dec 2022 04:54:24 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from alena-nb.promwad.com ([194.49.52.147]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2BDCrDuk001200 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Dec 2022 13:54:19 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich , Adriaan Schmidt Subject: [PATCH v6 18/21] isar-sstate: adopt sstate maintenance script Date: Tue, 13 Dec 2022 15:53:02 +0300 Message-Id: <20221213125305.10984-19-amikan@ilbers.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221213125305.10984-1-amikan@ilbers.de> References: <20221213125305.10984-1-amikan@ilbers.de> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: JmeEDP1zlrVr Fix sstate maintenance helper script to work with updated Bitbake siginfo data format. Signed-off-by: Anton Mikanovich Signed-off-by: Adriaan Schmidt --- scripts/isar-sstate | 70 ++++++++++++++++++++++----------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/scripts/isar-sstate b/scripts/isar-sstate index 8908b09f..53d0541f 100755 --- a/scripts/isar-sstate +++ b/scripts/isar-sstate @@ -126,7 +126,7 @@ import shutil import sys from tempfile import NamedTemporaryFile import time -import pickle +import json sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', 'bitbake', 'lib')) from bb.siggen import compare_sigfiles @@ -819,41 +819,41 @@ def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, ** continue sig_file = target.download(sig.path) - with open(sig_file, 'rb') as f: - try: - sigdata_raw = pickle.Unpickler(f) - sigdata = sigdata_raw.load() - except: - # invalid file or format... never mind - continue + try: + with bb.compress.zstd.open(sig_file, "rt", encoding="utf-8", num_threads=1) as f: + sigdata = json.load(f, object_hook=bb.siggen.SetDecoder) + bb.siggen.handle_renames(sigdata) + except: + # invalid file or format... never mind + continue - pn_issues = [] - for name, val in sigdata['varvals'].items(): - if not name[0].isupper(): - continue - if sigdata['basewhitelist'] and name in sigdata['basewhitelist'] or \ - sigdata['taskwhitelist'] and name in sigdata['taskwhitelist'] or \ - name in ADDITIONAL_IGNORED_VARNAMES: - continue - if not val: - continue - # remove leading whitespaces possibly added by appending - val = val.lstrip() - if not val[0] == '/': - continue - if val.startswith(build_dir): - pn_issues.append(f'\033[0;31m-> path in build-dir: {name} = "{val}"\033[0m') - hits_builddir += 1 - elif val.startswith(sources_dir): - pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m') - hits_srcdir += 1 - else: - hits_other += 1 - if verbose: - pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m') - if len(pn_issues) > 0: - print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m") - print('\n'.join(pn_issues)) + pn_issues = [] + for name, val in sigdata['varvals'].items(): + if not name[0].isupper(): + continue + if sigdata['basehash_ignore_vars'] and name in sigdata['basehash_ignore_vars'] or \ + sigdata['taskhash_ignore_tasks'] and name in sigdata['taskhash_ignore_tasks'] or \ + name in ADDITIONAL_IGNORED_VARNAMES: + continue + if not val: + continue + # remove leading whitespaces possibly added by appending + val = val.lstrip() + if not val[0] == '/': + continue + if val.startswith(build_dir): + pn_issues.append(f'\033[0;31m-> path in build-dir: {name} = "{val}"\033[0m') + hits_builddir += 1 + elif val.startswith(sources_dir): + pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m') + hits_srcdir += 1 + else: + hits_other += 1 + if verbose: + pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m') + if len(pn_issues) > 0: + print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m") + print('\n'.join(pn_issues)) target.release(sig_file) sum_hits = hits_srcdir + hits_builddir -- 2.17.1