From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7172560304523444224 X-Received: by 2002:a05:6e02:ec2:b0:300:5d50:15d3 with SMTP id i2-20020a056e020ec200b003005d5015d3mr1923207ilk.173.1672124473263; Mon, 26 Dec 2022 23:01:13 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6e02:f42:b0:304:a82a:30ef with SMTP id y2-20020a056e020f4200b00304a82a30efls2747079ilj.4.-pod-prod-gmail; Mon, 26 Dec 2022 23:01:12 -0800 (PST) X-Google-Smtp-Source: AMrXdXtXROljhgs/lwafYRF5Tk9+BX72vUWoN/jrjSFJ3nRNb3LaL2dIV3VOA7N9sLBsmZtQz9t3 X-Received: by 2002:a92:c151:0:b0:302:f5cd:d59a with SMTP id b17-20020a92c151000000b00302f5cdd59amr14580857ilh.1.1672124472838; Mon, 26 Dec 2022 23:01:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672124472; cv=none; d=google.com; s=arc-20160816; b=y8Kwicr1nhGTHI5pHlBF8d3OWcsJ54CelrHJyey+Y3lFQccGuSFyqElUCVbqio2JJL aMI0LU9q7a/Jr3FZjci2Me1sMqxPgNKRBxusK4L4UUfdX5OtXtefR2tb58yK7ZLWlbsS DmfDFERe4cRtHu00Wx4SRPN72aJIMNoJztRrmKcgnkTv45AyMOmoGvGPO0fVCRWI5szh Z6ANy4lM9zVB2FO0pwyNyW7nU2Bi1zQDLanth8GMx9XlKu9GUG9cOP3h4HyOw5yQh4DW EQXWJJxz8lG9PzffBjxW+IUXJv2eyQ/RWQ0OIDoG0BvKU90FOVG4+PhtFZaP3XVOgWpJ KcLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=plydLOYNi4BJ9B+zKiUfw6vylzeeqy4jLcvJJoZwl5c=; b=tRdsaPQLV/35tQYbUi+0zXalftWnM3hw3sNXl72172Y0VDWRx+tt5wL1SQJ+baitDN o2Xw4uUn/F8NDBzmn+cYuxu7Gu34PMUdkJkZrcni91z3Zd1wzjkyRDb7eLu15xlSA8+u HLa5X0PI45HFuXE3jSwVxYLo2lvOfXb5wpwXui4dF53FGoCh2MqCKwCjFda5efgOkVYL WGQCVWgA2XI+1pO5bgdaY4nQzMfXgQ3S9ClEgDRzZn5PMQ4Dc5MszVuErraz/PkIHADF AAL6hX9x3kqM9xU5hB8HksBXcVgvlPwMjUQv8XlodcMKmjfXHFcoFK+P+58c/d5olj5z BbjA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id i8-20020a02cc48000000b003636f49184dsi1046141jaq.7.2022.12.26.23.01.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 26 Dec 2022 23:01:12 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from alena-nb.promwad.com ([193.228.193.9]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2BR70jA2030666 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 27 Dec 2022 08:01:10 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich , Adriaan Schmidt Subject: [PATCH v7 17/20] isar-sstate: adapt sstate maintenance script Date: Tue, 27 Dec 2022 10:00:36 +0300 Message-Id: <20221227070039.23496-18-amikan@ilbers.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221227070039.23496-1-amikan@ilbers.de> References: <20221227070039.23496-1-amikan@ilbers.de> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: DkVHM9jR/01X Fix sstate maintenance helper script to work with updated Bitbake siginfo data format. Signed-off-by: Anton Mikanovich Signed-off-by: Adriaan Schmidt --- scripts/isar-sstate | 70 ++++++++++++++++++++++----------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/scripts/isar-sstate b/scripts/isar-sstate index 8908b09f..53d0541f 100755 --- a/scripts/isar-sstate +++ b/scripts/isar-sstate @@ -126,7 +126,7 @@ import shutil import sys from tempfile import NamedTemporaryFile import time -import pickle +import json sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', 'bitbake', 'lib')) from bb.siggen import compare_sigfiles @@ -819,41 +819,41 @@ def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, ** continue sig_file = target.download(sig.path) - with open(sig_file, 'rb') as f: - try: - sigdata_raw = pickle.Unpickler(f) - sigdata = sigdata_raw.load() - except: - # invalid file or format... never mind - continue + try: + with bb.compress.zstd.open(sig_file, "rt", encoding="utf-8", num_threads=1) as f: + sigdata = json.load(f, object_hook=bb.siggen.SetDecoder) + bb.siggen.handle_renames(sigdata) + except: + # invalid file or format... never mind + continue - pn_issues = [] - for name, val in sigdata['varvals'].items(): - if not name[0].isupper(): - continue - if sigdata['basewhitelist'] and name in sigdata['basewhitelist'] or \ - sigdata['taskwhitelist'] and name in sigdata['taskwhitelist'] or \ - name in ADDITIONAL_IGNORED_VARNAMES: - continue - if not val: - continue - # remove leading whitespaces possibly added by appending - val = val.lstrip() - if not val[0] == '/': - continue - if val.startswith(build_dir): - pn_issues.append(f'\033[0;31m-> path in build-dir: {name} = "{val}"\033[0m') - hits_builddir += 1 - elif val.startswith(sources_dir): - pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m') - hits_srcdir += 1 - else: - hits_other += 1 - if verbose: - pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m') - if len(pn_issues) > 0: - print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m") - print('\n'.join(pn_issues)) + pn_issues = [] + for name, val in sigdata['varvals'].items(): + if not name[0].isupper(): + continue + if sigdata['basehash_ignore_vars'] and name in sigdata['basehash_ignore_vars'] or \ + sigdata['taskhash_ignore_tasks'] and name in sigdata['taskhash_ignore_tasks'] or \ + name in ADDITIONAL_IGNORED_VARNAMES: + continue + if not val: + continue + # remove leading whitespaces possibly added by appending + val = val.lstrip() + if not val[0] == '/': + continue + if val.startswith(build_dir): + pn_issues.append(f'\033[0;31m-> path in build-dir: {name} = "{val}"\033[0m') + hits_builddir += 1 + elif val.startswith(sources_dir): + pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m') + hits_srcdir += 1 + else: + hits_other += 1 + if verbose: + pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m') + if len(pn_issues) > 0: + print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m") + print('\n'.join(pn_issues)) target.release(sig_file) sum_hits = hits_srcdir + hits_builddir -- 2.17.1