public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Anton Mikanovich <amikan@ilbers.de>
To: isar-users@googlegroups.com
Cc: Anton Mikanovich <amikan@ilbers.de>,
	Adriaan Schmidt <adriaan.schmidt@siemens.com>
Subject: [PATCH v8 17/20] isar-sstate: adapt sstate maintenance script
Date: Wed, 25 Jan 2023 21:23:34 +0200	[thread overview]
Message-ID: <20230125192337.86869-18-amikan@ilbers.de> (raw)
In-Reply-To: <20230125192337.86869-1-amikan@ilbers.de>

Fix sstate maintenance helper script to work with updated Bitbake
siginfo data format.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
 scripts/isar-sstate | 70 ++++++++++++++++++++++-----------------------
 1 file changed, 35 insertions(+), 35 deletions(-)

diff --git a/scripts/isar-sstate b/scripts/isar-sstate
index 8908b09..53d0541 100755
--- a/scripts/isar-sstate
+++ b/scripts/isar-sstate
@@ -126,7 +126,7 @@ import shutil
 import sys
 from tempfile import NamedTemporaryFile
 import time
-import pickle
+import json
 
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), '..', 'bitbake', 'lib'))
 from bb.siggen import compare_sigfiles
@@ -819,41 +819,41 @@ def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, pedantic, **
             continue
 
         sig_file = target.download(sig.path)
-        with open(sig_file, 'rb') as f:
-            try:
-                sigdata_raw = pickle.Unpickler(f)
-                sigdata = sigdata_raw.load()
-            except:
-                # invalid file or format... never mind
-                continue
+        try:
+            with bb.compress.zstd.open(sig_file, "rt", encoding="utf-8", num_threads=1) as f:
+                sigdata = json.load(f, object_hook=bb.siggen.SetDecoder)
+            bb.siggen.handle_renames(sigdata)
+        except:
+            # invalid file or format... never mind
+            continue
 
-            pn_issues = []
-            for name, val in sigdata['varvals'].items():
-                if not name[0].isupper():
-                    continue
-                if sigdata['basewhitelist'] and name in sigdata['basewhitelist'] or \
-                   sigdata['taskwhitelist'] and name in sigdata['taskwhitelist'] or \
-                   name in ADDITIONAL_IGNORED_VARNAMES:
-                    continue
-                if not val:
-                    continue
-                # remove leading whitespaces possibly added by appending
-                val = val.lstrip()
-                if not val[0] == '/':
-                    continue
-                if val.startswith(build_dir):
-                    pn_issues.append(f'\033[0;31m-> path in build-dir:   {name} = "{val}"\033[0m')
-                    hits_builddir += 1
-                elif val.startswith(sources_dir):
-                    pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m')
-                    hits_srcdir += 1
-                else:
-                    hits_other += 1
-                    if verbose:
-                        pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m')
-            if len(pn_issues) > 0:
-                print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m")
-                print('\n'.join(pn_issues))
+        pn_issues = []
+        for name, val in sigdata['varvals'].items():
+            if not name[0].isupper():
+                continue
+            if sigdata['basehash_ignore_vars'] and name in sigdata['basehash_ignore_vars'] or \
+               sigdata['taskhash_ignore_tasks'] and name in sigdata['taskhash_ignore_tasks'] or \
+               name in ADDITIONAL_IGNORED_VARNAMES:
+                continue
+            if not val:
+                continue
+            # remove leading whitespaces possibly added by appending
+            val = val.lstrip()
+            if not val[0] == '/':
+                continue
+            if val.startswith(build_dir):
+                pn_issues.append(f'\033[0;31m-> path in build-dir:   {name} = "{val}"\033[0m')
+                hits_builddir += 1
+            elif val.startswith(sources_dir):
+                pn_issues.append(f'\033[0;31m-> path in sources-dir: {name} = "{val}"\033[0m')
+                hits_srcdir += 1
+            else:
+                hits_other += 1
+                if verbose:
+                    pn_issues.append(f'\033[0;34m-> other absolute path: {name} = "{val}"\033[0m')
+        if len(pn_issues) > 0:
+            print(f"\033[1;33m==== issues found in {sig.arch}:{sig.pn}:{sig.task} ({sig.hash[:8]}) ====\033[0m")
+            print('\n'.join(pn_issues))
         target.release(sig_file)
 
     sum_hits = hits_srcdir + hits_builddir
-- 
2.34.1


  parent reply	other threads:[~2023-01-25 19:24 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-25 19:23 [PATCH v8 00/20] Migrate to Bitbake 2.0 Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 01/20] meta: change deprecated parse calls Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 02/20] scripts/contrib: add override conversion script Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 03/20] scripts/contrib: configure " Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 04/20] meta-isar: set default branch names Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 05/20] meta: remove non recommended syntax Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 06/20] bitbake: update to Bitbake 2.0.5 Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 07/20] meta: update bitbake variables Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 08/20] bitbake.conf: align hash vars with openembedded Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 09/20] meta: mark network and sudo tasks Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 10/20] meta: update overrides syntax Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 11/20] sstate: update bbclass Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 12/20] bitbake.conf: declare default XZ and ZSTD options Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 13/20] Revert "devshell: Use different termination test to avoid warnings" Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 14/20] meta: align with OE-core libraries update Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 15/20] Revert "Revert "devshell: Use different termination test to avoid warnings"" Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 16/20] CI: adapt tests to syntax change Anton Mikanovich
2023-01-25 19:23 ` Anton Mikanovich [this message]
2023-01-25 19:23 ` [PATCH v8 18/20] doc: require zstd tool Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 19/20] RECIPE-API-CHANGELOG: add tips after bitbake version update Anton Mikanovich
2023-01-25 19:23 ` [PATCH v8 20/20] docs: update override syntax Anton Mikanovich
2023-01-25 23:43 ` [PATCH v8 00/20] Migrate to Bitbake 2.0 Roberto A. Foglietta
2023-01-26  7:29   ` Anton Mikanovich
2023-01-26 13:23     ` Roberto A. Foglietta
2023-01-26 19:59       ` Henning Schild
2023-01-27  4:09         ` Roberto A. Foglietta
2023-01-31 11:26 ` Uladzimir Bely
2023-02-01  6:17 ` Uladzimir Bely
2023-02-02  9:02   ` Florian Bezdeka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230125192337.86869-18-amikan@ilbers.de \
    --to=amikan@ilbers.de \
    --cc=adriaan.schmidt@siemens.com \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox