[PATCH 1/2] isar-sstate lint: strip variable value

[PATCH 1/2] isar-sstate lint: strip variable value

[Felix Moessbauer]

By stripping any leading whitespaces, the script now also detects
cachability problems in variables that are just appended to.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 scripts/isar-sstate | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/scripts/isar-sstate b/scripts/isar-sstate
index 3a1945e7..5972144d 100755
--- a/scripts/isar-sstate
+++ b/scripts/isar-sstate
@@ -824,9 +824,12 @@ def sstate_lint(target, verbose, sources_dir, build_dir, exit_code, **kwargs):
                    sigdata['taskwhitelist'] and name in sigdata['taskwhitelist'] or \
                    name in ADDITIONAL_IGNORED_VARNAMES:
                     continue
-                if not val or not val[0] == '/':
+                if not val:
+                    continue
+                # remove leading whitespaces possibly added by appending
+                val = val.lstrip()
+                if not val[0] == '/':
                     continue
-                task = sigdata['task']
                 if val.startswith(build_dir):
                     pn_issues.append(f'\033[0;31m-> path in build-dir:   {name} = "{val}"\033[0m')
                     hits_builddir += 1
-- 
2.30.2

[PATCH 2/2] improve cachability of isar-bootstrap

[Felix Moessbauer]

This patch makes all paths used in the bootstrapping relative
to TOPDIR to be able to cache the isar-bootstrap task.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 604cd24c..9f512032 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -35,8 +35,8 @@ inherit deb-dl-dir
 
 python () {
     distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split()
-
     third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split()
+    topdir = d.getVar("TOPDIR", True)
 
     # The cached repo key can be both for bootstrapping and apt package
     # installation afterwards. However, debootstrap will include the key into
@@ -50,14 +50,14 @@ python () {
     for key in distro_bootstrap_keys:
         d.appendVar("SRC_URI", " %s" % key)
         fetcher = bb.fetch2.Fetch([key], d)
-        filename = fetcher.localpath(key)
-        d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " %s" % filename)
+        filename = os.path.relpath(fetcher.localpath(key), topdir)
+        d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", "${TOPDIR}/%s" % filename)
 
     for key in third_party_apt_keys:
         d.appendVar("SRC_URI", " %s" % key)
         fetcher = bb.fetch2.Fetch([key], d)
-        filename = fetcher.localpath(key)
-        d.appendVar("THIRD_PARTY_APT_KEYFILES", " %s" % filename)
+        filename = os.path.relpath(fetcher.localpath(key), topdir)
+        d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename)
 
     distro_apt_sources = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES", True) or ""
     for file in distro_apt_sources.split():
-- 
2.30.2

Re: [PATCH 2/2] improve cachability of isar-bootstrap

[Anton Mikanovich]

09.08.2022 17:07, Felix Moessbauer wrote:
> This patch makes all paths used in the bootstrapping relative
> to TOPDIR to be able to cache the isar-bootstrap task.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>   meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> index 604cd24c..9f512032 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> @@ -35,8 +35,8 @@ inherit deb-dl-dir
>   
>   python () {
>       distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split()
> -
>       third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or "").split()
> +    topdir = d.getVar("TOPDIR", True)
>   
>       # The cached repo key can be both for bootstrapping and apt package
>       # installation afterwards. However, debootstrap will include the key into
> @@ -50,14 +50,14 @@ python () {
>       for key in distro_bootstrap_keys:
>           d.appendVar("SRC_URI", " %s" % key)
>           fetcher = bb.fetch2.Fetch([key], d)
> -        filename = fetcher.localpath(key)
> -        d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " %s" % filename)
> +        filename = os.path.relpath(fetcher.localpath(key), topdir)
> +        d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", "${TOPDIR}/%s" % filename)
>   
>       for key in third_party_apt_keys:
>           d.appendVar("SRC_URI", " %s" % key)
>           fetcher = bb.fetch2.Fetch([key], d)
> -        filename = fetcher.localpath(key)
> -        d.appendVar("THIRD_PARTY_APT_KEYFILES", " %s" % filename)
> +        filename = os.path.relpath(fetcher.localpath(key), topdir)
> +        d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" % filename)
>   
>       distro_apt_sources = d.getVar(d.getVar("DISTRO_VARS_PREFIX") + "DISTRO_APT_SOURCES", True) or ""
>       for file in distro_apt_sources.split():

Hello Felix,

This patch brakes keys management in some cases:

$ bitbake -c do_generate_keyrings 
mc:qemuriscv64-sid-ports:isar-bootstrap-target
...
| gpg: can't open 
'/home/amikan/isar/build/../../downloads/archive_2022.key/home/amikan/isar/build/../../downloads/archive_2023.key': 
Not a directory

There can be more then one keyfile just like qemuriscv64 has.

P.S. qemuriscv64-sid-ports target is not fully buildable currently even 
on the
latest next, we are working on it already.

RE: [PATCH 2/2] improve cachability of isar-bootstrap

[Moessbauer, Felix]

> -----Original Message-----
> From: Anton Mikanovich <amikan@ilbers.de>
> Sent: Friday, August 12, 2022 9:30 AM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar-
> users@googlegroups.com
> Cc: Schmidt, Adriaan (T CED SES-DE) <adriaan.schmidt@siemens.com>
> Subject: Re: [PATCH 2/2] improve cachability of isar-bootstrap
> 
> 09.08.2022 17:07, Felix Moessbauer wrote:
> > This patch makes all paths used in the bootstrapping relative to
> > TOPDIR to be able to cache the isar-bootstrap task.
> >
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >   meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 10 +++++-----
> >   1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > index 604cd24c..9f512032 100644
> > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > @@ -35,8 +35,8 @@ inherit deb-dl-dir
> >
> >   python () {
> >       distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or
> > "").split()
> > -
> >       third_party_apt_keys = (d.getVar("THIRD_PARTY_APT_KEYS") or
> > "").split()
> > +    topdir = d.getVar("TOPDIR", True)
> >
> >       # The cached repo key can be both for bootstrapping and apt package
> >       # installation afterwards. However, debootstrap will include the
> > key into @@ -50,14 +50,14 @@ python () {
> >       for key in distro_bootstrap_keys:
> >           d.appendVar("SRC_URI", " %s" % key)
> >           fetcher = bb.fetch2.Fetch([key], d)
> > -        filename = fetcher.localpath(key)
> > -        d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", " %s" % filename)
> > +        filename = os.path.relpath(fetcher.localpath(key), topdir)
> > +        d.appendVar("DISTRO_BOOTSTRAP_KEYFILES", "${TOPDIR}/%s" %
> > + filename)
> >
> >       for key in third_party_apt_keys:
> >           d.appendVar("SRC_URI", " %s" % key)
> >           fetcher = bb.fetch2.Fetch([key], d)
> > -        filename = fetcher.localpath(key)
> > -        d.appendVar("THIRD_PARTY_APT_KEYFILES", " %s" % filename)
> > +        filename = os.path.relpath(fetcher.localpath(key), topdir)
> > +        d.appendVar("THIRD_PARTY_APT_KEYFILES", " ${TOPDIR}/%s" %
> > + filename)
> >
> >       distro_apt_sources = d.getVar(d.getVar("DISTRO_VARS_PREFIX") +
> "DISTRO_APT_SOURCES", True) or ""
> >       for file in distro_apt_sources.split():
> 
> Hello Felix,
> 
> This patch brakes keys management in some cases:
> 
> $ bitbake -c do_generate_keyrings
> mc:qemuriscv64-sid-ports:isar-bootstrap-target
> ...
> | gpg: can't open
> '/home/amikan/isar/build/../../downloads/archive_2022.key/home/amikan/isar
> /build/../../downloads/archive_2023.key':
> Not a directory

Just found the issue. When appending to the variable, a leading space was missing.
Will send a v2 soon.

Felix

> 
> There can be more then one keyfile just like qemuriscv64 has.
> 
> P.S. qemuriscv64-sid-ports target is not fully buildable currently even on the
> latest next, we are working on it already.