RE: Isar can't reuse sstate-cache build from different isar location

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

Hi,

I believe the problem was introduced in 9e7c63ee6. This causes absolute paths to end up in SRC_URI.

Can you try this patch:
---
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 8af73a9b..28264024 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -156,7 +156,7 @@ def get_aptsources_list(d):
     for p in apt_sources_list:
         try:
             f = bb.parse.resolve_file(p, d)
-            ret.append(f)
+            ret.append(p)
         except FileNotFoundError as e:
             bb.fatal(os.strerror(errno.ENOENT) + ' "' + p + '"')
     return ret
---

I haven't tested it. If this is not it I'll have to dig a little deeper.

Generally re-use of sstate artifacts is supported, and we were careful to avoid absolute paths in variables.
In fact, the isar-sstate script has a "lint" command to detect them, but that fails here, because the
path has a "file://" prefix.

Thanks,
Adriaan

Uladzimir Bely, Thursday, September 7, 2023 1:32 PM: 
> I tried to use the same SSTATE_DIR for different isar locations on the
> same machine under the same user and found that it does not work.
> 
> Quick steps to reproduce it:
> 
> ## Prepare build environment with two copies of isar:
> 
> ```
> ~ $ mkdir -m 777 test-sstate
> ~ $ cd test-sstate/
> ~/test-sstate $ docker run --privileged -it -v .:/build
> ghcr.io/siemens/kas/kas-isar:4.0
> builder@ffe5274283d3:~$ cd /build
> builder@ffe5274283d3:/build$ git clone
> https://github.com/ilbers/isar.git isar-1
> builder@ffe5274283d3:/build$ git clone
> https://github.com/ilbers/isar.git isar-2
> ```
> 
> ## Build isar-1:
> 
> ```
> builder@ffe5274283d3:/build$ cd /build/isar-1
> builder@ffe5274283d3:/build/isar-1$ . isar-init-build-env
> builder@ffe5274283d3:/build/isar-1/build$ echo
> 'SSTATE_DIR="/build/sstate-cache"' >> conf/local.conf
> builder@ffe5274283d3:/build/isar-1/build$ echo
> 'DL_DIR="/build/downloads"' >> conf/local.conf
> builder@ffe5274283d3:/build/isar-1/build$ time bitbake mc:qemuamd64-
> bullseye:isar-image-base
> real    4m38.759s
> user    0m4.248s
> sys     0m1.068s
> builder@ffe5274283d3:/build/isar-1/build$ du -sh /build/downloads
> 229M    /build/downloads
> builder@ffe5274283d3:/build/isar-1/build$ du -sh /build/sstate-cache
> 1.1G    /build/sstate-cache
> ```
> 
> ## Build isar-2:
> 
> ```
> builder@ffe5274283d3:/build$ cd /build/isar-2
> builder@ffe5274283d3:/build/isar-2$ . isar-init-build-env
> builder@ffe5274283d3:/build/isar-2/build$ echo
> 'SSTATE_DIR="/build/sstate-cache"' >> conf/local.conf
> builder@ffe5274283d3:/build/isar-2/build$ echo
> 'DL_DIR="/build/downloads"' >> conf/local.conf
> builder@ffe5274283d3:/build/isar-2/build$ time bitbake mc:qemuamd64-
> bullseye:isar-image-base
> real    4m32.314s
> user    0m3.975s
> sys     0m0.898s
> builder@ffe5274283d3:/build/isar-2/build$ du -sh /build/downloads
> 229M    /build/downloads
> builder@ffe5274283d3:/build/isar-2/build$ du -sh /build/sstate-cache/
> 2.2G    /build/sstate-cache/
> ```
> 
> Second build took almost the same time as first one. Sstate cache was
> not reused, second build just added new files to the cache.
> 
> I tried similar approach with Yocto, and it works in it.
> 
> After debugging, I have concluded, that this issue comes from
> "bootstrap" task that is not sstate-cache compatible. It's caused by
> our manipulations with SRC_URI we do in it, so that it's different for
> both builds. E.g.:
> 
> SRC_URI="file://locale file://chroot-setup.sh file:///build/isar-
> 1/meta/conf/distro/debian-bullseye.list"
> SRC_URI="file://locale file://chroot-setup.sh file:///build/isar-
> 2/meta/conf/distro/debian-bullseye.list"
> 
> SRC_URI is a variable that is tracked in sstate-cache. Since the last
> part (that we add "on fly" with get_aptsources_list() function) is
> different for both builds, sstate-cache can't be reused for "isar-
> bootstrap-<host,target>" and for all other tasks since they depend on
> those ones.
> 
> Any ideas how to properly deal with this?
> 
> By the way, kas users are not affected, since fixed paths are used
> (e.g., "/work/isar", "/build") under the container.
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-
> users/7f715c07ed85772dc5e2eafcb41f3ace734de9d4.camel%40ilbers.de.