public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Uladzimir Bely <ubely@ilbers.de>
To: isar-users@googlegroups.com
Subject: Isar can't reuse sstate-cache build from different isar location
Date: Thu, 07 Sep 2023 14:31:31 +0300	[thread overview]
Message-ID: <7f715c07ed85772dc5e2eafcb41f3ace734de9d4.camel@ilbers.de> (raw)

I tried to use the same SSTATE_DIR for different isar locations on the
same machine under the same user and found that it does not work.

Quick steps to reproduce it:

## Prepare build environment with two copies of isar:

```
~ $ mkdir -m 777 test-sstate
~ $ cd test-sstate/
~/test-sstate $ docker run --privileged -it -v .:/build
ghcr.io/siemens/kas/kas-isar:4.0
builder@ffe5274283d3:~$ cd /build
builder@ffe5274283d3:/build$ git clone
https://github.com/ilbers/isar.git isar-1
builder@ffe5274283d3:/build$ git clone
https://github.com/ilbers/isar.git isar-2
```

## Build isar-1:

```
builder@ffe5274283d3:/build$ cd /build/isar-1
builder@ffe5274283d3:/build/isar-1$ . isar-init-build-env 
builder@ffe5274283d3:/build/isar-1/build$ echo
'SSTATE_DIR="/build/sstate-cache"' >> conf/local.conf 
builder@ffe5274283d3:/build/isar-1/build$ echo
'DL_DIR="/build/downloads"' >> conf/local.conf
builder@ffe5274283d3:/build/isar-1/build$ time bitbake mc:qemuamd64-
bullseye:isar-image-base
real    4m38.759s
user    0m4.248s
sys     0m1.068s
builder@ffe5274283d3:/build/isar-1/build$ du -sh /build/downloads
229M    /build/downloads
builder@ffe5274283d3:/build/isar-1/build$ du -sh /build/sstate-cache
1.1G    /build/sstate-cache
```

## Build isar-2:

```
builder@ffe5274283d3:/build$ cd /build/isar-2
builder@ffe5274283d3:/build/isar-2$ . isar-init-build-env 
builder@ffe5274283d3:/build/isar-2/build$ echo
'SSTATE_DIR="/build/sstate-cache"' >> conf/local.conf 
builder@ffe5274283d3:/build/isar-2/build$ echo
'DL_DIR="/build/downloads"' >> conf/local.conf 
builder@ffe5274283d3:/build/isar-2/build$ time bitbake mc:qemuamd64-
bullseye:isar-image-base
real    4m32.314s
user    0m3.975s
sys     0m0.898s
builder@ffe5274283d3:/build/isar-2/build$ du -sh /build/downloads
229M    /build/downloads
builder@ffe5274283d3:/build/isar-2/build$ du -sh /build/sstate-cache/
2.2G    /build/sstate-cache/
```

Second build took almost the same time as first one. Sstate cache was
not reused, second build just added new files to the cache.

I tried similar approach with Yocto, and it works in it.

After debugging, I have concluded, that this issue comes from
"bootstrap" task that is not sstate-cache compatible. It's caused by
our manipulations with SRC_URI we do in it, so that it's different for
both builds. E.g.:

SRC_URI="file://locale file://chroot-setup.sh file:///build/isar-
1/meta/conf/distro/debian-bullseye.list"
SRC_URI="file://locale file://chroot-setup.sh file:///build/isar-
2/meta/conf/distro/debian-bullseye.list"

SRC_URI is a variable that is tracked in sstate-cache. Since the last
part (that we add "on fly" with get_aptsources_list() function) is
different for both builds, sstate-cache can't be reused for "isar-
bootstrap-<host,target>" and for all other tasks since they depend on
those ones.

Any ideas how to properly deal with this?

By the way, kas users are not affected, since fixed paths are used
(e.g., "/work/isar", "/build") under the container.

             reply	other threads:[~2023-09-07 11:31 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-07 11:31 Uladzimir Bely [this message]
2023-09-07 13:29 ` Schmidt, Adriaan
2023-09-07 15:23   ` Uladzimir Bely
2023-09-07 15:27     ` Uladzimir Bely
2023-09-07 15:56       ` Uladzimir Bely

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7f715c07ed85772dc5e2eafcb41f3ace734de9d4.camel@ilbers.de \
    --to=ubely@ilbers.de \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox