public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Adriaan Schmidt <adriaan.schmidt@siemens.com>
To: isar-users@googlegroups.com
Cc: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Subject: [PATCH v4 09/10] sstate: add caching to rootfs
Date: Tue, 26 Oct 2021 14:28:10 +0200	[thread overview]
Message-ID: <20211026122811.2654125-10-adriaan.schmidt@siemens.com> (raw)
In-Reply-To: <20211026122811.2654125-1-adriaan.schmidt@siemens.com>

We cache the output of do_rootfs. Again "sudo tar" to preserve
ownerships and permissions. In addition, it can happen that there are
active mounts while tar'ing the rootfs (there shouldn't be, but I've
seen cases where deb-src is mounted). Unfortunately --one-file-system
does not stop at bind mounts, so we use a trick with a temporary mount
to ensure that we don't package more than we want.

Special care needs to be taken with image and initramfs, which are
MACHINE specific. So they adapt sstate manifest location to ensure
that there is no false sharing.

Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
 meta/classes/image.bbclass     |  3 +++
 meta/classes/initramfs.bbclass |  3 +++
 meta/classes/rootfs.bbclass    | 27 +++++++++++++++++++++++++++
 3 files changed, 33 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 5a0f32e..eedc160 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -6,6 +6,9 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_TYPE
 STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_TYPE}/${PV}-${PR}"
 STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_TYPE}/*-*"
 
+# Sstate also needs to be machine-specific
+SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
+
 IMAGE_INSTALL ?= ""
 IMAGE_TYPE    ?= "ext4-img"
 IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
diff --git a/meta/classes/initramfs.bbclass b/meta/classes/initramfs.bbclass
index 10a642b..2cec85d 100644
--- a/meta/classes/initramfs.bbclass
+++ b/meta/classes/initramfs.bbclass
@@ -5,6 +5,9 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
 STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
 STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
 
+# Sstate also needs to be machine-specific
+SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
+
 INITRAMFS_INSTALL ?= ""
 INITRAMFS_PREINSTALL ?= ""
 INITRAMFS_ROOTFS ?= "${WORKDIR}/rootfs"
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index b38de66..e0604e0 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -265,3 +265,30 @@ python do_rootfs() {
     pass
 }
 addtask rootfs before do_build
+
+do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config"
+
+SSTATETASKS += "do_rootfs"
+ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate"
+do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs"
+do_rootfs[cleandirs] += "${ROOTFS_SSTATE}"
+do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}"
+do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare"
+
+# the buildchroot is owned by root, so we need some sudoing to pack and unpack
+rootfs_sstate_prepare() {
+    sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro
+    sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar --one-file-system rootfs
+    sudo umount ${WORKDIR}/mnt/rootfs
+}
+do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
+
+rootfs_sstate_finalize() {
+    sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar
+}
+
+python do_rootfs_setscene() {
+    sstate_setscene(d)
+    bb.build.exec_func('rootfs_sstate_finalize', d)
+}
+addtask do_rootfs_setscene
-- 
2.30.2


  parent reply	other threads:[~2021-10-26 12:28 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-26 12:28 [PATCH v4 00/10] Add sstate-cache Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 01/10] oe imports in central location Adriaan Schmidt
2021-10-26 12:48   ` Henning Schild
2021-10-26 12:28 ` [PATCH v4 02/10] images: create deploy dir Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 03/10] rootfs: recursively depend on packages Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 04/10] base: remove unneeded "before do_build" task dependencies Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 05/10] dpkg: add explicit dependency to isar-apt Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 06/10] meta: add sstate feature from oe Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 07/10] sstate: configure Adriaan Schmidt
2021-10-26 12:28 ` [PATCH v4 08/10] sstate: add caching to isar-bootstrap Adriaan Schmidt
2021-10-26 12:28 ` Adriaan Schmidt [this message]
2021-10-26 12:28 ` [PATCH v4 10/10] sstate: add caching to debian packages Adriaan Schmidt
2021-10-26 12:37 ` [PATCH v4 00/10] Add sstate-cache Schmidt, Adriaan
2021-10-26 13:43 ` Henning Schild
2021-10-28 15:23 ` Anton Mikanovich
2021-10-28 18:06   ` Henning Schild

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211026122811.2654125-10-adriaan.schmidt@siemens.com \
    --to=adriaan.schmidt@siemens.com \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox