RE: [PATCH 1/1] sttate: change rootfs task to cache

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

Schild, Henning (T CED SES-DE), 27. Januar 2022 21:19:

> Subject: [PATCH 1/1] sttate: change rootfs task to cache

Typo in commit msg -----^

> This patch moves sstate caching from the task "do_rootfs" to
> "do_rootfs_install". Which is earlier in the task chain.
> 
> There are two "bugs" that we fix by doing so, while hopefully not
> giving up on too much caching speed.
>  - rootfs_postprocess:
>    - image_postprocess_mark: potentially outdated BUILD_ID and friends
>    - rootfs_generate_manifest: manifest file gone missing in DEPLOY_DIR
> 
> Since rootfs_postprocess is kind of an open field it might also fix
> issues in ROOTFS_POSTPROCESS_COMMAND functions in layers. Any function
> deriving stuff (like a manifest) or changing stuff (like the mark) could
> potentially benefit from "not caching".
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/rootfs.bbclass | 25 ++++++++++++-------------
>  1 file changed, 12 insertions(+), 13 deletions(-)
> 
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index 1fc487896f4d..e368244ce1ca 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -157,6 +157,7 @@ rootfs_install_pkgs_install() {
>  do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
>  do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND}
> ${ROOTFS_INSTALL_COMMAND}"
>  do_rootfs_install[depends] = "isar-bootstrap-${@'target' if
> d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build"
> +do_rootfs_install[depends] += " base-apt:do_cache isar-apt:do_cache_config"
>  do_rootfs_install[recrdeptask] = "do_deploy_deb"
>  python do_rootfs_install() {
>      configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or
> "").split()
> @@ -270,29 +271,27 @@ python do_rootfs() {
>  }
>  addtask rootfs before do_build
> 
> -do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config"

You removed that line.
I don't remember all of the details, but there were some cases where
this explicit dependency was needed when restoring from cache to
an empty tmp/ dir. Might need some testing...

Adriaan

> -
> -SSTATETASKS += "do_rootfs"
> +SSTATETASKS += "do_rootfs_install"
>  ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate"
> -do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs"
> -do_rootfs[cleandirs] += "${ROOTFS_SSTATE}"
> -do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}"
> -do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare"
> +do_rootfs_install[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs"
> +do_rootfs_install[cleandirs] += "${ROOTFS_SSTATE}"
> +do_rootfs_install[sstate-plaindirs] = "${ROOTFS_SSTATE}"
> +do_rootfs_install[sstate-interceptfuncs] = "rootfs_install_sstate_prepare"
> 
>  # the buildchroot is owned by root, so we need some sudoing to pack and
> unpack
> -rootfs_sstate_prepare() {
> +rootfs_install_sstate_prepare() {
>      sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro
>      sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar --one-file-
> system rootfs
>      sudo umount ${WORKDIR}/mnt/rootfs
>  }
> -do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> +do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> 
> -rootfs_sstate_finalize() {
> +rootfs_install_sstate_finalize() {
>      sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar
>  }
> 
> -python do_rootfs_setscene() {
> +python do_rootfs_install_setscene() {
>      sstate_setscene(d)
> -    bb.build.exec_func('rootfs_sstate_finalize', d)
> +    bb.build.exec_func('rootfs_install_sstate_finalize', d)
>  }
> -addtask do_rootfs_setscene
> +addtask do_rootfs_install_setscene
> --
> 2.34.1