Re: [PATCH 1/1] sttate: change rootfs task to cache

[Henning Schild <henning.schild@siemens.com>]

Am Fri, 28 Jan 2022 07:57:31 +0100
schrieb "Schmidt, Adriaan (T CED SES-DE)" <adriaan.schmidt@siemens.com>:

> Schild, Henning (T CED SES-DE), 27. Januar 2022 21:19:
> 
> > Subject: [PATCH 1/1] sttate: change rootfs task to cache  
> 
> Typo in commit msg -----^

Thanks!

> > This patch moves sstate caching from the task "do_rootfs" to
> > "do_rootfs_install". Which is earlier in the task chain.
> > 
> > There are two "bugs" that we fix by doing so, while hopefully not
> > giving up on too much caching speed.
> >  - rootfs_postprocess:
> >    - image_postprocess_mark: potentially outdated BUILD_ID and
> > friends
> >    - rootfs_generate_manifest: manifest file gone missing in
> > DEPLOY_DIR
> > 
> > Since rootfs_postprocess is kind of an open field it might also fix
> > issues in ROOTFS_POSTPROCESS_COMMAND functions in layers. Any
> > function deriving stuff (like a manifest) or changing stuff (like
> > the mark) could potentially benefit from "not caching".
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/rootfs.bbclass | 25 ++++++++++++-------------
> >  1 file changed, 12 insertions(+), 13 deletions(-)
> > 
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass index 1fc487896f4d..e368244ce1ca
> > 100644 --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -157,6 +157,7 @@ rootfs_install_pkgs_install() {
> >  do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
> >  do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND}
> > ${ROOTFS_INSTALL_COMMAND}"
> >  do_rootfs_install[depends] = "isar-bootstrap-${@'target' if
> > d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else
> > 'host'}:do_build" +do_rootfs_install[depends] += "
> > base-apt:do_cache isar-apt:do_cache_config"
> > do_rootfs_install[recrdeptask] = "do_deploy_deb" python
> > do_rootfs_install() { configure_cmds =
> > (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or "").split()
> > @@ -270,29 +271,27 @@ python do_rootfs() {
> >  }
> >  addtask rootfs before do_build
> > 
> > -do_rootfs[depends] = "base-apt:do_cache isar-apt:do_cache_config"  
> 
> You removed that line.
> I don't remember all of the details, but there were some cases where
> this explicit dependency was needed when restoring from cache to
> an empty tmp/ dir. Might need some testing...

It got moved up under a preexisting do_rootfs_install[depends]. In fact
we might want to move all do_rootfs_install[] assignments there for
readability. But hey, i think i will move it back down so we have the
actual function []s and the sstate specific []s.

Henning

> Adriaan
> 
> > -
> > -SSTATETASKS += "do_rootfs"
> > +SSTATETASKS += "do_rootfs_install"
> >  ROOTFS_SSTATE = "${WORKDIR}/rootfs-sstate"
> > -do_rootfs[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs"
> > -do_rootfs[cleandirs] += "${ROOTFS_SSTATE}"
> > -do_rootfs[sstate-plaindirs] = "${ROOTFS_SSTATE}"
> > -do_rootfs[sstate-interceptfuncs] = "rootfs_sstate_prepare"
> > +do_rootfs_install[dirs] += "${ROOTFS_SSTATE} ${WORKDIR}/mnt/rootfs"
> > +do_rootfs_install[cleandirs] += "${ROOTFS_SSTATE}"
> > +do_rootfs_install[sstate-plaindirs] = "${ROOTFS_SSTATE}"
> > +do_rootfs_install[sstate-interceptfuncs] =
> > "rootfs_install_sstate_prepare"
> > 
> >  # the buildchroot is owned by root, so we need some sudoing to
> > pack and unpack
> > -rootfs_sstate_prepare() {
> > +rootfs_install_sstate_prepare() {
> >      sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro
> >      sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar
> > --one-file- system rootfs
> >      sudo umount ${WORKDIR}/mnt/rootfs
> >  }
> > -do_rootfs_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> > +do_rootfs_install_sstate_prepare[lockfiles] =
> > "${REPO_ISAR_DIR}/isar.lock"
> > 
> > -rootfs_sstate_finalize() {
> > +rootfs_install_sstate_finalize() {
> >      sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar
> >  }
> > 
> > -python do_rootfs_setscene() {
> > +python do_rootfs_install_setscene() {
> >      sstate_setscene(d)
> > -    bb.build.exec_func('rootfs_sstate_finalize', d)
> > +    bb.build.exec_func('rootfs_install_sstate_finalize', d)
> >  }
> > -addtask do_rootfs_setscene
> > +addtask do_rootfs_install_setscene
> > --
> > 2.34.1  
>