RE: [PATCH v2 0/1] sstate: do not keep intermediate sstate files

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

Uladzimir Bely <ubely@ilbers.de>, 24. Februar 2022 09:20:
> In the email from Thursday, 24 February 2022 10:56:48 +03 user Schmidt,
> Adriaan wrote:
> > Uladzimir Bely, 24. Februar 2022 08:36:
> > > Sstate is quite demanding for disk space. And it often causes failures
> > > in CI, related to 'out-of-space'.
> >
> > If those few GB cause problems, then you really need a larger disk :)
> > But I get the point, and I agree that we should try to save space.
> >
> 
> These few GBs result to ~100GiB per CI job (when doing full builds) :)
> 
> > > This patchset minimizes space usage by intermediate state files by
> > > removing them:
> > > - after sstate-cache populated: with separate task in task queue
> >
> > I would like to avoid adding extra tasks. There seems to be a hook
> > to insert functions (SSTATEPOSTCREATEFUNCS), but as sstate is not
> > really documented, I have not yet figured out how to make our use case
> work.
> 
> Yes, it would be better if there are some post-sstate functions we could use
> instead of inserting one more task after one we are handling...

This seems to work for bootstrap:
===
SSTATETASKS += "do_bootstrap"
SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"

bootstrap_sstate_prepare() {
    # this runs in SSTATE_BUILDDIR, which will be deleted automatically
    sudo tar -C $(dirname "${ROOTFSDIR}") -cpf "bootstrap.tar" --one-file-system $(basename "${ROOTFSDIR}")
    sudo chown $(id -u):$(id -g) "bootstrap.tar"
}

bootstrap_sstate_finalize() {
    # this runs in SSTATE_INSTDIR
    # - after building the bootstrap, the tar won't be there, but we also don't need to unpack
    # - after restoring from cache, there will be a tar which we unpack and then delete
    if [ -f "bootstrap.tar" ]; then
        sudo tar -C $(dirname "${ROOTFSDIR}") -xpf "bootstrap.tar"
        sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}"
        rm "bootstrap.tar"
    fi
}

python do_bootstrap_setscene() {
    sstate_setscene(d)
}

addtask do_bootstrap_setscene
do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}"
===

It no longer uses the more-or-less documented interfaces, but
instead hooks deeper into the completely undocumented sstate logic,
so no `sstate-inteceptfuncs` and no explicitly created and cleaned-up
staging directories.

The same approach should work for rootfs, and could also improve dpkg caching.
I will do some more testing and prepare a patch.

> > > - after sstate-cache unpacked: by direct call of this task
> > >
> > > Some results shown below for three different cases:
> > >
> > > - sstate disabled;
> > > - sstate enabled;
> > > - sstate enabled + cleanup patch.
> > >
> > > bullseye-arm64 cross build with full IMAGE_INSTALL list:
> > >
> > > $ sudo du -sh build-*
> > > 5,7G    build-nosstate
> > > 12G     build-sstate
> > > 7,6G    build-sstate-cleanup
> > >
> > > bullseye-arm64 cross build with empty IMAGE_INSTALL list:
> > >
> > > $ sudo du -sh build-*
> > > 2,5G    build-nosstate
> > > 5,3G    build-sstate
> > > 3,4G    build-sstate-cleanup
> > >
> > > bullseye-amd64 native build with empty IMAGE_INSTALL list:
> > >
> > > $ sudo du -sh build-*
> > > 3,2G    build-nosstate
> > > 6,4G    build-sstate
> > > 4,2G    build-sstate-cleanup
> > >
> > >
> > > Changes since v1:
> > > - patchset renamed from "sstate: compress rootfs sstate files";
> > > - sstate intermediate files compression replaced by removing;
> > > - dpkg intermediate files are also handled.
> >
> > It's unnecessary for dpkg, because we use hardlinks to the *.deb file,
> > so no extra disk space is consumed for the sstate preparation.
> 
> Yes, I saw it, just implemented in the similar way. It may be useful if files
> are copied to another (slow) disk after build finishes - I'm afraid hardlinks
> won't work in this case.

In this case we're only linking inside WORKDIR (not to the actual sstate cache).
But with the above approach all those intermediate files/links will be
handled in a better way.

Adriaan

> >
> > Adriaan
> >
> > > Uladzimir Bely (1):
> > >   sstate: do not keep intermediate sstate files
> > >
> > >  meta/classes/dpkg-base.bbclass                      | 7 +++++++
> > >  meta/classes/rootfs.bbclass                         | 7 +++++++
> > >  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 7 +++++++
> > >  3 files changed, 21 insertions(+)
> > >
> > > --
> > > 2.20.1
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "isar-users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to isar-users+unsubscribe@googlegroups.com.
> > > To view this discussion on the web visit
> > > https://groups.google.com/d/msgid/isar-users/20220224073629.23717-1-> >
> ubely%40ilbers.de.
> 
> 
> --
> Uladzimir Bely
> 
>