Re: [PATCH v2 0/1] sstate: do not keep intermediate sstate files

[Henning Schild <henning.schild@siemens.com>]

Am Thu, 24 Feb 2022 09:04:40 +0000
schrieb "Schmidt, Adriaan" <adriaan.schmidt@siemens.com>:

> Uladzimir Bely <ubely@ilbers.de>, 24. Februar 2022 09:20:
> > In the email from Thursday, 24 February 2022 10:56:48 +03 user
> > Schmidt, Adriaan wrote:  
> > > Uladzimir Bely, 24. Februar 2022 08:36:  
> > > > Sstate is quite demanding for disk space. And it often causes
> > > > failures in CI, related to 'out-of-space'.  
> > >
> > > If those few GB cause problems, then you really need a larger
> > > disk :) But I get the point, and I agree that we should try to
> > > save space. 
> > 
> > These few GBs result to ~100GiB per CI job (when doing full builds)
> > :) 
> > > > This patchset minimizes space usage by intermediate state files
> > > > by removing them:
> > > > - after sstate-cache populated: with separate task in task
> > > > queue  
> > >
> > > I would like to avoid adding extra tasks. There seems to be a hook
> > > to insert functions (SSTATEPOSTCREATEFUNCS), but as sstate is not
> > > really documented, I have not yet figured out how to make our use
> > > case  
> > work.
> > 
> > Yes, it would be better if there are some post-sstate functions we
> > could use instead of inserting one more task after one we are
> > handling...  
> 
> This seems to work for bootstrap:
> ===
> SSTATETASKS += "do_bootstrap"
> SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
> SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
> 
> bootstrap_sstate_prepare() {
>     # this runs in SSTATE_BUILDDIR, which will be deleted
> automatically sudo tar -C $(dirname "${ROOTFSDIR}") -cpf
> "bootstrap.tar" --one-file-system $(basename "${ROOTFSDIR}") sudo
> chown $(id -u):$(id -g) "bootstrap.tar" }
> 
> bootstrap_sstate_finalize() {
>     # this runs in SSTATE_INSTDIR
>     # - after building the bootstrap, the tar won't be there, but we
> also don't need to unpack # - after restoring from cache, there will
> be a tar which we unpack and then delete if [ -f "bootstrap.tar" ];
> then sudo tar -C $(dirname "${ROOTFSDIR}") -xpf "bootstrap.tar"
>         sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}"
>         rm "bootstrap.tar"
>     fi
> }
> 
> python do_bootstrap_setscene() {
>     sstate_setscene(d)
> }
> 
> addtask do_bootstrap_setscene
> do_bootstrap_setscene[dirs] = "${DEPLOY_DIR_BOOTSTRAP}"
> ===
> 
> It no longer uses the more-or-less documented interfaces, but
> instead hooks deeper into the completely undocumented sstate logic,
> so no `sstate-inteceptfuncs` and no explicitly created and cleaned-up
> staging directories.

I read code but did not look for docs, "sstate-inteceptfuncs" is used
in OE like once so also nothing too common it seems. It might be worth
double checking what has happened in more recent sstate. If what we end
up with looks like it could survive a bump it might just be fine.

> The same approach should work for rootfs, and could also improve dpkg
> caching. I will do some more testing and prepare a patch.

Cool! Thanks!

Henning


> > > > - after sstate-cache unpacked: by direct call of this task
> > > >
> > > > Some results shown below for three different cases:
> > > >
> > > > - sstate disabled;
> > > > - sstate enabled;
> > > > - sstate enabled + cleanup patch.
> > > >
> > > > bullseye-arm64 cross build with full IMAGE_INSTALL list:
> > > >
> > > > $ sudo du -sh build-*
> > > > 5,7G    build-nosstate
> > > > 12G     build-sstate
> > > > 7,6G    build-sstate-cleanup
> > > >
> > > > bullseye-arm64 cross build with empty IMAGE_INSTALL list:
> > > >
> > > > $ sudo du -sh build-*
> > > > 2,5G    build-nosstate
> > > > 5,3G    build-sstate
> > > > 3,4G    build-sstate-cleanup
> > > >
> > > > bullseye-amd64 native build with empty IMAGE_INSTALL list:
> > > >
> > > > $ sudo du -sh build-*
> > > > 3,2G    build-nosstate
> > > > 6,4G    build-sstate
> > > > 4,2G    build-sstate-cleanup
> > > >
> > > >
> > > > Changes since v1:
> > > > - patchset renamed from "sstate: compress rootfs sstate files";
> > > > - sstate intermediate files compression replaced by removing;
> > > > - dpkg intermediate files are also handled.  
> > >
> > > It's unnecessary for dpkg, because we use hardlinks to the *.deb
> > > file, so no extra disk space is consumed for the sstate
> > > preparation.  
> > 
> > Yes, I saw it, just implemented in the similar way. It may be
> > useful if files are copied to another (slow) disk after build
> > finishes - I'm afraid hardlinks won't work in this case.  
> 
> In this case we're only linking inside WORKDIR (not to the actual
> sstate cache). But with the above approach all those intermediate
> files/links will be handled in a better way.
> 
> Adriaan
> 
> > >
> > > Adriaan
> > >  
> > > > Uladzimir Bely (1):
> > > >   sstate: do not keep intermediate sstate files
> > > >
> > > >  meta/classes/dpkg-base.bbclass                      | 7 +++++++
> > > >  meta/classes/rootfs.bbclass                         | 7 +++++++
> > > >  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 7 +++++++
> > > >  3 files changed, 21 insertions(+)
> > > >
> > > > --
> > > > 2.20.1
> > > >
> > > > --
> > > > You received this message because you are subscribed to the
> > > > Google Groups "isar-users" group.
> > > > To unsubscribe from this group and stop receiving emails from
> > > > it, send an email to isar-users+unsubscribe@googlegroups.com.
> > > > To view this discussion on the web visit
> > > > https://groups.google.com/d/msgid/isar-users/20220224073629.23717-1->
> > > > >  
> > ubely%40ilbers.de.
> > 
> > 
> > --
> > Uladzimir Bely
> > 
> >   
>