Re: [PATCH 1/1] sstate: compress rootfs sstate files

[Henning Schild <henning.schild@siemens.com>]

Am Wed, 23 Feb 2022 18:52:54 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:

> In the email from Wednesday, 23 February 2022 18:43:21 +03 user
> Henning Schild wrote:
> > This is wrong. Everything you put into sstate is already compressed
> > there.
> > 
> > So what you probably want is to "move" stuff into sstate and delete
> > setscene after unpack so temporary tarballs maybe do not hang
> > around.
> > 
> > That will save much more space and time, but i am not sure it would
> > be "allowed". That is why i added Adriaan on cc.  
> 
> Yes, it may be wrong because this tarballs might be needed to compare
> hashes by sstate. But why not use them already compressed and save
> space in workdirs?

The real way to save the space is to not keep sstate stuff at all (in
WORKDIR). When putting things into sstate we should remove it from
WORKDIR right after it went into sstate. When getting stuff out we
should remove staging stuff at the end of setscene.

SSTATE_BUILDDIR seems to take care of some of the bits, i did not
manage to write the correct code for all of that. But compressing bits
we do not actually want to keep does not seem right.

So a clear NACK from me.

Henning

> > 
> > Henning
> > 
> > Am Wed, 23 Feb 2022 10:17:49 +0100
> > 
> > schrieb Uladzimir Bely <ubely@ilbers.de>:  
> > > This saves about 1-2 GiB per multiconfig, at the expense of
> > > approximately 20-40 seconds for packing the files.
> > > 
> > > Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> > > ---
> > > 
> > >  meta/classes/rootfs.bbclass                         | 4 ++--
> > >  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++--
> > >  2 files changed, 4 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/meta/classes/rootfs.bbclass
> > > b/meta/classes/rootfs.bbclass index 2bdb3b6d..15e87bec 100644
> > > --- a/meta/classes/rootfs.bbclass
> > > +++ b/meta/classes/rootfs.bbclass
> > > @@ -298,13 +298,13 @@ do_rootfs_install[sstate-interceptfuncs] =
> > > "rootfs_install_sstate_prepare" # the buildchroot is owned by
> > > root, so we need some sudoing to pack and unpack
> > > rootfs_install_sstate_prepare() { sudo mount --bind
> > > ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro
> > > -    sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar
> > > --one-file-system rootfs
> > > +    sudo tar -C ${WORKDIR}/mnt -cpzf
> > > ${ROOTFS_SSTATE}/rootfs.tar.gz --one-file-system rootfs sudo
> > > umount ${WORKDIR}/mnt/rootfs
> > > 
> > >  }
> > >  do_rootfs_install_sstate_prepare[lockfiles] =
> > > 
> > > "${REPO_ISAR_DIR}/isar.lock"
> > > 
> > >  rootfs_install_sstate_finalize() {
> > > 
> > > -    sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar
> > > +    sudo tar -C ${WORKDIR} -xpzf ${ROOTFS_SSTATE}/rootfs.tar.gz
> > > 
> > >  }
> > >  
> > >  python do_rootfs_install_setscene() {
> > > 
> > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
> > > 2f483f5a..99d1d945 100644 ---
> > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
> > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -402,11
> > > +402,11 @@ do_bootstrap[sstate-plaindirs] = "${BOOTSTRAP_SSTATE}"
> > > do_bootstrap[sstate-interceptfuncs] = "bootstrap_sstate_prepare"
> > > 
> > >  bootstrap_sstate_prepare() {
> > > 
> > > -    sudo tar -C $(dirname "${ROOTFSDIR}") -cpf
> > > ${BOOTSTRAP_SSTATE}/bootstrap.tar --one-file-system $(basename
> > > "${ROOTFSDIR}")
> > > +    sudo tar -C $(dirname "${ROOTFSDIR}") -cpzf
> > > ${BOOTSTRAP_SSTATE}/bootstrap.tar.gz --one-file-system $(basename
> > > "${ROOTFSDIR}") }
> > > 
> > >  bootstrap_sstate_finalize() {
> > > 
> > > -    sudo tar -C $(dirname "${ROOTFSDIR}") -xpf
> > > ${BOOTSTRAP_SSTATE}/bootstrap.tar
> > > +    sudo tar -C $(dirname "${ROOTFSDIR}") -xpzf
> > > ${BOOTSTRAP_SSTATE}/bootstrap.tar.gz sudo ln -Tfsr "${ROOTFSDIR}"
> > > "${DEPLOY_ISAR_BOOTSTRAP}" }  
> 
>