Re: [PATCH v3 0/5] Improving apt cache

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

On Fri, 2023-01-20 at 08:31 +0100, Uladzimir Bely wrote:
> Currently, apt cache (e.g. `var/cache/apt/archives`) import and
> export
> functions are not optimal. Multiple files are copied from global
> DL_DIR to package WORKDIR, increasing disk IO and space needed.
> 
> Also, various chroots (bootstrap, buildchroot, sbuild chroot) include
> their apt caches to sstate cache files.
> 
> This patchset switches to hardlinks instead of copies and removes apt
> cache from bootstrapped images ans sstate caches.

I tested this series (v3) in two layers: amd64 (internal) and arm64
(meta-iot2050) and can confirm that it works. Also we get a significant
performance improvement and massively reduced disk consumption.

Thanks for implementing this!

Acked-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Felix

> 
> Changes since v2:
>  - Don't use CACHEDIR.TAG, simply exclude var/cache/apt directory
> when creating rootfs tarball for sstate-cache.
>  - Use symlinks when exchanging deb files between WORKDIR/rootfs
> and /var/cache/apt/archives inside sbuild chroot
> 
> Changes since v1:
>  - Simplified cleanup of apt cache in debootstrap rootfs.
>  - Now "ln" instead of "cp -l" used.
>  - Removed apt cache contents from sstate cache. The idea is proposed
> in patch 3, but it was reworked and fixed. Firstly, CACHEDIR.TAG
> can't
> be just a file (e.g. created by 'touch'), it should include some
> specific signature [1]. Secondly, it's easier to just create this tag
> in bootstrapped rootfs and it will be automatically used in all
> derivatives (sbuild-chroot/buildchroot/image). So, the original patch
> from Roberto A. Foglietta was simplified.
> 
> This patchset includes (or absorbs) the logic from p1..p3 patches of
> the series Roberto prosposed. What concerns additional patches, they
> don't let us benefit much, but require quite significant changes
> in Isar, so we should check twice if they are worth including.
> 
> Uladzimir Bely (5):
>   Clean apt cache from debootstrapped rootfs dirs
>   Use hardlinks in deb-dl-dir import/export
>   Exclude apt cache from sstate caches
>   Use symlinks when importing debian packages to sbuild chroot
>   Lightweight copy of rootfs directories if possible
> 
>  meta/classes/deb-dl-dir.bbclass                     | 4 ++--
>  meta/classes/dpkg.bbclass                           | 4 ++--
>  meta/classes/imagetypes_container.bbclass           | 2 +-
>  meta/classes/rootfs.bbclass                         | 7 ++++---
>  meta/classes/sdk.bbclass                            | 2 +-
>  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 6 +++++-
>  6 files changed, 15 insertions(+), 10 deletions(-)
> 
> -- 
> 2.20.1
>