[PATCH v2 0/3] Improving apt cache

[Uladzimir Bely <ubely@ilbers.de>]

Currently, apt cache (e.g. `var/cache/apt/archives`) import and export
functions are not optimal. Multiple files are imported from global
DL_DIR to package WORKDIR, increasing disk IO and size needed.

Also, various chroots (bootstrap, buildchroot, sbuild chroot) include
their apt caches to sstate cache files.

This patchset switches to hardlinks instead of copies and removes apt
cache from bootstrapped images ans sstate caches.

Currently (measured on qemuarm64-bullseye cross-compilation with
maximum 8 parallel tasks (e.g. 8 CPU cores)):
- build directory size reduced from 8906 to 6675 MiB
- runtime maximum disk usage reduced from 15965 to 8501 MiB

TODO:
- cleanup other rootfs's (sbuild-chroot, buildchroot).
  Actually, this won't bring much benefit, but why not cleanup final
  rootfs's apt cache when the build finishes?
- deal with additional copying in sbuild routines (patch 3).
  We could use hardlinks instead of copying packages between upper
  layer (where sbuild temporarly keeps them) and workdir rootfs,
  but it is not expected to bring much benefit, since upper layer
  temporary nature. Additinal measurements are required.

Changes since v1:
 - Simplified cleanup of apt cache in debootstrap rootfs.
 - Now "ln" instead of "cp -l" used.
 - Removed apt cache contents from sstate cache. The idea is proposed
in patch 3, but it was reworked and fixed. Firstly, CACHEDIR.TAG can't
be just a file (e.g. created by 'touch'), it should include some
specific signature [1]. Secondly, it's easier to just create this tag
in bootstrapped rootfs and it will be automatically used in all
derivatives (sbuild-chroot/buildchroot/image). So, the original patch
from Roberto A. Foglietta was simplified.

This patchset includes (or absorbs) the logic from p1..p3 patches of
the series Roberto prosposed. What concerns additional patches, they
don't let us benefit much, but require quite significant changes
in Isar, so we should check twice if they are worth including.

[1] https://bford.info/cachedir/
- 

Roberto A. Foglietta (1):
  Changes for a faster build using less disk space

Uladzimir Bely (2):
  Clean apt cache from debootstrapped rootfs dirs
  Use hardlinks in deb-dl-dir functions

 meta/classes/deb-dl-dir.bbclass                     |  6 +++---
 meta/classes/rootfs.bbclass                         |  3 ++-
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 10 +++++++++-
 3 files changed, 14 insertions(+), 5 deletions(-)

-- 
2.20.1