Re: [PATCH v3 0/5] Improving apt cache

["Roberto A. Foglietta" <roberto.foglietta@gmail.com>]

On Mon, 30 Jan 2023 at 10:56, Roberto A. Foglietta
<roberto.foglietta@gmail.com> wrote:
>
> On Mon, 30 Jan 2023 at 09:45, Moessbauer, Felix
> <felix.moessbauer@siemens.com> wrote:
> >
> > On Fri, 2023-01-20 at 08:31 +0100, Uladzimir Bely wrote:
> > > Currently, apt cache (e.g. `var/cache/apt/archives`) import and
> > > export
> > > functions are not optimal. Multiple files are copied from global
> > > DL_DIR to package WORKDIR, increasing disk IO and space needed.
> > >
> > > Also, various chroots (bootstrap, buildchroot, sbuild chroot) include
> > > their apt caches to sstate cache files.
> > >
> > > This patchset switches to hardlinks instead of copies and removes apt
> > > cache from bootstrapped images ans sstate caches.
> >
> > I just saw that this pattern does NOT work in case the cache is on a
> > different filesystem. This unfortunately is the case for all CI systems
> > with locally mounted caches, as well as for kas-container builds with
> > DL_DIR outside the KAS_WORK_DIR.
>
> The entire cache system needs to be reworked and this was pretty
> clear. However, every step (hard link included) brings us nearer to
> this conclusion and moreover it helps to reduce the building time (now
> 11m29s on complete image) that allows us to perform more tests in less
> and lesser time. I am currently on this and I have a good feeling that
> I will reach a general solution soon. As you can imagine the cash
> system is not an easy piece to rework due to its implications in many
> different starting points it can have. So, build and rebuild are just
> two basic cases but variations of the top or ISAR layers that cause a
> partial rebuilding and breaks in building are many other cases.
> Obviously the main idea is to reach a reasonable goal AND then break
> it in several small steps that can be integrated upstream because a
> single large patch would be equivalent to fork another 'next' branch
> which is possible but IHMO not acceptable for a long time. After all,
> also the GPLv3 on composition is not a long-term way to go. So, in the
> best case it makes sense that we reach the common goal to rationalise
> and speed-up the cache subsystem.
>

apt-cache and sstate-cache are two different subsystems: yes they are
but they are not separated much enough to allow a general
straightforward solution for apt-cache to work but just in dpkg.class.

Best regards, R-