Re: Better way to handle apt cache needed

["Roberto A. Foglietta" <roberto.foglietta@gmail.com>]

On Mon, 9 Jan 2023 at 09:12, Roberto A. Foglietta
<roberto.foglietta@gmail.com> wrote:
>
>
>
> On Sat, 31 Dec 2022 at 09:59, Roberto A. Foglietta <roberto.foglietta@gmail.com> wrote:
> >
> > Hi all,
> >
> >  sorry for the image but some numbers should be displayed in a proper
> > form. I continued to work on the building optimization of ISAR and the
> > results are quite interesting. Almost 2x with some great achievements
> > in some specific areas like cache reduction (22x) or in some
> > particular cases like using a little of the repository downloaded
> > which brings to a disk usage reduced by 11x times. About building
> > time, the ratios are a little under-estimated because network
> > downloads for debootstrap and apt update take a fixed time that cannot
> > be reduced. My first tests indicate that also custom debian packages
> > and repackaged .deb are correctly deployed into the system despite the
> > changes. More tests are welcome.
> >
>
> Dear all,
>
> my personal fork was broken in some ways for some time but today, I completed the first round of rebasing and finally it is up again.
> These are the numbers, formatted in HTML for a nice reading and an image attached as well.
>
> From these numbers the first thing we notice is that the size of the deb cache
> does not matter at all because 10x more is not slower at all. This makes per-
> fectly sense because linking 385 or 1260 does not make such a difference.
>
>           full debs           minimal           rebnext
>        ------------ basic-os -------------- ==============
>         3498 Mb (max)   |  3489 Mb (max)     1.00x   1.00x
>         2588 Mb (rest)  |  2587 Mb (rest)    1.00x   1.00x
>         3417 Mb (deb)   |   364 Mb (deb)     9.39x
>          814 Mb (wic)   |   814 Mb (wic)      -
>          273 Mb (cache) |   273 Mb (cache)   1.00x
>        time: 3m09s      | time: 3m10s        2.58x
>
> Compared to the original the performances increase is impressive, in both cases:
>
>           original           cherries + schroot + rebase
>        ------------ basic-os -------------- ==============
>        43954 Mb (max)   |  3498 Mb (max)    12.57x  16.07x
>        26548 Mb (rest)  |  2588 Mb (rest)   10.26x  14.50x
>         3741 Mb (deb)   |  3417 Mb (deb)      -
>          820 Mb (wic)   |   814 Mb (wic)      -
>        11789 Mb (cache) |   273 Mb (cache)  43.18x
>        time: 8m33s      | time: 3m09s        2.71x
>
>           original           cherries + schroot + rebase
>        ------------ complete ------------- ===============
>        52507 Mb (max)   | 28606 Mb (max)     1.84x   2.23x
>        43311 Mb (rest)  | 19413 Mb (rest)    2.23x   3.33x
>         3741 Mb (deb)   |  3417 Mb (deb)      -
>         9159 Mb (wic)   |  9155 Mb (wic)      -
>        11799 Mb (cache) |   283 Mb (cache)  41.69x
>        time: 20m13s     | time: 11m44s       1.72x
>
> Considering that the local .deb cache growing with the time because new updates
> will be added, and comparing the performance gains between the basic-os and the
> complete image, we can say that on the long run also the complete case will
> reach the 2x of performance at least in time building.

Results has been publish and the 1st round rebase branch has set as default

https://github.com/robang74/isar/blob/rebnext/doc/build-optim.md

Best regards, R-