From: "Roberto A. Foglietta" <roberto.foglietta@gmail.com>
To: "Moessbauer, Felix" <felix.moessbauer@siemens.com>
Cc: "ubely@ilbers.de" <ubely@ilbers.de>,
"isar-users@googlegroups.com" <isar-users@googlegroups.com>,
"Bezdeka, Florian" <florian.bezdeka@siemens.com>,
"ibr@radix50.net" <ibr@radix50.net>,
"Schild, Henning" <henning.schild@siemens.com>
Subject: Re: Better way to handle apt cache needed
Date: Tue, 14 Feb 2023 11:01:15 +0100 [thread overview]
Message-ID: <CAJGKYO4PLekK2mEmAgoWhKax9kr_AcuTgJFdYMeuPDu+yPrYbQ@mail.gmail.com> (raw)
In-Reply-To: <CAJGKYO4tocJxLNkJ9aMJgSMx+=TW2xJSoHh4LWkrg2PZ=Zjz8Q@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 3422 bytes --]
On Fri, 10 Feb 2023 at 17:05, Roberto A. Foglietta <
roberto.foglietta@gmail.com> wrote:
> On Wed, 25 Jan 2023 at 05:48, Roberto A. Foglietta <
> roberto.foglietta@gmail.com> wrote:
>
>> On Thu, 19 Jan 2023 at 19:08, Roberto A. Foglietta
>
>
SSTATE CACHE REWORK + APT CACHE MOUNT BIND
==========================================
Debug code is still in place but the APT cache and lists are offered by
binding.
Binding is the favourite way to offer APT cache and repositories lists but
not
with sbuild because the lock contention is not supported with binding. It is
using soft links but in an improved declination.
devel4 (public) + npriv12 (private)
------------------------------------
fresh cache
------------ complete --------------
32417 Mb (max) | 27740 Mb (max)
23294 Mb (rest) | 18657 Mb (rest)
4283 Mb (deb) | 4283 Mb (deb)
8989 Mb (wic) | 8987 Mb (wic) <-- cache!
4398 Mb (cache) | 4398 Mb (cache)
time: 12m01s | time: 2m30s [*] 4.81x
------------ original ---------------------
time: 21m17s | time: 8m28s
1.77x 3.39x
devel4 (public) + npriv12 (private)
------------------------------------
fresh cache
------------ basic-os --------------
3946 Mb (max) | 3176 Mb (max)
2976 Mb (rest) | 2278 Mb (rest)
4283 Mb (deb) | 4283 Mb (deb)
803 Mb (wic) | 802 Mb (wic) <-- cache!
520 Mb (cache) | 520 Mb (cache)
time: 3m05s | time: 20s 9.25x
------------ original ---------------------
time: 7m58s | time: 2m17s
2.58x 6.85x
These numbers show that Ilbers' ISAR improved a lot during the time
original, fresh 21m17s
complete, next, devel3, fresh 16m49s 1.27x
complete, next.plus, devel3, fresh 15m49s 1.35x
complete, npriv.12, devel4, fresh 12m01s 1.77x
original, cache 8m28s
complete, next, devel3, cache 5m15s 1.61x
complete, next.plus, devel3, cache 3m18s 2.57x
complete, npriv.12, devel4, cache 2m30s 3.39x
For completeness: next.plus is the next with all the patches applied from
the
mailing list, all those patches that are supposed to improve the building
speed.
Considering that the nvidia-modules compilation takes 2m4s and producing
the wic
image takes 2m8s in the complete build but these activities are not part of
the
optimisation, we have these numbers:
original, fresh 17m05s
complete, next, devel3, fresh 12m37s 1.35x
complete, next.plus, devel3, fresh 11m37s 1.48x
complete, npriv.12, devel4, fresh 7m49s 2.19x
original, cache 6m20s
complete, next, devel3, cache 3m07s 2.03x
complete, next.plus, devel3, cache 1m10s 5.43x
complete, npriv.12, devel4, cache 0m22s 17.27x
The last number 22s is coherent with the basic-os cached building time.
[*] after 2m the complete fresh build, SSD needs to relax internally which
is
fully acceptable in a real-case scenario. The original time values are
still
correct because at the time they have been taken there was not a script
that
immediately after the complete build was started the cached building.
[-- Attachment #1.2: Type: text/html, Size: 4845 bytes --]
[-- Attachment #2: performances-report-v5.txt --]
[-- Type: text/plain, Size: 3273 bytes --]
SSTATE CACHE REWORK + APT CACHE MOUNT BIND
==========================================
Debug code is still in place but the APT cache and lists are offered by binding.
Binding is the favourite way to offer APT cache and repositories lists but not
with sbuild because the lock contention is not supported with binding. It is
using soft links but in an improved declination.
devel4 (public) + npriv12 (private)
------------------------------------
fresh cache
------------ complete --------------
32417 Mb (max) | 27740 Mb (max)
23294 Mb (rest) | 18657 Mb (rest)
4283 Mb (deb) | 4283 Mb (deb)
8989 Mb (wic) | 8987 Mb (wic) <-- cache!
4398 Mb (cache) | 4398 Mb (cache)
time: 12m01s | time: 2m30s [*] 4.81x
------------ original ---------------------
time: 21m17s | time: 8m28s
1.77x 3.39x
devel4 (public) + npriv12 (private)
------------------------------------
fresh cache
------------ basic-os --------------
3946 Mb (max) | 3176 Mb (max)
2976 Mb (rest) | 2278 Mb (rest)
4283 Mb (deb) | 4283 Mb (deb)
803 Mb (wic) | 802 Mb (wic) <-- cache!
520 Mb (cache) | 520 Mb (cache)
time: 3m05s | time: 20s 9.25x
------------ original ---------------------
time: 7m58s | time: 2m17s
2.58x 6.85x
These numbers show that Ilbers' ISAR improved a lot during the time:
original, fresh 21m17s
complete, next, devel3, fresh 16m49s 1.27x
complete, next.plus, devel3, fresh 15m49s 1.35x
complete, npriv.12, devel4, fresh 12m01s 1.77x
original, cache 8m28s
complete, next, devel3, cache 5m15s 1.61x
complete, next.plus, devel3, cache 3m18s 2.57x
complete, npriv.12, devel4, cache 2m30s 3.39x
For completeness: next.plus is the next with all the patches applied from the
mailing list, all those patches that are supposed to improve the building speed.
While devel3 and devel4 differ only about how they inherit custom classes.
Considering that the nvidia-modules compilation takes 2m4s and producing the wic
image takes 2m8s in the complete build but these activities are not part of the
optimisation, we have these numbers:
original, fresh 17m05s
complete, next, devel3, fresh 12m37s 1.35x
complete, next.plus, devel3, fresh 11m37s 1.48x
complete, npriv.12, devel4, fresh 7m49s 2.19x
original, cache 6m20s
complete, next, devel3, cache 3m07s 2.03x
complete, next.plus, devel3, cache 1m10s 5.43x
complete, npriv.12, devel4, cache 0m22s 17.27x
The last number 22s is coherent with the basic-os cached building time.
[*] after 2m the complete fresh build, SSD needs to relax internally which is
fully acceptable in a real-case scenario. The original time values are still
correct because at the time they have been taken there was not a script that
immediately after the complete build was started the cached building.
next prev parent reply other threads:[~2023-02-14 10:01 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-28 9:02 Moessbauer, Felix
2022-12-28 9:21 ` Baurzhan Ismagulov
2022-12-28 9:45 ` Moessbauer, Felix
2022-12-28 10:23 ` Uladzimir Bely
2022-12-28 11:04 ` Moessbauer, Felix
2022-12-29 23:15 ` Roberto A. Foglietta
2022-12-30 4:38 ` Uladzimir Bely
2022-12-30 7:08 ` Roberto A. Foglietta
2022-12-30 6:05 ` Moessbauer, Felix
2022-12-30 8:27 ` Roberto A. Foglietta
2022-12-30 10:04 ` Moessbauer, Felix
2022-12-30 13:11 ` Moessbauer, Felix
2022-12-30 13:33 ` Roberto A. Foglietta
2022-12-30 13:47 ` Roberto A. Foglietta
2022-12-31 8:59 ` Roberto A. Foglietta
2022-12-31 21:03 ` Roberto A. Foglietta
2023-01-09 8:12 ` Roberto A. Foglietta
2023-01-09 9:58 ` Roberto A. Foglietta
2023-01-19 18:08 ` Roberto A. Foglietta
2023-01-25 4:48 ` Roberto A. Foglietta
2023-02-10 16:05 ` Roberto A. Foglietta
2023-02-14 10:01 ` Roberto A. Foglietta [this message]
2023-02-14 16:46 ` Roberto A. Foglietta
2022-12-30 12:29 ` Roberto A. Foglietta
2022-12-28 9:22 ` Florian Bezdeka
2023-01-02 16:15 ` Henning Schild
2023-01-05 6:31 ` Uladzimir Bely
2023-01-05 17:10 ` Roberto A. Foglietta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJGKYO4PLekK2mEmAgoWhKax9kr_AcuTgJFdYMeuPDu+yPrYbQ@mail.gmail.com \
--to=roberto.foglietta@gmail.com \
--cc=felix.moessbauer@siemens.com \
--cc=florian.bezdeka@siemens.com \
--cc=henning.schild@siemens.com \
--cc=ibr@radix50.net \
--cc=isar-users@googlegroups.com \
--cc=ubely@ilbers.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox