From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7182122219497062400 X-Received: by 2002:a05:6902:91a:b0:738:9bde:1a8d with SMTP id bu26-20020a056902091a00b007389bde1a8dmr7083348ybb.93.1673258345809; Mon, 09 Jan 2023 01:59:05 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a25:ae46:0:b0:7b4:ce9:d911 with SMTP id g6-20020a25ae46000000b007b40ce9d911ls3068712ybe.5.-pod-prod-gmail; Mon, 09 Jan 2023 01:59:05 -0800 (PST) X-Google-Smtp-Source: AMrXdXs+ksMFTGLJVWkUTs4m398mIQOfujYIOypqPmWrEcBSno4gooYh7xrhtYK+Kj4eah1eS2EK X-Received: by 2002:a25:ceca:0:b0:76c:4dd0:38c2 with SMTP id x193-20020a25ceca000000b0076c4dd038c2mr50937007ybe.51.1673258345057; Mon, 09 Jan 2023 01:59:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673258345; cv=none; d=google.com; s=arc-20160816; b=XDUS9wlJ18Th5w+WSVWl2xkHtZ8QDMfCerDzD96HQrxIj4wPNAf9g+vjwf6YF02DKI v58YaUNGg6+FpDhX/BPyqYX5feUph2Sih6XX6WtPxffs3i9cY/6p6VavPQSod5PXMaQB 5xORFQEEpw489oNTotqk1qaESfabLEPdOYAoL4I4X2cjCVO/HXuSoVCegllCAlkntQ3I oVNhKo3i0pmZKisHixLPzkzIcXPKlSnccU0yOZ0tZ/JNlD+teG6ZBwcljBKdVm0jR2qX fLhHSPOMA5iPNaulK/5Sv8ML+8ZcNjqrYRMPxDR0D9WE3gFFNyZuVDfzw7wT7znfMZwZ 4OAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=aUADZKiAWVDP1NfdIsknDjtGlRKhuM1tgo/btmh8pRM=; b=qGbmyMMGIVwa74E5swH3n0giO17JYOYuY+EDjgRHsORaWVoKfIUmoHnmzw0KYxWiK6 JK91Nhz1UTaTZPhVDJ16t+EeupOICXYNivzpD33Dmyo7bDbDk4pdZN/dQ2e/RWG1TYsH EjG0Vweb6z/O2eR9+5bWPH+I8WNoGNXRr/zkzF9bi5DB7LB7AE/dRTgIr2IZqKhGK609 WxeOWkDVR2LjWIsY7iM6wkqM5/Q//JT1mA6FqH8jjwfYM/lzu5rXPersQCeP6Jl/PJ7a OggjcntS0ooOhGUEV+gAQS0wp6ltqIRl+6YNyft3P+/Qx4w1amqVjAsKA7viGaVrljla bwFw== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=dxVu8dtW; spf=pass (google.com: domain of roberto.foglietta@gmail.com designates 2607:f8b0:4864:20::830 as permitted sender) smtp.mailfrom=roberto.foglietta@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com. [2607:f8b0:4864:20::830]) by gmr-mx.google.com with ESMTPS id c2-20020a25b442000000b007b62d9cf791si877532ybg.2.2023.01.09.01.59.05 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Jan 2023 01:59:05 -0800 (PST) Received-SPF: pass (google.com: domain of roberto.foglietta@gmail.com designates 2607:f8b0:4864:20::830 as permitted sender) client-ip=2607:f8b0:4864:20::830; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=dxVu8dtW; spf=pass (google.com: domain of roberto.foglietta@gmail.com designates 2607:f8b0:4864:20::830 as permitted sender) smtp.mailfrom=roberto.foglietta@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-qt1-x830.google.com with SMTP id h26so7440423qtu.2 for ; Mon, 09 Jan 2023 01:59:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=aUADZKiAWVDP1NfdIsknDjtGlRKhuM1tgo/btmh8pRM=; b=dxVu8dtWc2VuEavUZ5ExArUsxIl88IAo+EizQoStq29Lrwt13h/FKrt38mucIa5IhN /MY4pRJiFYoO2aQiFt/rV66OuT4Ox7W02PC+fh/yxvMOkx/gIIbmdrQMcTpTTwyXML8r Mhm0qqepG3AXiqeKFDWGgN8pgw0aAvrEDpqQ+b7GjZXWBGGla1aF4zUb5HFsTpzj0gYI +lZQ4xClz+DeoqtR/wtuFmZ2qVcvPErU49zdGeSU/Jcq+RJTYZVn6UiVejaUzGgpYLI+ 1eS8pIzyoCMETr8O2JlmgVgKqHtC9t+0QIixlcUnG5+g4dBNlwOsUv2HJ/fSkU5Bl/7X Pp+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=aUADZKiAWVDP1NfdIsknDjtGlRKhuM1tgo/btmh8pRM=; b=kWCbCJNDKkl26APoBpBK8BkoO/gQSp3WfUhju3OL8xuATVNsFSQ6wviP77VfCX/qvU r0XeNJvcr82G+cQgbNS2MrfFtx4DP7umVXkKsiRsAnG6lH5oRsf5T8NPCuZ6BnK0c2cv p16H1dwxVH/ftXOBLnBmvqZyw4qudJlglqL0m44g7i6KIH/o4SGW6pZgWL9ZRyFfATPS AbzpQbXY/edOfanqRt31p/nkyh0K05JzepwR13i3BFY854RlevSE3uQhwT2plTFSBeCK Zpcux4VS2BXA6oROsZI5Sk9cJNAufjBWXkJZNSMS8IEuoqs/z7Jrce+L3TU/sZQAFlmD KssA== X-Gm-Message-State: AFqh2kr8X+JVdqTSiEWnRuN1oaj6Y6Wkb+d1d+362V/rIWOgwpbCVWcT By5svM7P3F9i4mHhLk1nBxcTc24JGxpEH35vgiFsLh+99U37 X-Received: by 2002:ac8:4d83:0:b0:3a9:7a71:c8f3 with SMTP id a3-20020ac84d83000000b003a97a71c8f3mr2874662qtw.19.1673258344619; Mon, 09 Jan 2023 01:59:04 -0800 (PST) MIME-Version: 1.0 References: <371e4d826cca6aaba11a4222fef547b134ed6ce7.camel@siemens.com> <38d18c245baa4f685642eafa9a52ab9b9ae9001c.camel@siemens.com> <4769513.OV4Wx5bFTl@hp> <95542c23ea3bd00e31898fc1080854c58e41a8b2.camel@siemens.com> <13d9675831e4d315390357c3bfd925e66f84d136.camel@siemens.com> In-Reply-To: From: "Roberto A. Foglietta" Date: Mon, 9 Jan 2023 10:58:27 +0100 Message-ID: Subject: Re: Better way to handle apt cache needed To: "Moessbauer, Felix" Cc: "ubely@ilbers.de" , "isar-users@googlegroups.com" , "Bezdeka, Florian" , "ibr@radix50.net" , "Schild, Henning" Content-Type: text/plain; charset="UTF-8" X-TUID: TFmrFpItOL5v On Mon, 9 Jan 2023 at 09:12, Roberto A. Foglietta wrote: > > > > On Sat, 31 Dec 2022 at 09:59, Roberto A. Foglietta wrote: > > > > Hi all, > > > > sorry for the image but some numbers should be displayed in a proper > > form. I continued to work on the building optimization of ISAR and the > > results are quite interesting. Almost 2x with some great achievements > > in some specific areas like cache reduction (22x) or in some > > particular cases like using a little of the repository downloaded > > which brings to a disk usage reduced by 11x times. About building > > time, the ratios are a little under-estimated because network > > downloads for debootstrap and apt update take a fixed time that cannot > > be reduced. My first tests indicate that also custom debian packages > > and repackaged .deb are correctly deployed into the system despite the > > changes. More tests are welcome. > > > > Dear all, > > my personal fork was broken in some ways for some time but today, I completed the first round of rebasing and finally it is up again. > These are the numbers, formatted in HTML for a nice reading and an image attached as well. > > From these numbers the first thing we notice is that the size of the deb cache > does not matter at all because 10x more is not slower at all. This makes per- > fectly sense because linking 385 or 1260 does not make such a difference. > > full debs minimal rebnext > ------------ basic-os -------------- ============== > 3498 Mb (max) | 3489 Mb (max) 1.00x 1.00x > 2588 Mb (rest) | 2587 Mb (rest) 1.00x 1.00x > 3417 Mb (deb) | 364 Mb (deb) 9.39x > 814 Mb (wic) | 814 Mb (wic) - > 273 Mb (cache) | 273 Mb (cache) 1.00x > time: 3m09s | time: 3m10s 2.58x > > Compared to the original the performances increase is impressive, in both cases: > > original cherries + schroot + rebase > ------------ basic-os -------------- ============== > 43954 Mb (max) | 3498 Mb (max) 12.57x 16.07x > 26548 Mb (rest) | 2588 Mb (rest) 10.26x 14.50x > 3741 Mb (deb) | 3417 Mb (deb) - > 820 Mb (wic) | 814 Mb (wic) - > 11789 Mb (cache) | 273 Mb (cache) 43.18x > time: 8m33s | time: 3m09s 2.71x > > original cherries + schroot + rebase > ------------ complete ------------- =============== > 52507 Mb (max) | 28606 Mb (max) 1.84x 2.23x > 43311 Mb (rest) | 19413 Mb (rest) 2.23x 3.33x > 3741 Mb (deb) | 3417 Mb (deb) - > 9159 Mb (wic) | 9155 Mb (wic) - > 11799 Mb (cache) | 283 Mb (cache) 41.69x > time: 20m13s | time: 11m44s 1.72x > > Considering that the local .deb cache growing with the time because new updates > will be added, and comparing the performance gains between the basic-os and the > complete image, we can say that on the long run also the complete case will > reach the 2x of performance at least in time building. Results has been publish and the 1st round rebase branch has set as default https://github.com/robang74/isar/blob/rebnext/doc/build-optim.md Best regards, R-