From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7182796543154454528 X-Received: by 2002:a05:6e02:1ba2:b0:310:9fdc:9343 with SMTP id n2-20020a056e021ba200b003109fdc9343mr2776512ili.124.1675072612120; Mon, 30 Jan 2023 01:56:52 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:8942:0:b0:6e0:16e5:8907 with SMTP id b2-20020a5d8942000000b006e016e58907ls2444211iot.7.-pod-prod-gmail; Mon, 30 Jan 2023 01:56:51 -0800 (PST) X-Google-Smtp-Source: AK7set+X4cgM/JaiZCNwLYPahfI6+rD4MkIU6dTTAfX6jSxBoXDVO/v/hXrNI+ZMBAnEKNhjAc5n X-Received: by 2002:a6b:6d09:0:b0:71b:d76c:fadd with SMTP id a9-20020a6b6d09000000b0071bd76cfaddmr2434626iod.3.1675072611534; Mon, 30 Jan 2023 01:56:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675072611; cv=none; d=google.com; s=arc-20160816; b=cAQhFUNEWKRz/gzfFQhKIxpk5qvL8qreAQJ27lbgUW6QnhWpvdAfUAdUz5Red4jveJ F6MrROC1MFMiYnzKyTBxdvd8Gw1LH6NAZOz9v4+zMP6Jek22NbWpcVZpKQdhPQ8ANTzf 5ACTvzF4pZtnewWeOMQndJhm63fJyZKmwgv+3O1UzuYSBCXPEIMryT7dmQwvH90Ac/U0 a9WTcaLH7jfYH1nvArwnybGAVBSJmQL9gRXL/IIoLd6uAOQZ3zvQxAzKDe07oRCkxyWU G0TNRtkf5QymYOO9in2bVc8nUX2o5dvz/Ojj32hJyuNYL2E/4cmvnwKRIXhCQtV/2QT0 swLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=Xyxqvni8beohjN1k+Sh2dnJ3/lLDI03VMd0QBZmKoVU=; b=k+CQ1Yg6NNuJUgWAlxQP7prTPNALkyjzUUWzuxvfyrsu+t6AMBmOME1D8KfpQJtXvv sO+JlPPwynf7RKIEyH04E2/O6E8hEIAjVlftKkkZkvF1cyZnxU2b1Cl27cakVi55lcQn HvOTNGUBjn+zOZeWNJgujhqj54kqY1DphuTNo+yo8Bh+avCek4etDqousEM7/9UkNXdq y3G2KT2jfZclNmdg+POCoF8WhDXWvFBvpADYlInUbItBcB13AcoKaPqNryBa76oCssfW S+sWrI/quJNSA2hY9Mp3r92+wt/1Xbf36NsR8cubMw3Z9xfu/GgDOeXQams0guXIzknM xE0Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=KLfvM2n1; spf=pass (google.com: domain of roberto.foglietta@gmail.com designates 2607:f8b0:4864:20::731 as permitted sender) smtp.mailfrom=roberto.foglietta@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com. [2607:f8b0:4864:20::731]) by gmr-mx.google.com with ESMTPS id k6-20020a0566022a4600b006e02ea7519csi583009iov.4.2023.01.30.01.56.51 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 30 Jan 2023 01:56:51 -0800 (PST) Received-SPF: pass (google.com: domain of roberto.foglietta@gmail.com designates 2607:f8b0:4864:20::731 as permitted sender) client-ip=2607:f8b0:4864:20::731; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=KLfvM2n1; spf=pass (google.com: domain of roberto.foglietta@gmail.com designates 2607:f8b0:4864:20::731 as permitted sender) smtp.mailfrom=roberto.foglietta@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-qk1-x731.google.com with SMTP id ou35so3250276qkn.5 for ; Mon, 30 Jan 2023 01:56:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Xyxqvni8beohjN1k+Sh2dnJ3/lLDI03VMd0QBZmKoVU=; b=KLfvM2n1xb2AedPT4M2yxocUtnCvB1RcSlXgrqmqz2n3BwWXJlSCkm/oeYvWtZ2mAd WQl6Y3onsTacLef9os56Pg6k7DHCxh/++0Vo2AhKRmxaagisORqJmBWFqy47SvWGtOhw OzuPJ0ohqYlX9L+vf3OaBs10o6xB872dWmUvvyHzq8OPKcdzmc246ZJD7Cq47jTPwEwV 6iXcLOlIKaF7g12+of3a4A3WE/ACMHt0AD+esOmHPRl47RpntQYgYp7AD3WpGOh8Duiy NkM0zudUN0R139R0lOuiktSoHvDneQuPHVk5798SDJ9PMe2nGtvGQ5zEW1fXNusYHMoX TOYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Xyxqvni8beohjN1k+Sh2dnJ3/lLDI03VMd0QBZmKoVU=; b=nPR05KPgAKSpPHeFUE8Gtli1wT5a59zRlg7eMwq2QGrz6dwBPi+Jz2zmZeewNcJad0 xNL3nUeuCcS8v8SHhCWGD+6x7MQjEkSawQ6kmoLe/OceZ/uSa6HH5noImQyj9JMGJsgo aW1oySPTZpMQQQsywthigOfUFKwRJpTJaDe4QVw8hDHc4LClsyRErPKfY9Mh9PcbOhn6 ejkus3gZNjluiEcMarUR6cICo1YdN1sc5IrZrvL4Vw+1xc8nUc5UizZk26ufe27Z2Rt+ 2+pHDyPltJwvnV6Yck8daMwq2TSZur/88Z/Fo6+MoaFHusgoEC2bd3I89NYkkTMgQIAG llFg== X-Gm-Message-State: AFqh2krJSj2rrQM68mBx54MQ84OBnciIVPO7DyaZT/XSFrNzUqPzZTQe nQcyt29soQUjoSYIG02lsQi0MMzFcEHv1alDQMOXfO6D9Q== X-Received: by 2002:ae9:e314:0:b0:706:6b11:8224 with SMTP id v20-20020ae9e314000000b007066b118224mr1538465qkf.197.1675072610942; Mon, 30 Jan 2023 01:56:50 -0800 (PST) MIME-Version: 1.0 References: <20230120073106.5453-1-ubely@ilbers.de> In-Reply-To: From: "Roberto A. Foglietta" Date: Mon, 30 Jan 2023 10:56:13 +0100 Message-ID: Subject: Re: [PATCH v3 0/5] Improving apt cache To: "Moessbauer, Felix" Cc: "ubely@ilbers.de" , "isar-users@googlegroups.com" , "Kiszka, Jan" , "Schild, Henning" Content-Type: text/plain; charset="UTF-8" X-TUID: tzSBXu62DsPi On Mon, 30 Jan 2023 at 09:45, Moessbauer, Felix wrote: > > On Fri, 2023-01-20 at 08:31 +0100, Uladzimir Bely wrote: > > Currently, apt cache (e.g. `var/cache/apt/archives`) import and > > export > > functions are not optimal. Multiple files are copied from global > > DL_DIR to package WORKDIR, increasing disk IO and space needed. > > > > Also, various chroots (bootstrap, buildchroot, sbuild chroot) include > > their apt caches to sstate cache files. > > > > This patchset switches to hardlinks instead of copies and removes apt > > cache from bootstrapped images ans sstate caches. > > I just saw that this pattern does NOT work in case the cache is on a > different filesystem. This unfortunately is the case for all CI systems > with locally mounted caches, as well as for kas-container builds with > DL_DIR outside the KAS_WORK_DIR. The entire cache system needs to be reworked and this was pretty clear. However, every step (hard link included) brings us nearer to this conclusion and moreover it helps to reduce the building time (now 11m29s on complete image) that allows us to perform more tests in less and lesser time. I am currently on this and I have a good feeling that I will reach a general solution soon. As you can imagine the cash system is not an easy piece to rework due to its implications in many different starting points it can have. So, build and rebuild are just two basic cases but variations of the top or ISAR layers that cause a partial rebuilding and breaks in building are many other cases. Obviously the main idea is to reach a reasonable goal AND then break it in several small steps that can be integrated upstream because a single large patch would be equivalent to fork another 'next' branch which is possible but IHMO not acceptable for a long time. After all, also the GPLv3 on composition is not a long-term way to go. So, in the best case it makes sense that we reach the common goal to rationalise and speed-up the cache subsystem. Best regards, R-