From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7182796543154454528 X-Received: by 2002:a17:902:a717:b0:192:f979:9abe with SMTP id w23-20020a170902a71700b00192f9799abemr1444581plq.20.1674199870181; Thu, 19 Jan 2023 23:31:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:8a81:b0:229:24a8:b4c9 with SMTP id x1-20020a17090a8a8100b0022924a8b4c9ls4456139pjn.1.-pod-control-gmail; Thu, 19 Jan 2023 23:31:09 -0800 (PST) X-Google-Smtp-Source: AMrXdXuPdqfty/KWkjjWmG6amSU42lfBbob+Ymy4IqOvVfLHbOxUWWBtcVhxxc7ePZk/BBXzQFFg X-Received: by 2002:a17:90a:4587:b0:229:bc1:7cd1 with SMTP id v7-20020a17090a458700b002290bc17cd1mr14865720pjg.33.1674199869233; Thu, 19 Jan 2023 23:31:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674199869; cv=none; d=google.com; s=arc-20160816; b=iafBq4dEu5iqF94QpHkMwIRyZTlIGRuG/QE2iODjD0vTL5PYS/gOzWIp01sUxG6dde fL99KTseyukp49gunzdZDthgTcZneCrRFLU7t6Hs+U0sNvvthInhzc8n0PX+9vuryGe7 9+tghIP4px659sdJzXM+K4n+WwJ7oyq60YUsBTLpowcwKInmrd2ZEt3LBTkZe4+C6x9p 6zrKo+dZ3DBFrKMZDBJrTJbp7caBa7oSPCl3GneBFoFTwGz4JdOAEiOBuHgU+HdFYUEY piRjbDEa/iD58/S6RfZripmcrkiXjhSk9shAe91mfZC4eaFrI3PXX90JNDIdWgbmlgmO Tcsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=usi71A42LXHIPxPGGUabWoGDhcX3ohopz1bV9RME3zU=; b=saBABm+eV2mz5qN5BNbkN885UwwFknRdir1Yvxwm3y0BOOhEsXUIqx2EhkXSOcO/fS oLiJ6kBmE7/d5mlNwv/rCmA119NhPhlwmm4U8oM4bgkwL9Ls2mLhrnYb4dw3MYd7GGqc +u5OI0xG87Dt0tZp6MCgq0frdcnkyDydwx6uQLD+SdzmGJ5Z46ewXRhDDTGVcpDdp2W9 IEYt2DjkmUSev3/WvHABUfQqPpAJw0mAiOttHZgZJEXergXsMnimIlcnL1Qc608jlK0E hmh83oksnb+OQ/EO5lyh1B3oWIRhsPsLIP0QCB3rCN2jU3fjmwmzj9joHnITWVuYTQ8T hVEQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id hi11-20020a17090b30cb00b00213290fa218si154432pjb.2.2023.01.19.23.31.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jan 2023 23:31:09 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 30K7V6DP001626 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 20 Jan 2023 08:31:07 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v3 0/5] Improving apt cache Date: Fri, 20 Jan 2023 08:31:01 +0100 Message-Id: <20230120073106.5453-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: s5oI6pYrRDf6 Currently, apt cache (e.g. `var/cache/apt/archives`) import and export functions are not optimal. Multiple files are copied from global DL_DIR to package WORKDIR, increasing disk IO and space needed. Also, various chroots (bootstrap, buildchroot, sbuild chroot) include their apt caches to sstate cache files. This patchset switches to hardlinks instead of copies and removes apt cache from bootstrapped images ans sstate caches. Changes since v2: - Don't use CACHEDIR.TAG, simply exclude var/cache/apt directory when creating rootfs tarball for sstate-cache. - Use symlinks when exchanging deb files between WORKDIR/rootfs and /var/cache/apt/archives inside sbuild chroot Changes since v1: - Simplified cleanup of apt cache in debootstrap rootfs. - Now "ln" instead of "cp -l" used. - Removed apt cache contents from sstate cache. The idea is proposed in patch 3, but it was reworked and fixed. Firstly, CACHEDIR.TAG can't be just a file (e.g. created by 'touch'), it should include some specific signature [1]. Secondly, it's easier to just create this tag in bootstrapped rootfs and it will be automatically used in all derivatives (sbuild-chroot/buildchroot/image). So, the original patch from Roberto A. Foglietta was simplified. This patchset includes (or absorbs) the logic from p1..p3 patches of the series Roberto prosposed. What concerns additional patches, they don't let us benefit much, but require quite significant changes in Isar, so we should check twice if they are worth including. Uladzimir Bely (5): Clean apt cache from debootstrapped rootfs dirs Use hardlinks in deb-dl-dir import/export Exclude apt cache from sstate caches Use symlinks when importing debian packages to sbuild chroot Lightweight copy of rootfs directories if possible meta/classes/deb-dl-dir.bbclass | 4 ++-- meta/classes/dpkg.bbclass | 4 ++-- meta/classes/imagetypes_container.bbclass | 2 +- meta/classes/rootfs.bbclass | 7 ++++--- meta/classes/sdk.bbclass | 2 +- meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 6 +++++- 6 files changed, 15 insertions(+), 10 deletions(-) -- 2.20.1