From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7182796543154454528 X-Received: by 2002:a17:90a:8b8d:b0:226:bf8f:1f8d with SMTP id z13-20020a17090a8b8d00b00226bf8f1f8dmr610713pjn.126.1672987694314; Thu, 05 Jan 2023 22:48:14 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:902:f394:b0:192:62df:a3e7 with SMTP id f20-20020a170902f39400b0019262dfa3e7ls35738479ple.9.-pod-prod-gmail; Thu, 05 Jan 2023 22:48:13 -0800 (PST) X-Google-Smtp-Source: AMrXdXt3xqJ19B7iW5lvr3eWVQiOKf0ux0cciQwaMsRENVygNjvEs9xm0VAjaxIJuhw25q9f9QAq X-Received: by 2002:a17:90a:4503:b0:226:cf55:913f with SMTP id u3-20020a17090a450300b00226cf55913fmr4648128pjg.29.1672987693197; Thu, 05 Jan 2023 22:48:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672987693; cv=none; d=google.com; s=arc-20160816; b=HQl1LOykBHAC6QLDLw3hpyDO/cibCC6gI6bAFrUKOzCexKAoaqSike8+70LbC3yVvt Y7IrkafG4Cw9aUrxV6oJoD5KpGk8oCbG20DWvb1MHTIV52r6RiFRxkLIP/53skfzQmuW 6+wBLUSlSnEqg6IpjkfiBGYjRCjG3WEvfDHopdKNtNkfFkd3RhMvcm53NxHOlVoPdznG LLRddEGCWXIAJ2tKT3GipSVqWBQL2wdjA72Ms6SkprtUyeAOwIstbC65N2xlRpnCAp5E WpX7M98Ckg6zBcEDbQB0sfxrLCuG1cahzARAWVJu1N+0yEd2pK1twNQfZFZp4CjqIJCI NVCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=aBwoF8B1nN/AuTxAC4v/SWE4MYeAO9DTgfA9Th9mJCY=; b=MZwxZ08CM+UO7H8SM34fJ4FfM0f86jB0sr6muVarQti9th8OCL+rVDBBNSN5ZNwO5j PVHtWkuPsGi0xMDkGCYsV1tV122rH0nKYSX5zMavQHzUCZWByhAFbsm5DgWLwdmB6Kfh oELrAea3vDmTpkwk8KRNk1kqZuVGVvh5ZRY7zJin8Z1s+66rozFxqehQvgZqjOEy/dNS GmrUJpxPzjr+OBjBBUA6jukcjnWjykp8BTqLNf16erXN6V5Dhjjnu3ObU0KtCk5XH8oI Go1vlnp5SraEWrBhdhL9kJvDn4KtuO7HznmHpzZomvyMZks3fQEffmD68YURz9KvGtfs ANaQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id em10-20020a17090b014a00b00225f8c9bf80si402534pjb.0.2023.01.05.22.48.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 05 Jan 2023 22:48:13 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 3066m9J6011292 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 6 Jan 2023 07:48:10 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v2 3/3] Changes for a faster build using less disk space Date: Fri, 6 Jan 2023 07:48:09 +0100 Message-Id: <20230106064809.10412-4-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230106064809.10412-1-ubely@ilbers.de> References: <20230106064809.10412-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: DcD8adGD0wV6 From: "Roberto A. Foglietta" Faster build for larger projects with a lot of debian packages This patch leverages few ways to accomplish the task to optimise the large buildings. Put a CACHEDIR.TAG file in each cache folder in such a way every tar that uses the option --exclude-caches could be avoided to include debian packages saving a lot of time and disk space. Signed-off-by: Roberto A. Foglietta Signed-off-by: Uladzimir Bely --- meta/classes/deb-dl-dir.bbclass | 2 +- meta/classes/rootfs.bbclass | 3 ++- meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 7 ++++++- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 7db25251..3afad4e8 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -102,7 +102,7 @@ deb_dl_dir_export() { -maxdepth 1 -type f -iname '*\.deb' |\ while read p; do # skip files from a previous export - [ -f "${pc}/${p##*/}" ] && continue + [ -e "${pc}/${p##*/}" ] && continue # can not reuse bitbake function here, this is basically # "repo_contains_package" package=$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name ${p##*/}) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 786682d9..f274443d 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -308,7 +308,8 @@ rootfs_install_sstate_prepare() { # so we use some mount magic to prevent that mkdir -p ${WORKDIR}/mnt/rootfs sudo mount --bind ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro - sudo tar -C ${WORKDIR}/mnt -cpSf rootfs.tar --one-file-system rootfs + lopts="--one-file-system --exclude-caches" + sudo tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts rootfs sudo umount ${WORKDIR}/mnt/rootfs sudo chown $(id -u):$(id -g) rootfs.tar } diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index cb0079ec..44019f95 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -395,6 +395,10 @@ EOSUDO # Cleanup apt cache sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean + + # Don't include apt cache into sstate cache by marking the directory + # with a CACHEDIR.TAG file with a special signature + sudo -Es sh -c "echo Signature: 8a477f597d28d172789f06886806bc55 > ${ROOTFSDIR}/var/cache/apt/archives/CACHEDIR.TAG" } addtask bootstrap before do_build after do_generate_keyrings @@ -405,7 +409,8 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar --one-file-system $(basename "${ROOTFSDIR}") + lopts="--one-file-system --exclude-caches" + sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") sudo chown $(id -u):$(id -g) bootstrap.tar } -- 2.20.1