From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:adf:a1d6:: with SMTP id v22mr16403588wrv.416.1584779523359; Sat, 21 Mar 2020 01:32:03 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:2250:: with SMTP id a16ls4099433wmm.3.canary-gmail; Sat, 21 Mar 2020 01:32:02 -0700 (PDT) X-Google-Smtp-Source: ADFU+vvNStpM0HKths64r0C8Wg2iC/nLfFP7rAbDWtxWY0uXJnCgV1IJc50xxhq+onvWiXvtmIu3 X-Received: by 2002:a05:600c:22cd:: with SMTP id 13mr14692258wmg.186.1584779522573; Sat, 21 Mar 2020 01:32:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1584779522; cv=none; d=google.com; s=arc-20160816; b=CivnkPaL4TqqjUVJ2IGgNXAJjP0y+KPyU7ahyGuNrKcrJ5TTyjBK3Q+Gf5ok/AEn7E PA+6lpmG4GwEDsLA7e1BHcQU3gK/sEfBzWMKa3S/HLCfOLjQoWNKJEOsg5NG1JupNXvB fdGNr1HQ3TmoymLFVIEbBnVbDv/MVW4qPWehWk77c43DDmHNPfzFhMZHsDwvvZVT5JSU OA19Oy+F8Gj90lvY6ZjgoGBV/vxiPUWoaRpDWd6ykcChDJrhD7ynY2mMH+7ctM9MuSQI aScuNvAkzRygrgCavPFO27tPgfimn0XcvnXDE6jng2+RzYnfhqlFmmBi4IGdsPRbnyrC H1FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=UYTdsNzQ7IBb6b6cmbvrko9+BEC7mBU1QkZz4uNco5U=; b=dws+0eFseDileCgY2c2Q+R01mi7gIQEkcPZPMLfI+BZ2P2x1Jbl+2IwcjI1Dib3ahN YBe9eWKdCcIsYb2T0RCIvm3EFM4y5V5z6i+YM0i6n98XG5HEbcX/rGqYzQiiN+F8i7TC 88eR78o887XBJa9EAiU/1NI8FDvdkAEHbrDZd2fvNVh4rPlwZuYRK7/9A6KHs3K5Z8fX KDGagiZc6O8NjCJi+WH/GSjX85Wr30I+XwibLgnMLN0+szyO0R6zfDSgC0PBS7TNmvGT azQPjigkplTElFwRaefhm7Zf3HQXLz9v3ZYbpVI9ArlEcpwYAtPVF2bwDG1XMvxN7c5S qSuw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id d16si396640wrv.4.2020.03.21.01.32.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2020 01:32:02 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 02L8W1XN002830 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 21 Mar 2020 09:32:01 +0100 Received: from localhost.localdomain ([139.22.32.253]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 02L8VuGO021533; Sat, 21 Mar 2020 09:32:01 +0100 From: Henning Schild To: isar-users@googlegroups.com Cc: Vijai Kumar K , Baurzhan Ismagulov , Jan Kiszka , Henning Schild Subject: [PATCHv7 10/29] meta: import DL_DIR debs before apt-get download steps Date: Sat, 21 Mar 2020 09:31:29 +0100 Message-Id: <20200321083148.26160-11-henning.schild@siemens.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200321083148.26160-1-henning.schild@siemens.com> References: <20200321083148.26160-1-henning.schild@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: o34Sb94HlOzT From: Henning Schild If we ever downloaded a file there is no need to do that again. We can not only use that DL_DIR to later construct a repository but also to speed up our rebuilds. Signed-off-by: Henning Schild --- meta/classes/deb-dl-dir.bbclass | 10 ++++++++++ meta/classes/dpkg-gbp.bbclass | 1 + meta/classes/dpkg.bbclass | 1 + meta/classes/image-tools-extension.bbclass | 1 + meta/classes/rootfs.bbclass | 6 ++++++ 5 files changed, 19 insertions(+) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index 066ce1a3..2858fea2 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -3,6 +3,16 @@ # # SPDX-License-Identifier: MIT +deb_dl_dir_import() { + export pc="${DEBDIR}/${DISTRO}" + export rootfs="${1}" + [ ! -d "${pc}" ] && return 0 + flock -s "${pc}".lock -c ' \ + sudo find "${pc}" -type f -iname '*\.deb' -exec \ + cp -f --no-preserve=owner -t "${rootfs}"/var/cache/apt/archives/ '{}' + + ' +} + deb_dl_dir_export() { export pc="${DEBDIR}/${DISTRO}" export rootfs="${1}" diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass index 7e908eba..afa1e19b 100644 --- a/meta/classes/dpkg-gbp.bbclass +++ b/meta/classes/dpkg-gbp.bbclass @@ -12,6 +12,7 @@ GBP_EXTRA_OPTIONS ?= "--git-pristine-tar" do_install_builddeps_append() { dpkg_do_mounts + deb_dl_dir_import "${BUILDCHROOT_DIR}" sudo -E chroot ${BUILDCHROOT_DIR} \ apt-get install -y -o Debug::pkgProblemResolver=yes \ --no-install-recommends --download-only ${GBP_DEPENDS} diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index 7e2c86d6..585365ee 100644 --- a/meta/classes/dpkg.bbclass +++ b/meta/classes/dpkg.bbclass @@ -7,6 +7,7 @@ inherit dpkg-base do_install_builddeps() { dpkg_do_mounts E="${@ isar_export_proxies(d)}" + deb_dl_dir_import "${BUILDCHROOT_DIR}" sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ ${PP}/${PPS} ${DISTRO_ARCH} --download-only deb_dl_dir_export "${BUILDCHROOT_DIR}" diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass index ec67d940..6590ee7a 100644 --- a/meta/classes/image-tools-extension.bbclass +++ b/meta/classes/image-tools-extension.bbclass @@ -25,6 +25,7 @@ do_install_imager_deps() { buildchroot_do_mounts E="${@ isar_export_proxies(d)}" + deb_dl_dir_import ${BUILDCHROOT_DIR} sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' \ apt-get update \ -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 706b43a7..81759ebc 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -113,6 +113,12 @@ rootfs_install_resolvconf() { fi } +ROOTFS_INSTALL_COMMAND += "rootfs_import_package_cache" +rootfs_import_package_cache[weight] = "5" +rootfs_import_package_cache() { + deb_dl_dir_import ${ROOTFSDIR} +} + ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" rootfs_install_pkgs_download[weight] = "600" rootfs_install_pkgs_download[isar-apt-lock] = "release-after" -- 2.24.1