From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6771003123014041600 X-Received: by 2002:a2e:b0c9:: with SMTP id g9mr13982702ljl.134.1580226187322; Tue, 28 Jan 2020 07:43:07 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:8488:: with SMTP id g130ls1765871lfd.11.gmail; Tue, 28 Jan 2020 07:43:06 -0800 (PST) X-Google-Smtp-Source: APXvYqxwvkB7wX/+E17Xfe5vv1+VmQGmj/eljGROmZ87gfY6tRKfaA+s8z1f4yzFkeJfzVqX4Mm3 X-Received: by 2002:ac2:5f59:: with SMTP id 25mr2648742lfz.195.1580226186459; Tue, 28 Jan 2020 07:43:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1580226186; cv=none; d=google.com; s=arc-20160816; b=zMKJXvN+zQMHjMFrqBSY7BBu7FbayjXNtNVRfcBZ6/Ctm3EY9etAhbTKh1To7pPano O0f5/MsqY/YwJz2S61Zc8joD2J9YfJhSEVce29HpxfYUc3mFZRKiLL22eP83Mrm8+caZ PiNgaPGGwe1MMX+EK7dbF3+gA6zUCbBjBlE17uPhy3i5LOms+x5vI5moXSRC2VKRNXFN /CDCEJPeUFdb7LRolc89J91ps9btnh+dVXmuNwBwsaYaZiMdt9RWhyMO6eAzY+/W5h7O ia/jGTGOP07mh9c+milrbknBdo2MOyq4xYfxPvosVnBs6wqk0muBf7O0v4a/axALWN8D RESw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=msK6sSqlC3WjFL4KezXRehslu8CxGJQkDHAd4GpAby4=; b=pabD/Ui0/+DTe40E+IwsL0AZZMKcEZo/YgGfw+fDzbPhbDxCISSuVTxl7mnInh03DQ 9EpZqtOpKvnXVdxkOV7UDwqUq2AsjL2894Dh8aUZp9P41LC0b8Rj/hUH0yqbjOblMsYA 6MWfWg5hver0aTFgNLGGjJbNfHu5Jn+dkMtusnBpETfi9uyIyGNAWBpxGYikkVO4qNj5 du6RcZE9jL8MOtjIVbY85/8jTigtsrT+pR1xvSrqe31o+h3Q+1Tvhv/xcxwyQEgQUgGs 09X4sqY5PxIQJ+krpeZb0aLk/6LtSgVa17U53NpCIOuPu+oS2VCgbf91rX163vXuWpqE rzvw== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=uwpjL8Vh; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::343 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com. [2a00:1450:4864:20::343]) by gmr-mx.google.com with ESMTPS id j30si877469lfp.5.2020.01.28.07.43.06 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 28 Jan 2020 07:43:06 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::343 as permitted sender) client-ip=2a00:1450:4864:20::343; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=uwpjL8Vh; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::343 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-wm1-x343.google.com with SMTP id b2so3092390wma.0 for ; Tue, 28 Jan 2020 07:43:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=msK6sSqlC3WjFL4KezXRehslu8CxGJQkDHAd4GpAby4=; b=uwpjL8Vhtc26sA1E1P4/ltvrklZhu9Owy7UD8HxerHP4QFpj4tzQcCgwidCYJpO8X6 bBZyWlzSiegQ2aI70I96i0/sN3XiBI2D56ymZiDtKLzrKseeTTg5fHRbIRciP1Xu5SWZ mwE/5XcwiU7iebfBTV+CFXFZ615gg58yvsWm9fIQCcBskX9IS+pUD0ZeM901jruOYhIz rgLv1Y9txeKh96FelKf6gw9lHIekYj+3j++IVG4+D3uUyI+zsk0KJQVv9rC0k4m5NOfK vkO8SvdJ33WUJ3ReDxl3R7mPHfi25zRlteqMFxKvNRTUdMzk3xYkvIiYYtulVp8WhvFF ZI4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=msK6sSqlC3WjFL4KezXRehslu8CxGJQkDHAd4GpAby4=; b=Pi6OAvWAv3buDO41GVMF97W/OsnngZ+dvaiQ5cb685hEx81hE5HMq6A/jwL4xKUB3H qmbWAxOJ4v41e9+newZ2QFSKtCaO8TfWwIE5GkISIOuCXUUBVH/ELNgqesRW4rr7yiRk L8RDDNHkXAbVjHCO0CAIqI/U8vPjgJKajE4dOzJ/ZT9gL47LudvCMbaT76bLjtJiTMR9 v9WAvZpwdwzxY6wLg+GyaVgrUakQISp0O/rJb64GyhhTJ8VcFPTnT7r6LLIHw64Ak+eA isn5ggOqNZmKq/3Uy+Fk87U/NDQgFRN6pnv0zxERvIjwm5D6KJ86P4ryAVUv0qGWRod8 ttqA== X-Gm-Message-State: APjAAAVxSVRf11UGECe5w70DToNizJnG+EvNp/FpdwTDQzEuGA6ti2Ks XQRzhtxKF0sBUdKVyQTlV67UXxp3Qbg= X-Received: by 2002:a1c:df09:: with SMTP id w9mr5417603wmg.143.1580226185951; Tue, 28 Jan 2020 07:43:05 -0800 (PST) Return-Path: Received: from oxygen.mgc.mentorg.com (nat-sch.mentorg.com. [139.181.36.34]) by smtp.gmail.com with ESMTPSA id u8sm3601200wmm.15.2020.01.28.07.43.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2020 07:43:05 -0800 (PST) From: Vijai Kumar K To: isar-users@googlegroups.com Cc: Henning Schild Subject: [PATCH v3 10/26] meta: import DL_DIR debs before apt-get download steps Date: Tue, 28 Jan 2020 21:11:57 +0530 Message-Id: <20200128154213.22353-11-vijaikumar.kanagarajan@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200128154213.22353-1-vijaikumar.kanagarajan@gmail.com> References: <20200128154312.337e61f6@md1za8fc.ad001.siemens.net> <20200128154213.22353-1-vijaikumar.kanagarajan@gmail.com> X-TUID: E6ABY30PSA7o From: Henning Schild If we ever downloaded a file there is no need to do that again. We can not only use that DL_DIR to later construct a repository but also to speed up our rebuilds. Signed-off-by: Henning Schild --- meta/classes/deb-dl-dir.bbclass | 10 ++++++++++ meta/classes/dpkg-gbp.bbclass | 1 + meta/classes/dpkg.bbclass | 1 + meta/classes/image-tools-extension.bbclass | 1 + meta/classes/rootfs.bbclass | 6 ++++++ 5 files changed, 19 insertions(+) diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass index e41e981..ab4b1f0 100644 --- a/meta/classes/deb-dl-dir.bbclass +++ b/meta/classes/deb-dl-dir.bbclass @@ -3,6 +3,16 @@ # # SPDX-License-Identifier: MIT +deb_dl_dir_import() { + export pc="${DEBDIR}/${DISTRO}" + export rootfs="${1}" + [ ! -d "${pc}" ] && return 0 + flock -s "${pc}".lock -c ' \ + sudo find "${pc}" -type f -iname '*\.deb' -exec \ + cp -f --no-preserve=owner -t "${rootfs}"/var/cache/apt/archives/ '{}' + + ' +} + deb_dl_dir_export() { export pc="${DEBDIR}/${DISTRO}" export rootfs="${1}" diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass index 7e908eb..afa1e19 100644 --- a/meta/classes/dpkg-gbp.bbclass +++ b/meta/classes/dpkg-gbp.bbclass @@ -12,6 +12,7 @@ GBP_EXTRA_OPTIONS ?= "--git-pristine-tar" do_install_builddeps_append() { dpkg_do_mounts + deb_dl_dir_import "${BUILDCHROOT_DIR}" sudo -E chroot ${BUILDCHROOT_DIR} \ apt-get install -y -o Debug::pkgProblemResolver=yes \ --no-install-recommends --download-only ${GBP_DEPENDS} diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index 7e2c86d..585365e 100644 --- a/meta/classes/dpkg.bbclass +++ b/meta/classes/dpkg.bbclass @@ -7,6 +7,7 @@ inherit dpkg-base do_install_builddeps() { dpkg_do_mounts E="${@ isar_export_proxies(d)}" + deb_dl_dir_import "${BUILDCHROOT_DIR}" sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ ${PP}/${PPS} ${DISTRO_ARCH} --download-only deb_dl_dir_export "${BUILDCHROOT_DIR}" diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass index ec67d94..6590ee7 100644 --- a/meta/classes/image-tools-extension.bbclass +++ b/meta/classes/image-tools-extension.bbclass @@ -25,6 +25,7 @@ do_install_imager_deps() { buildchroot_do_mounts E="${@ isar_export_proxies(d)}" + deb_dl_dir_import ${BUILDCHROOT_DIR} sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' \ apt-get update \ -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index a6b6110..eac553c 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -113,6 +113,12 @@ rootfs_install_resolvconf() { fi } +ROOTFS_INSTALL_COMMAND += "rootfs_import_package_cache" +rootfs_import_package_cache[weight] = "5" +rootfs_import_package_cache() { + deb_dl_dir_import ${ROOTFSDIR} +} + ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" rootfs_install_pkgs_download[weight] = "600" rootfs_install_pkgs_download[isar-apt-lock] = "release-after" -- 2.17.1