From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6752900179667451904 X-Received: by 2002:adf:9e92:: with SMTP id a18mr4071205wrf.34.1572282095433; Mon, 28 Oct 2019 10:01:35 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:8b1d:: with SMTP id n29ls12443399wra.8.gmail; Mon, 28 Oct 2019 10:01:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqwxM/B9EmcFkkzOyTdWbfm+8WI1QEfAC9NC/QcMSNgNr976vfrK8SVf0yK9Waj9M5ie7eG3 X-Received: by 2002:adf:fdd0:: with SMTP id i16mr16898806wrs.227.1572282094760; Mon, 28 Oct 2019 10:01:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572282094; cv=none; d=google.com; s=arc-20160816; b=esyta90PcJvk/t9nHFlI98D7SkEx1bGMfk6ZYd4XA0MHh5AjGE9VOCaYgTv+DMTT5l vnhHiXMzuSQZVVEO/SPyAb5Ibamn62IqNnm5U+mqjd3qqBZXnxikeEUsv4LQBgqcDa3U XHs3PUPWb0kxod9Jmn6YaP4A1BtCOAGLasXsEfcjYWoenLtfyJwVTxAU0qxYumfXyz+K ek3pYKvGYyRbWhejIQYmJw5ifx8Z/ZU0s51hVrRrckKLYIIWpov/yI076ol3/Fn8KSbk ccQPNs/gECUkCVotJWhLU0lyxQ6N2BcStD87pNinnUXa1D8PGsyS+jIwWdoI8tO7LnJH hipQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=DKpAcErS1/Hwe4UAoKcUVScjB+c95iu17BelsVAJqd8=; b=gs8+MQOCoSJoJ4mE4T0OWEww401dr0mNBVZJIAOZKAPKkwtIfCsJM1t3jKtI7cpgdD +g38DKjYDW6FT5r9QNR/W7WI6FykEcrPPfaBMLry7uBlvjZBeRbJCZP8m69+JbprNl35 ueeDoCTvmI8bfx9gfYJnOtNsChyp+/deh3MXLEgsJPgm8gYAvdOAS21nFFq5GJzcO9iF B3jZPoytQEfvMTR9gnPn86WjsV9JRZ0gsLQ61jm8XJhonZl248pg19au70Ki9G/63FXg ++oUnDyqSI7ubIIPVP+QVzR42gjyoDmtPLLMvG7L89fUNem20w65uJDBtUFhHZ+DbFpO YzQw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id z8si10683wmi.1.2019.10.28.10.01.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Oct 2019 10:01:34 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id x9SH1YWX032237 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 28 Oct 2019 18:01:34 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.244]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x9SH1YP8025390; Mon, 28 Oct 2019 18:01:34 +0100 From: Henning Schild To: isar-users@googlegroups.com Cc: Henning Schild Subject: [PATCH 02/17] dpkg-base: add download caching of apt:// downloads Date: Mon, 28 Oct 2019 18:01:17 +0100 Message-Id: <20191028170132.28072-3-henning.schild@siemens.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191028170132.28072-1-henning.schild@siemens.com> References: <20191028170132.28072-1-henning.schild@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: /gdApdZK16+b From: Henning Schild Store the results of "apt-get source" in the DL_DIR. This means we really only need to fetch once on incemental rebuilds or if multiple recipes use the same SRC_URI. We can also later collect the results and put them in a repository. Signed-off-by: Henning Schild --- meta/classes/dpkg-base.bbclass | 20 ++++++++++++++++++-- meta/conf/bitbake.conf | 1 + 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 3f1e99f..b39df5c 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -37,14 +37,30 @@ do_apt_fetch() { -o Dir::Etc::SourceParts="-" \ -o APT::Get::List-Cleanup="0" - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ - sh -c 'cd ${PP} && apt-get -y --only-source source ${SRC_APT}' + for uri in "${SRC_APT}"; do + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}" + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd ${PP} && apt-get -y --only-source source "$2"' my_script "${DISTRO}" "${uri}" + done + dpkg_undo_mounts } addtask apt_fetch after do_unpack before do_patch do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" +addtask cleanall_apt before do_cleanall +do_cleanall_apt[nostamp] = "1" +do_cleanall_apt() { + if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then + exit + fi + for uri in "${SRC_APT}"; do + rm -rf "${DEBSRCDIR}"/"${DISTRO}"/"$uri" + done +} + def get_package_srcdir(d): s = os.path.abspath(d.getVar("S", True)) workdir = os.path.abspath(d.getVar("WORKDIR", True)) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index bca1114..4c0a809 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -26,6 +26,7 @@ DEPLOY_DIR = "${TMPDIR}/deploy" FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}" FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}" GITDIR = "${DL_DIR}/git" +DEBSRCDIR = "${DL_DIR}/deb-src" P = "${PN}-${PV}" PF = "${PN}-${PV}-${PR}" PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}" -- 2.23.0