From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6771003123014041600 X-Received: by 2002:adf:f1c6:: with SMTP id z6mr27911067wro.279.1580226170357; Tue, 28 Jan 2020 07:42:50 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ebc5:: with SMTP id v5ls9511427wrn.8.gmail; Tue, 28 Jan 2020 07:42:49 -0800 (PST) X-Google-Smtp-Source: APXvYqznvCarwbxIJqMkfGABHaHwsFi9FVDqlzzy7p1UONBlOx5Q4mUTV6wbWBwGAUf4dokSd3Tg X-Received: by 2002:adf:90e1:: with SMTP id i88mr30364557wri.95.1580226169729; Tue, 28 Jan 2020 07:42:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1580226169; cv=none; d=google.com; s=arc-20160816; b=De6JdrTNIX9GG5sV2f/yypFfNuRy0vzBj2sOjaLVugvC7UU+1cgNwPDhgn13EkMjrk ju8nqX7YJGz+q5A/Zp1sfOkYCrO9PmlUAzS9naV80U+8CFzwVZPj2BqPhiMDnEtB8veG dRbqQ/ya1AYX7tlJ5ZdPuT/echE62zu4PUvvmnVWhWH9KmuWyyQLRcGy55I7Q68nsdQI FBHyjIGhtuv0gLdazyhtdxYBzcKJBEsGLDHxi62tOGQdSvuB1Zp1NF/ok9x5ovCuM0hu zcoSM4DqPfqOo6HG2y6iH/xOljvfehtf/AF/eo2rlE19TYhBgY3Le+hffMIUnIjt8inW xshQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D8MwIeDjoBrmqFsXmNacD1x4lgXlE2YGKcZMb8nOvjQ=; b=E4nMGKfgJkAh3X26khrjrx7AxMKA3umrsVwT8gXRltu5UwQqWdbDkBjeq4lwvubwyo 19/rO0erukzcSGhZ8jeZ4VoaHOPSnppguePQCcm97JULy/WL8I/HsGUO071SRweXQ6D8 gNHgwAljqsgjdnSC0EMgfRapQXmUQALWHIckkZ2Rx0Ip+GHxkckNyV5XAOIjYQemyHul sgoVaL84cYaREexeTsLrg2WYnlijHwCqMjZb0qy+AuPKSXBu0Qq2S4NzcuEb+A6LyLjY KuyoJNysIfw5XCi+VCTd8P3B+svrmCPth6nLcPgu3g5gSdsbhueECnxS5Tj2Nf9bTdii 4A8Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=qNIRzjxe; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::333 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com. [2a00:1450:4864:20::333]) by gmr-mx.google.com with ESMTPS id 202si104198wme.0.2020.01.28.07.42.49 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 28 Jan 2020 07:42:49 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::333 as permitted sender) client-ip=2a00:1450:4864:20::333; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=qNIRzjxe; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::333 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-wm1-x333.google.com with SMTP id p9so3037510wmc.2 for ; Tue, 28 Jan 2020 07:42:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=D8MwIeDjoBrmqFsXmNacD1x4lgXlE2YGKcZMb8nOvjQ=; b=qNIRzjxeh6I8A/QRfoWNp6Si6/4UV8XcDP/qetrNXG9V+dPcAq5D7V14D918Tfd33B f51HPp8B3ceJ8oXdOQ7aZE8gf+4eJmolI8kxW7KvtBG5RQyPoAEBdqU4jimEoJm4cZbn ePuPfiHY4yrJb/wKjkd+D0sNfqy1O4f/7bULy7j1Gai/eRBbn9YSIeUcgHoNgjcUO3Sl xlt0Rl9lWeB4FwUDmXTRuXFxrCtBhWLQk8rjSfdt984svwlJOQc9l5D/Ltrm3b1zbsPU 2/IL82GuXLtXgI72QqJ7Ji/D2CXDkpkoPgMM0MqTbTA4kBKHiKQWF4poqjuq1OgTAweq 8EIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=D8MwIeDjoBrmqFsXmNacD1x4lgXlE2YGKcZMb8nOvjQ=; b=jX0P3fVjrdg46dr7yGCGCqZsav5tLJm6FTtBy/q5latOcIE+m5DueX1Lu+rSQSXWEJ 14cTxUXuDl0/czIBUHMgyqGMLuymyYl86Gow08Mhd0d0w02Pf/TQKSp4YqCBo6blLMSV kAsZUnqLQ7xUkW0sCSz1ZlJK0cmXOHmG74T5vK8PFLl2qBjZeR1k2+cFWS7zEanlBZtp lT+RgN/ykVFiHzPmLNYCtSI+xCrZTh7PdsFozhOlUHDWFLo8qGbU2+VpXpQjfigUzCgS QgDoNE/8gu5oR4EilJZLVWMVikFaOY1t1aDV773Dbny1K8P749Hv7HdQf7xV8c1j4OO6 NXLg== X-Gm-Message-State: APjAAAUBapsB/BlfAl3fM5CT/aWD8XxPHHrYiIqsJJ9Gw+d36MfqMlBc IxzJ+mUl5TwaNXnsSck6jLnPmTdls1k= X-Received: by 2002:a1c:7205:: with SMTP id n5mr6002833wmc.9.1580226169259; Tue, 28 Jan 2020 07:42:49 -0800 (PST) Return-Path: Received: from oxygen.mgc.mentorg.com (nat-sch.mentorg.com. [139.181.36.34]) by smtp.gmail.com with ESMTPSA id u8sm3601200wmm.15.2020.01.28.07.42.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2020 07:42:48 -0800 (PST) From: Vijai Kumar K To: isar-users@googlegroups.com Cc: Henning Schild Subject: [PATCH v3 02/26] dpkg-base: add download caching of apt:// downloads Date: Tue, 28 Jan 2020 21:11:49 +0530 Message-Id: <20200128154213.22353-3-vijaikumar.kanagarajan@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200128154213.22353-1-vijaikumar.kanagarajan@gmail.com> References: <20200128154312.337e61f6@md1za8fc.ad001.siemens.net> <20200128154213.22353-1-vijaikumar.kanagarajan@gmail.com> X-TUID: n9SDRFjgJlR6 From: Henning Schild Store the results of "apt-get source" in the DL_DIR. This means we really only need to fetch once on incremental rebuilds or if multiple recipes use the same SRC_URI. We can also later collect the results and put them in a repository. Signed-off-by: Henning Schild --- meta/classes/dpkg-base.bbclass | 20 ++++++++++++++++++-- meta/conf/bitbake.conf | 1 + 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index f38168c..7a97d91 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -63,14 +63,30 @@ do_apt_fetch() { -o Dir::Etc::SourceParts="-" \ -o APT::Get::List-Cleanup="0" - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ - sh -c 'cd ${PP} && apt-get -y --only-source source ${SRC_APT}' + for uri in "${SRC_APT}"; do + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}" + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd ${PP} && apt-get -y --only-source source "$2"' my_script "${DISTRO}" "${uri}" + done + dpkg_undo_mounts } addtask apt_fetch after do_unpack before do_patch do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" +addtask cleanall_apt before do_cleanall +do_cleanall_apt[nostamp] = "1" +do_cleanall_apt() { + if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then + exit + fi + for uri in "${SRC_APT}"; do + rm -rf "${DEBSRCDIR}"/"${DISTRO}"/"$uri" + done +} + def get_package_srcdir(d): s = os.path.abspath(d.getVar("S", True)) workdir = os.path.abspath(d.getVar("WORKDIR", True)) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 0eb3b54..ab608da 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -26,6 +26,7 @@ DEPLOY_DIR = "${TMPDIR}/deploy" FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}" FILESEXTRAPATHS ?= "__default:" GITDIR = "${DL_DIR}/git" +DEBSRCDIR = "${DL_DIR}/deb-src" P = "${PN}-${PV}" PF = "${PN}-${PV}-${PR}" PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}" -- 2.17.1