From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6771003123014041600 X-Received: by 2002:a1c:5419:: with SMTP id i25mr5553407wmb.150.1580222594835; Tue, 28 Jan 2020 06:43:14 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:fc89:: with SMTP id g9ls9402931wrr.5.gmail; Tue, 28 Jan 2020 06:43:14 -0800 (PST) X-Google-Smtp-Source: APXvYqwPgdbwnGM/UQduWv6Tefg5jND5kLLJ1QKfmE8Aw45oYjowxfUpWhX9+Yp/9hyBjNkm5A5n X-Received: by 2002:adf:cd04:: with SMTP id w4mr736310wrm.219.1580222594006; Tue, 28 Jan 2020 06:43:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1580222594; cv=none; d=google.com; s=arc-20160816; b=wFSZgFKGr8h6fSSZ9cFhGy0CT4ENfWJ68zjfh5H8Nr5zk7cu0PgrahejSELEuWzpc9 aImxLqMPjlOxY5OEHRyVbBL9/wsXaK0iA0waTaufuLdBlEYm0U6TP5GMU7Lko1ehvrNS Yes61SUlNN/qnubqYQudJGZPtq7j39AKDAYhIwpDkRi/45gzL8pwXyLf3G9lzuOGaG8h 1Wmjhxq7m/MKmTJrdGzEBcWjbwIgFfnnftX1lkAlLYYB8yk3XJzvRx/pqi5LQBoVCSfp FqwThxtC8VwqGXKLNRqrxIM10z91Je/bhYG0n+iKcr/4WXHf78VVbNizwkKhnOOkgMUM ieNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=tfxvH3YdyarlFy+ADQQAtw+YYKoNKuaaoQbmZW06juc=; b=GP6iiZDZrD4dPcRMVoNtnKBJiEYGZxaoHhFgJO6RXyuNHxvhHXQnLWttr2jYs0xQ9/ L+WKATFyFUvNsEiBuRjqxCE8ygLhKTFVs3yb04lviiwpGhcTDPHlMhAUBd2/utjtqXnr Bg1JJ+OOvCRh3cw3HgdMIamL4w/szS80GJlN3gGkr6Z3Te29IlO6dlCcAr8gujYiFUpe 1l50bCQodfxgHknDd3ygQ5JgzC5qhrQonjlpR/4oX2/FCNrTH5MJ4V1ZBLUUfjqk22CF Yb+l145uTyZp2lFe+51fi5s079mKc7sPPOOmobV475XrA1SsCyHX5wZVpJZQOTXkVzIW ZoLg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id w11si126527wmk.0.2020.01.28.06.43.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Jan 2020 06:43:13 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 00SEhDpp001611 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 28 Jan 2020 15:43:13 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.193]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 00SEhCdg011270; Tue, 28 Jan 2020 15:43:12 +0100 Date: Tue, 28 Jan 2020 15:43:12 +0100 From: Henning Schild To: vijai kumar Cc: isar-users Subject: Re: [PATCHv2 09/25] meta: create DL_DIR support for all apt-get downloaded .debs Message-ID: <20200128154312.337e61f6@md1za8fc.ad001.siemens.net> In-Reply-To: <8bb53ef9-64ee-449e-a3b2-58eda2dab41e@googlegroups.com> References: <20191216115011.17664-1-henning.schild@siemens.com> <20191216115011.17664-10-henning.schild@siemens.com> <8bb53ef9-64ee-449e-a3b2-58eda2dab41e@googlegroups.com> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: rxCBStZlHOFA Hi, thanks for the report. Since you already have a patch ready, can you please share it? I would suggest to include it into the series, or apply it on top. Or squash it in. Henning On Sat, 25 Jan 2020 06:06:59 -0800 vijai kumar wrote: > On Monday, December 16, 2019 at 5:20:15 PM UTC+5:30, Henning Schild > wrote: > > > > From: Henning Schild > > > > > Hook in between the "apt-get --download-only"s and the "apt-get"s > > and copy out all the debs for later. > > > > Signed-off-by: Henning Schild > > > --- > > meta/classes/deb-dl-dir.bbclass | 15 +++++++++++++++ > > meta/classes/dpkg-base.bbclass | 1 + > > meta/classes/dpkg-gbp.bbclass | 1 + > > meta/classes/dpkg.bbclass | 1 + > > meta/classes/image-locales-extension.bbclass | 2 +- > > meta/classes/image-tools-extension.bbclass | 1 + > > meta/classes/rootfs.bbclass | 12 ++++++++++-- > > meta/conf/bitbake.conf | 1 + > > 8 files changed, 31 insertions(+), 3 deletions(-) > > create mode 100644 meta/classes/deb-dl-dir.bbclass > > > > diff --git a/meta/classes/deb-dl-dir.bbclass > > b/meta/classes/deb-dl-dir.bbclass > > new file mode 100644 > > index 0000000..e41e981 > > --- /dev/null > > +++ b/meta/classes/deb-dl-dir.bbclass > > @@ -0,0 +1,15 @@ > > +# This software is a part of ISAR. > > +# Copyright (C) 2019 Siemens AG > > +# > > +# SPDX-License-Identifier: MIT > > + > > +deb_dl_dir_export() { > > + export pc="${DEBDIR}/${DISTRO}" > > + export rootfs="${1}" > > + mkdir -p "${pc}" > > + flock "${pc}".lock -c ' \ > > + sudo find "${rootfs}"/var/cache/apt/archives/ -type f > > -iname '*\.deb' \ > > + -exec cp -f '{}' "${pc}" \; > > + sudo chown -R $(id -u):$(id -g) "${pc}" > > + ' > > +} > > diff --git a/meta/classes/dpkg-base.bbclass > > b/meta/classes/dpkg-base.bbclass > > index aa11660..fd0b07f 100644 > > --- a/meta/classes/dpkg-base.bbclass > > +++ b/meta/classes/dpkg-base.bbclass > > @@ -8,6 +8,7 @@ inherit buildchroot > > inherit debianize > > inherit terminal > > inherit repository > > +inherit deb-dl-dir > > > > DEPENDS ?= "" > > > > diff --git a/meta/classes/dpkg-gbp.bbclass > > b/meta/classes/dpkg-gbp.bbclass index 2afe9db..7e908eb 100644 > > --- a/meta/classes/dpkg-gbp.bbclass > > +++ b/meta/classes/dpkg-gbp.bbclass > > @@ -15,6 +15,7 @@ do_install_builddeps_append() { > > sudo -E chroot ${BUILDCHROOT_DIR} \ > > apt-get install -y -o Debug::pkgProblemResolver=yes \ > > --no-install-recommends --download-only > > ${GBP_DEPENDS} > > + deb_dl_dir_export "${BUILDCHROOT_DIR}" > > sudo -E chroot ${BUILDCHROOT_DIR} \ > > apt-get install -y -o Debug::pkgProblemResolver=yes \ > > --no-install-recommends ${GBP_DEPENDS} > > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > > index b26c645..7e2c86d 100644 > > --- a/meta/classes/dpkg.bbclass > > +++ b/meta/classes/dpkg.bbclass > > @@ -9,6 +9,7 @@ do_install_builddeps() { > > > > While testing this series in a downstream project, I do not see a > part of KBUILD_DEPENDS cached. > > We don't have the "linux-custom recipe rework" changes by Cedric > there. The old Linux recipe overrides > this task with its own variant. > > I have added deb_dl_* there as well to fix the issue. > > This might become a problem when more such recipes pop-up. Unlikely, > but possible. > Recipe authors should know they have to take care of this if they > decide to override > do_install_builddeps. > > > Thanks, > Vijai Kumar K > > > E="${@ isar_export_proxies(d)}" > > sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ > > ${PP}/${PPS} ${DISTRO_ARCH} --download-only > > + deb_dl_dir_export "${BUILDCHROOT_DIR}" > > sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ > > ${PP}/${PPS} ${DISTRO_ARCH} > > dpkg_undo_mounts > > diff --git a/meta/classes/image-locales-extension.bbclass > > b/meta/classes/image-locales-extension.bbclass > > index 3c0758f..0f0d0ca 100644 > > --- a/meta/classes/image-locales-extension.bbclass > > +++ b/meta/classes/image-locales-extension.bbclass > > @@ -25,7 +25,7 @@ def get_nopurge(d): > > j.split()[0].split(".")[0], > > j.split()[0])))) > > > > -ROOTFS_INSTALL_COMMAND_BEFORE_CLEAN += > > "image_install_localepurge_download" > > +ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += > > "image_install_localepurge_download" > > image_install_localepurge_download[weight] = "40" > > image_install_localepurge_download() { > > sudo -E chroot '${ROOTFSDIR}' \ > > diff --git a/meta/classes/image-tools-extension.bbclass > > b/meta/classes/image-tools-extension.bbclass > > index b8531cb..ec67d94 100644 > > --- a/meta/classes/image-tools-extension.bbclass > > +++ b/meta/classes/image-tools-extension.bbclass > > @@ -34,6 +34,7 @@ do_install_imager_deps() { > > --allow-unauthenticated --allow-downgrades > > --download-only install \ > > ${IMAGER_INSTALL}' > > > > + deb_dl_dir_export ${BUILDCHROOT_DIR} > > sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' \ > > apt-get -o Debug::pkgProblemResolver=yes > > --no-install-recommends -y \ > > --allow-unauthenticated --allow-downgrades install \ > > diff --git a/meta/classes/rootfs.bbclass > > b/meta/classes/rootfs.bbclass index b4dbfa3..a6b6110 100644 > > --- a/meta/classes/rootfs.bbclass > > +++ b/meta/classes/rootfs.bbclass > > @@ -1,6 +1,8 @@ > > # This software is a part of ISAR. > > # Copyright (c) Siemens AG, 2019 > > > > +inherit deb-dl-dir > > + > > ROOTFS_ARCH ?= "${DISTRO_ARCH}" > > ROOTFS_DISTRO ?= "${DISTRO}" > > ROOTFS_PACKAGES ?= "" > > @@ -119,8 +121,14 @@ rootfs_install_pkgs_download() { > > /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only > > ${ROOTFS_PACKAGES} > > } > > > > -ROOTFS_INSTALL_COMMAND_BEFORE_CLEAN ??= "" > > -ROOTFS_INSTALL_COMMAND += "${ROOTFS_INSTALL_COMMAND_BEFORE_CLEAN}" > > +ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= "" > > +ROOTFS_INSTALL_COMMAND += > > "${ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT}" > > + > > +ROOTFS_INSTALL_COMMAND += "rootfs_export_package_cache" > > +rootfs_export_package_cache[weight] = "5" > > +rootfs_export_package_cache() { > > + deb_dl_dir_export ${ROOTFSDIR} > > +} > > > > ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if > > (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}" > > rootfs_install_clean_files[weight] = "2" > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > > index ab608da..d86c5b9 100644 > > --- a/meta/conf/bitbake.conf > > +++ b/meta/conf/bitbake.conf > > @@ -26,6 +26,7 @@ DEPLOY_DIR = "${TMPDIR}/deploy" > > FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}" > > FILESEXTRAPATHS ?= "__default:" > > GITDIR = "${DL_DIR}/git" > > +DEBDIR = "${DL_DIR}/deb" > > DEBSRCDIR = "${DL_DIR}/deb-src" > > P = "${PN}-${PV}" > > PF = "${PN}-${PV}-${PR}" > > -- > > 2.24.1 > > > > >