From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6816610675337461760 X-Received: by 2002:a05:6402:688:: with SMTP id f8mr16115928edy.233.1587540502648; Wed, 22 Apr 2020 00:28:22 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:1b4a:: with SMTP id p10ls537401ejg.9.gmail; Wed, 22 Apr 2020 00:28:22 -0700 (PDT) X-Google-Smtp-Source: APiQypLLpQrTUByBku9ZMegko7D2PZaSFgtgb2MU1zjcIdFw30hM2bzaSOFGE2F8Z3jeXm46yuu3 X-Received: by 2002:a17:906:359b:: with SMTP id o27mr25068031ejb.282.1587540502099; Wed, 22 Apr 2020 00:28:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587540502; cv=none; d=google.com; s=arc-20160816; b=fCUmVZrWd3hpe04/Tq2ll+GuQfgH8oLiWdEzlCdlM3KbZiXIAweJFGUHWl+YmH8NnS X5Y6I+M3tfVuv7hIyQRtrLocRl8BPtFP0XXqWGXpwqUc2Dxnlby0hIGdaoQBqBCgP2s0 iPkqBzj6AeS+zbv6na27l4K6a+MkBwZPRTIN+VRiFpebpSsHR/JP53kn3d2rh3WyUUeC HwnErdeX/HKV1jG3qedWecC9/S2iHyeTpm8x0+8IoI8otXHgnjDMKJrYQPYNGREOsDcI bV9rinDvemkBGjPq2crtLBLI20ncghExtwq6pxH2WOX51gIWqLsyc0OAboLSbjMlVm5y DzoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=5P4przlyySClcjfUqBvq216ypIsHP7Zi/TTRDX2wD0M=; b=QsNVPbZ7JW/2d4c4wQ6yb4QVJ3tQdARkOqM+lX4rTNUSWPlHDZOpfxF9Qw12ftkpvK r9cEanFjjUVO7UAzuQwrTaFjOiaaxXaHjfPRZdVMPhdUAqUE4bqSEMc5ToAyVk79OK3k ckrd7KkmMCR0JtcsYg9tftvHLxgtjAa30bBo0TRiezOHkgmvZ5+K4XEUyeHVMKmZ35O3 BYFtqSfN0mS/xPwiD+UpRxr6M0HfcqYjZcVBavQbz8QYKmK8OhvOCNCiX0ZF1W9bKRqX HMfxE7QCKuSloDyTsYT5s0qtFJfIs2sKw9u8omENa58xbJ0hS2WT4FzbzfQwFcicONlH JKtQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id v14si322608edr.4.2020.04.22.00.28.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Apr 2020 00:28:22 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 03M7SLw2019389 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Apr 2020 09:28:21 +0200 Received: from md1za8fc.ad001.siemens.net ([167.87.23.203]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 03M7SKMi020600; Wed, 22 Apr 2020 09:28:20 +0200 Date: Wed, 22 Apr 2020 09:28:19 +0200 From: Henning Schild To: Vijai Kumar K Cc: , , , Vijai Kumar K Subject: Re: [PATCH v5 03/13] meta: cache deb srcs as part of postprocessing Message-ID: <20200422092819.49af1c46@md1za8fc.ad001.siemens.net> In-Reply-To: <20200417093040.15130-4-Vijaikumar_Kanagarajan@mentor.com> References: <20200417093040.15130-1-Vijaikumar_Kanagarajan@mentor.com> <20200417093040.15130-4-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: zESWOGm8FYSd On Fri, 17 Apr 2020 15:00:30 +0530 Vijai Kumar K wrote: > Collect the deb sources of the corresponding deb binaries cached > in DEBDIR as part of image postprocess. > > Signed-off-by: Vijai Kumar K > --- > meta/classes/deb-dl-dir.bbclass | 39 > +++++++++++++++++++++++++++++++++ meta/classes/image.bbclass | > 2 +- meta/classes/rootfs.bbclass | 8 +++++++ > 3 files changed, 48 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/deb-dl-dir.bbclass > b/meta/classes/deb-dl-dir.bbclass index 29a3d67..472b9fe 100644 > --- a/meta/classes/deb-dl-dir.bbclass > +++ b/meta/classes/deb-dl-dir.bbclass > @@ -5,6 +5,45 @@ > > inherit repository > > +debsrc_download() { > + export rootfs="$1" > + export rootfs_distro="$2" > + mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}" > + ( flock 9 > + set -e > + printenv | grep -q BB_VERBOSE_LOGS && set -x > + sudo -E -s <<'EOSUDO' > + mkdir -p "${rootfs}/deb-src" > + mountpoint -q "${rootfs}/deb-src" || \ > + mount --bind "${DEBSRCDIR}" "${rootfs}/deb-src" > +EOSUDO > + find "${rootfs}/var/cache/apt/archives/" -maxdepth 1 -type f > -iname '*\.deb' | while read package; do Using that cache as input is not a good idea. It does not tell you what you have installed. It might be overfull because of the download cache, and it might be too empty because of cache cleanups. You probably want to iterate of the list of installed packages, or better that manifest we have. Henning > + local src="$( dpkg-deb --show --showformat '${Source}' > "${package}" )" > + # If the binary package version and source package version > are different, then the > + # source package version will be present inside "()" of the > Source field. > + local version="$( echo "$src" | cut -sd "(" -f2 | cut -sd > ")" -f1 )" > + if [ -z ${version} ]; then > + version="$( dpkg-deb --show --showformat '${Version}' > "${package}" )" > + fi > + # Now strip any version information that might be available. > + src="$( echo "$src" | cut -d' ' -f1 )" > + # If there is no source field, then the source package has > the same name as the > + # binary package. > + if [ -z "${src}" ];then > + src="$( dpkg-deb --show --showformat '${Package}' > "${package}" )" > + fi > + > + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \ > + sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd > "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source > source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" > "${version}" > + done > + sudo -E -s <<'EOSUDO' > + mountpoint -q "${rootfs}/deb-src" && \ > + umount -l "${rootfs}/deb-src" > + rm -rf "${rootfs}/deb-src" > +EOSUDO > + ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock" > +} > + > deb_dl_dir_import() { > export pc="${DEBDIR}/${2}" > export rootfs="${1}" > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 6b04c0a..fcaebd6 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -63,7 +63,7 @@ image_do_mounts() { > } > > ROOTFSDIR = "${IMAGE_ROOTFS}" > -ROOTFS_FEATURES += "clean-package-cache generate-manifest" > +ROOTFS_FEATURES += "clean-package-cache generate-manifest > cach-deb-src" ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} > ${IMAGE_INSTALL}" ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" > > diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass > index cee358c..ee57989 100644 > --- a/meta/classes/rootfs.bbclass > +++ b/meta/classes/rootfs.bbclass > @@ -185,6 +185,14 @@ python do_rootfs_install() { > } > addtask rootfs_install before do_rootfs_postprocess after do_unpack > > +ROOTFS_POSTPROCESS_COMMAND += > "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-deb-src', > 'cache_deb_src', '', d)}" +cache_deb_src() { > + rootfs_install_resolvconf > + deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_DISTRO} > + debsrc_download ${ROOTFSDIR} ${ROOTFS_DISTRO} > + rootfs_install_clean_files > +} > + > ROOTFS_POSTPROCESS_COMMAND += > "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', > 'rootfs_postprocess_clean_package_cache', '', d)}" > rootfs_postprocess_clean_package_cache() { sudo -E chroot > '${ROOTFSDIR}' \