From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6790334981638979584 X-Received: by 2002:a5d:5688:: with SMTP id f8mr7501362wrv.245.1586340270633; Wed, 08 Apr 2020 03:04:30 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:4690:: with SMTP id u16ls4780974wrq.11.gmail; Wed, 08 Apr 2020 03:04:30 -0700 (PDT) X-Google-Smtp-Source: APiQypL0j2WpnZ9ScqitSKYf4P7trS7AgfP38OvPIYtyKVX23/3QLDUzaUDqk+0wMwX1XE5T6Tk/ X-Received: by 2002:a5d:6a4e:: with SMTP id t14mr7883216wrw.101.1586340270030; Wed, 08 Apr 2020 03:04:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1586340270; cv=none; d=google.com; s=arc-20160816; b=fQTr0rbd0JF6DJSZVsGJ42GUFxgXai36twLm/uu111CMyOdndgNO91SMDeSgSHbTHB nE5fS9clgYtbbw2Pt+rOc6tQpcmw6e47SPKXMETY7qwuzBCRMVrameFw8SXxaQKcsNiM ZTx25Mre8o+uRHqReyEAW6erFfV9uT4gxl+o7QSc973c95uoSgbuE7aGTYESFBouyROX ZV1K/UAHXw2HCQbWMJSr/YPEQ2FIwZCHDNqxdtoYZP7dmr6QmIaBVEMYvJVkkIWVRiV4 CjUKnAYHhOB00DAOPkIrCCYRTG2ru0YWg31yJ/WKcsc76cNw5kImbgntaFuuxS9CibzB jDaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=1bDQ0AjAFzIs/mWR0oWGq8knQrRUgMqheoHE+I1WQsY=; b=PZj7PG9jXoMteKOy24scQphZnEiEsP3Jo9/jWIsM8LPLo+VKBuAMXZEOXshESWdprk +JXU9dqH2Ss79GP41H+CzclW7X/SOzzwdzsZb65YAlvrEznI0pMp/q+DwA0ACESoTjzg zHF9eqJyf1d5bsD0w8NIK1tEo3DZP6g/4F2hbAHDftHW5qGIpW/70T3ZdvVKgJ76mQb/ GlW0t29ZNm3l8v6rBmLjHUrazjk/kZXuR1NdtbHWBx8H5fa1SmxKgET6dtg7VadTwMub sjQ0pwxUbBhluKHfJzLmIH/uZ3kofITX+exOFHzeZj/lEbHLJS3SUV7zzQ001u5rVqgb 2AVg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id d6si289990wrq.3.2020.04.08.03.04.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Apr 2020 03:04:30 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 038A4TSv011433 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 8 Apr 2020 12:04:29 +0200 Received: from md1za8fc.ad001.siemens.net ([158.92.8.69]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 038A4SXN015955; Wed, 8 Apr 2020 12:04:28 +0200 Date: Wed, 8 Apr 2020 12:04:27 +0200 From: Henning Schild To: Vijai Kumar K Cc: , Vijai Kumar K Subject: Re: [PATCH v4 2/2] meta: cache deb srcs as part of postprocessing Message-ID: <20200408120427.366b3968@md1za8fc.ad001.siemens.net> In-Reply-To: <20200403130551.2158-2-Vijaikumar_Kanagarajan@mentor.com> References: <20200403130551.2158-1-Vijaikumar_Kanagarajan@mentor.com> <20200403130551.2158-2-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: AmCE554cf48r Am Fri, 3 Apr 2020 18:35:51 +0530 schrieb Vijai Kumar K : > Collect the deb sources of the corresponding deb binaries cached > in DEBDIR as part of postprocess for those to be later included > into the final base-apt by do_cache. > > Signed-off-by: Vijai Kumar K > --- > meta/classes/image.bbclass | 2 +- > meta/classes/rootfs.bbclass | 46 > +++++++++++++++++++++++++++++++++++++ 2 files changed, 47 > insertions(+), 1 deletion(-) > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 9fa58f8..1c7a527 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -60,7 +60,7 @@ image_do_mounts() { > } > > ROOTFSDIR = "${IMAGE_ROOTFS}" > -ROOTFS_FEATURES += "copy-package-cache clean-package-cache > generate-manifest" +ROOTFS_FEATURES += "copy-package-cache > clean-package-cache generate-manifest cache-deb-src" ROOTFS_PACKAGES > += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" ROOTFS_MANIFEST_DEPLOY_DIR > ?= "${DEPLOY_DIR_IMAGE}" > diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass > index 8bb003d..7bfdfc9 100644 > --- a/meta/classes/rootfs.bbclass > +++ b/meta/classes/rootfs.bbclass > @@ -201,6 +201,52 @@ rootfs_generate_manifest () { > ${ROOTFS_MANIFEST_DEPLOY_DIR}/"${PF}".manifest > } > > +ROOTFS_POSTPROCESS_COMMAND += > "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-deb-src', > 'cache_deb_src', '', d)}" +cache_deb_src() { > + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then > + return 0 > + fi Should the source packages not all end up in the cache, so they can and probably should be fetched from there. > + mkdir -p "${DEBSRCDIR}"/"${DISTRO}" > + > + sudo -s <<'EOSUDO' > + cp -L /etc/resolv.conf '${ROOTFSDIR}/etc' > + mkdir -p '${ROOTFSDIR}/deb-src' > + mountpoint -q '${ROOTFSDIR}/deb-src' || \ > + mount --bind '${DEBSRCDIR}' '${ROOTFSDIR}/deb-src' > +EOSUDO > + > + sudo -E chroot ${ROOTFSDIR} /usr/bin/apt-get update > + > + find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read > package; do > + local src="$( dpkg-deb --show --showformat '${Source}' > "${package}" )" > + # If the binary package version and source package version > are different, then the > + # source package version will be present inside "()" of the > Source field. > + local version="$( echo "$src" | cut -sd "(" -f2 | cut -sd > ")" -f1 )" > + if [ -z ${version} ]; then > + version="$( dpkg-deb --show --showformat '${Version}' > "${package}" )" > + fi > + # Now strip any version information that might be available. > + src="$( echo "$src" | cut -d' ' -f1 )" > + # If there is no source field, then the source package has > the same name as the > + # binary package. > + if [ -z "${src}" ];then > + src="$( dpkg-deb --show --showformat '${Package}' > "${package}" )" > + fi > + > + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${ROOTFSDIR} > \ > + sh -c 'mkdir -p "/deb-src/${1}/${2}" && cd > "/deb-src/${1}/${2}" && \ > + apt-get -y --download-only --only-source source > "$2"="$3"' \ > + download-src "${DISTRO}" "${src}" "${version}" > + done Looks like we are going online without proxy configuration here. It also needs a BB_NO_NETWORK guard. And i would suggest to generate the list of things you want to fetch, factor out the fetcher from dpkg-base and reuse is instead of copying it. And i would personally like a new series of patches to be sent without "in-reply-to". Maybe its my client but i find these deeply nested threads very hard to follow. Henning > + sudo -s <<'EOSUDO' > + mountpoint -q '${ROOTFSDIR}/deb-src' && \ > + umount -l ${ROOTFSDIR}/deb-src > + rm -rf '${ROOTFSDIR}/etc/resolv.conf' > +EOSUDO > +} > + > do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" > python do_rootfs_postprocess() { > # Take care that its correctly mounted: