From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6790334981638979584 X-Received: by 2002:a17:90b:82:: with SMTP id bb2mr4481965pjb.25.1586342248317; Wed, 08 Apr 2020 03:37:28 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a62:6456:: with SMTP id y83ls4060444pfb.7.gmail; Wed, 08 Apr 2020 03:37:27 -0700 (PDT) X-Google-Smtp-Source: APiQypI83k0cy0aJfJea7iCq+WZRtSSYmf3iugPU79aZvoG2DEftN3Gj/vl/hE6y/Mewt+7GbKDP X-Received: by 2002:a63:a70b:: with SMTP id d11mr6448274pgf.358.1586342247630; Wed, 08 Apr 2020 03:37:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1586342247; cv=none; d=google.com; s=arc-20160816; b=KqaWRXMyRkY9HY9w8iPNGx+dRhQr0FaUoMHY4t7Vh9g+1tZMxDYyPrcfmhK9BZ/iVx taqe6gnIqVX60fI47Zw8Hr2capUlANYxZaJ1+aODasdeMyzVokZ3FRNK9mriDy22Hndu 270Jei/FJcT5elBme+KD1GEmvOHmIiCBDLBVaPc2sa93AyopQom2xtkqYhxIgAn6vPCq pLl0zLUvDw/sKBMKxiK2JvovmyGqVCifykA1kPMWxHFEwO6wOpp+DRH+zStV/XRPdWdX baJzSWGcPgQgLhNlwGIhhWDEOeU+L15ry0+ZY88vAi6ZdDtYtBPE1JSYlg2RSavC36VI t5xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=C0619CJtQDFerd9P2YpXMkY77TZLz5eeW2S6yuMlSzo=; b=n8A9bvlv74te2YEYLTIsB1pDIQr7ba0QymDORcx05UUX1vHrQLcc/XyZryWJqGebDt Xqkjtaqcyjyse1HMccJ2FJGFg+vZtrQrTL1IKoYhtKkpuF752aRxggIcGEh6kSRFYxUY EHxDmISQI6UO98RIDjOLM2UGQdP7PB0GKG4QRtZGU1Zm7MGOMGOLmNaHZyHeswWcVb/s 7hg37Z8JBzvh4E2eYHexHWk7Q3TD+SGqHgXkAdSAvNdBsyUQKCYMEm44c0/Rnq/FKMd+ 2QGG7Ucfh3to8d7N62wLgJXiJMPtkMyQ2awPx4s3F//qoh6vqE2v/fJp2/IpUVHz26Xk kTDQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BEn6xA81; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com. [2607:f8b0:4864:20::842]) by gmr-mx.google.com with ESMTPS id w18si566242pfi.4.2020.04.08.03.37.27 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 08 Apr 2020 03:37:27 -0700 (PDT) Received-SPF: pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) client-ip=2607:f8b0:4864:20::842; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BEn6xA81; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-qt1-x842.google.com with SMTP id b10so5128857qtt.9 for ; Wed, 08 Apr 2020 03:37:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=C0619CJtQDFerd9P2YpXMkY77TZLz5eeW2S6yuMlSzo=; b=BEn6xA81qkBBvLz0Bt4nCem+Pz+KRr7pPT+mLpH3rXTT+ISZil5bNUUb1zdmPGz+HP oFg9G+RgosERhj2U38YRNbT+DQfrvfsK29AsPUwB6quAIyyJcKzgICofQw2F5ujx/HzS oIsy3W1C8O94IIqkZk4WVswyBDSco8GFeClAo6mo6vlQnQK1SrNKzZdlGhEY1r3hFnIN zqGQPmrthe56JyRm2sXGEQVY5QOKW42Io/MRy4QYEeXiIfM0ezKNj54VYZKW004reSTf pBASzu4HRT8IolN9LX9LE3Q5CqaeyiYX6fNXf3aczbCSw5VVSmy+xE6Hx8hIn5dWJl06 dcDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=C0619CJtQDFerd9P2YpXMkY77TZLz5eeW2S6yuMlSzo=; b=pKonKb64lH0K4fesXZFp4epGkSfkWKgt0b6HIjA3MmkjsPVLlLqFAI/B6HGachsvJ6 HKj4UZZMeiWwoh7PX2cyn0Csterv1ituz9Ra7Fki/Rut/QzhuqPuVc0Al1HcFKwHKql3 BouCWF5p5H3TCiN1S9J41eN7foVeMRe2+StN6yoxy+5hP1+C4Lag52Pa1JzecMk++vu7 2zmBA1d3vAuZfjdNgvMTMIzAwdhMyHdYgYd1XWilMF2RwLDltcNTX7dzbRPxyTfKf41C cKgMnyFr9IQakfS5xbcqr07UbYD1mdux0ctRa66Y749d81RBG7/bg+yfbwhbhBQ2vtF+ eWjw== X-Gm-Message-State: AGi0PuY3Oui+d/pyrnWTVDM4lkyEH4ayJnBPiumIwAbpBen16Miqg3mg YWwtXh8Ea7b8AHDu31cKxDjNSPM9kGAMXiY1r+Y= X-Received: by 2002:ac8:6d31:: with SMTP id r17mr6913433qtu.68.1586342246575; Wed, 08 Apr 2020 03:37:26 -0700 (PDT) MIME-Version: 1.0 References: <20200403130551.2158-1-Vijaikumar_Kanagarajan@mentor.com> <20200403130551.2158-2-Vijaikumar_Kanagarajan@mentor.com> <20200408120427.366b3968@md1za8fc.ad001.siemens.net> In-Reply-To: <20200408120427.366b3968@md1za8fc.ad001.siemens.net> From: vijai kumar Date: Wed, 8 Apr 2020 16:07:15 +0530 Message-ID: Subject: Re: [PATCH v4 2/2] meta: cache deb srcs as part of postprocessing To: Henning Schild Cc: isar-users , Vijai Kumar K Content-Type: text/plain; charset="UTF-8" X-TUID: xigEySbgWxqN On Wed, Apr 8, 2020 at 3:34 PM Henning Schild wrote: > > Am Fri, 3 Apr 2020 18:35:51 +0530 > schrieb Vijai Kumar K : > > > Collect the deb sources of the corresponding deb binaries cached > > in DEBDIR as part of postprocess for those to be later included > > into the final base-apt by do_cache. > > > > Signed-off-by: Vijai Kumar K > > --- > > meta/classes/image.bbclass | 2 +- > > meta/classes/rootfs.bbclass | 46 > > +++++++++++++++++++++++++++++++++++++ 2 files changed, 47 > > insertions(+), 1 deletion(-) > > > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > > index 9fa58f8..1c7a527 100644 > > --- a/meta/classes/image.bbclass > > +++ b/meta/classes/image.bbclass > > @@ -60,7 +60,7 @@ image_do_mounts() { > > } > > > > ROOTFSDIR = "${IMAGE_ROOTFS}" > > -ROOTFS_FEATURES += "copy-package-cache clean-package-cache > > generate-manifest" +ROOTFS_FEATURES += "copy-package-cache > > clean-package-cache generate-manifest cache-deb-src" ROOTFS_PACKAGES > > += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" ROOTFS_MANIFEST_DEPLOY_DIR > > ?= "${DEPLOY_DIR_IMAGE}" > > diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass > > index 8bb003d..7bfdfc9 100644 > > --- a/meta/classes/rootfs.bbclass > > +++ b/meta/classes/rootfs.bbclass > > @@ -201,6 +201,52 @@ rootfs_generate_manifest () { > > ${ROOTFS_MANIFEST_DEPLOY_DIR}/"${PF}".manifest > > } > > > > +ROOTFS_POSTPROCESS_COMMAND += > > "${@bb.utils.contains('ROOTFS_FEATURES', 'cache-deb-src', > > 'cache_deb_src', '', d)}" +cache_deb_src() { > > + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then > > + return 0 > > + fi > > Should the source packages not all end up in the cache, so they can and > probably should be fetched from there. Sorry. But I am not able to understand this. Can you please explain it again? > > > + mkdir -p "${DEBSRCDIR}"/"${DISTRO}" > > + > > + sudo -s <<'EOSUDO' > > + cp -L /etc/resolv.conf '${ROOTFSDIR}/etc' > > + mkdir -p '${ROOTFSDIR}/deb-src' > > + mountpoint -q '${ROOTFSDIR}/deb-src' || \ > > + mount --bind '${DEBSRCDIR}' '${ROOTFSDIR}/deb-src' > > +EOSUDO > > + > > + sudo -E chroot ${ROOTFSDIR} /usr/bin/apt-get update > > + > > + find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read > > package; do > > + local src="$( dpkg-deb --show --showformat '${Source}' > > "${package}" )" > > + # If the binary package version and source package version > > are different, then the > > + # source package version will be present inside "()" of the > > Source field. > > + local version="$( echo "$src" | cut -sd "(" -f2 | cut -sd > > ")" -f1 )" > > + if [ -z ${version} ]; then > > + version="$( dpkg-deb --show --showformat '${Version}' > > "${package}" )" > > + fi > > + # Now strip any version information that might be available. > > + src="$( echo "$src" | cut -d' ' -f1 )" > > + # If there is no source field, then the source package has > > the same name as the > > + # binary package. > > + if [ -z "${src}" ];then > > + src="$( dpkg-deb --show --showformat '${Package}' > > "${package}" )" > > + fi > > + > > + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${ROOTFSDIR} > > \ > > + sh -c 'mkdir -p "/deb-src/${1}/${2}" && cd > > "/deb-src/${1}/${2}" && \ > > + apt-get -y --download-only --only-source source > > "$2"="$3"' \ > > + download-src "${DISTRO}" "${src}" "${version}" > > + done > > Looks like we are going online without proxy configuration here. It > also needs a BB_NO_NETWORK guard. Will take care of that. > > And i would suggest to generate the list of things you want to fetch, > factor out the fetcher from dpkg-base and reuse is instead of copying > it. Sure. I will have a look into how I can reuse that part. > > And i would personally like a new series of patches to be sent without > "in-reply-to". Maybe its my client but i find these deeply nested > threads very hard to follow. No Problem. Will send the next series separately. Thanks, Vijai Kumar K > > Henning > > > + sudo -s <<'EOSUDO' > > + mountpoint -q '${ROOTFSDIR}/deb-src' && \ > > + umount -l ${ROOTFSDIR}/deb-src > > + rm -rf '${ROOTFSDIR}/etc/resolv.conf' > > +EOSUDO > > +} > > + > > do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" > > python do_rootfs_postprocess() { > > # Take care that its correctly mounted: >