public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: vijai kumar <vijaikumar.kanagarajan@gmail.com>
To: Henning Schild <henning.schild@siemens.com>
Cc: isar-users <isar-users@googlegroups.com>
Subject: Re: [PATCHv2 02/25] dpkg-base: add download caching of apt:// downloads
Date: Fri, 10 Jan 2020 15:28:39 +0530	[thread overview]
Message-ID: <CALLGG_KKXSxj3SpjHiJL5B99vEOm5W98FRWz7a7tOheQrFrcow@mail.gmail.com> (raw)
In-Reply-To: <20200110105111.3d935274@md1za8fc.ad001.siemens.net>

[-- Attachment #1: Type: text/plain, Size: 3539 bytes --]

On Fri 10 Jan, 2020, 3:21 PM Henning Schild, <henning.schild@siemens.com>
wrote:

> Am Thu, 9 Jan 2020 03:27:25 -0800
> schrieb vijai kumar <vijaikumar.kanagarajan@gmail.com>:
>
> > On Monday, December 16, 2019 at 5:20:15 PM UTC+5:30, Henning Schild
> > wrote:
> > >
> > > From: Henning Schild <hennin...@siemens.com <javascript:>>
> > >
> > > Store the results of "apt-get source" in the DL_DIR. This means we
> > > really only need to fetch once on incremental rebuilds or if
> > > multiple recipes use the same SRC_URI.
> > > We can also later collect the results and put them in a repository.
> > >
> > > Signed-off-by: Henning Schild <hennin...@siemens.com <javascript:>>
> > > ---
> > >  meta/classes/dpkg-base.bbclass | 20 ++++++++++++++++++--
> > >  meta/conf/bitbake.conf         |  1 +
> > >  2 files changed, 19 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/meta/classes/dpkg-base.bbclass
> > > b/meta/classes/dpkg-base.bbclass
> > > index f38168c..7a97d91 100644
> > > --- a/meta/classes/dpkg-base.bbclass
> > > +++ b/meta/classes/dpkg-base.bbclass
> > > @@ -63,14 +63,30 @@ do_apt_fetch() {
> > >          -o Dir::Etc::SourceParts="-" \
> > >          -o APT::Get::List-Cleanup="0"
> > >
> > > -    sudo -E chroot --userspec=$( id -u ):$( id -g )
> > > ${BUILDCHROOT_DIR} \
> > > -        sh -c 'cd ${PP} && apt-get -y --only-source source
> > > ${SRC_APT}'
> > > +    for uri in "${SRC_APT}"; do
> > > +        sudo -E chroot --userspec=$( id -u ):$( id -g )
> > > ${BUILDCHROOT_DIR} \
> > > +            sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd
> > > /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only
> > > --only-source source "$2"' my_script "${DISTRO}" "${uri}"
> > > +        sudo -E chroot --userspec=$( id -u ):$( id -g )
> > > ${BUILDCHROOT_DIR} \
> > > +            sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd
> > > ${PP} && apt-get -y --only-source source "$2"' my_script
> > > "${DISTRO}" "${uri}"
> >
> > Hi Henning,
> >
> > Thank you for the base-apt rework.
> > BTW. What does my_script do here?
>
> It fills $0 of the embedded shell script with a "program name".
>
> Henning
>

Ah. I see. I should have known :) my bad.

Thanks,
Vijai Kumar K

> Thanks,
> > Vijai Kumar K
> >
> > +    done
> > > +
> > >      dpkg_undo_mounts
> > >  }
> > >
> > >  addtask apt_fetch after do_unpack before do_patch
> > >  do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
> > >
> > > +addtask cleanall_apt before do_cleanall
> > > +do_cleanall_apt[nostamp] = "1"
> > > +do_cleanall_apt() {
> > > +    if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then
> > > +        exit
> > > +    fi
> > > +    for uri in "${SRC_APT}"; do
> > > +        rm -rf "${DEBSRCDIR}"/"${DISTRO}"/"$uri"
> > > +    done
> > > +}
> > > +
> > >  def get_package_srcdir(d):
> > >      s = os.path.abspath(d.getVar("S", True))
> > >      workdir = os.path.abspath(d.getVar("WORKDIR", True))
> > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > > index 0eb3b54..ab608da 100644
> > > --- a/meta/conf/bitbake.conf
> > > +++ b/meta/conf/bitbake.conf
> > > @@ -26,6 +26,7 @@ DEPLOY_DIR = "${TMPDIR}/deploy"
> > >  FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
> > >  FILESEXTRAPATHS ?= "__default:"
> > >  GITDIR = "${DL_DIR}/git"
> > > +DEBSRCDIR = "${DL_DIR}/deb-src"
> > >  P = "${PN}-${PV}"
> > >  PF = "${PN}-${PV}-${PR}"
> > >  PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE',
> > > False),d)[0] or 'defaultpkgname'}"
> > > --
> > > 2.24.1
> > >
> > >
> >
>
>

[-- Attachment #2: Type: text/html, Size: 5786 bytes --]

  reply	other threads:[~2020-01-10  9:59 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-16 11:49 [PATCHv2 00/25] base-apt-rework Henning Schild
2019-12-16 11:49 ` [PATCHv2 01/25] repository: new class to deal with repos Henning Schild
2019-12-16 11:49 ` [PATCHv2 02/25] dpkg-base: add download caching of apt:// downloads Henning Schild
2020-01-09 11:27   ` vijai kumar
2020-01-10  9:51     ` Henning Schild
2020-01-10  9:58       ` vijai kumar [this message]
2019-12-16 11:49 ` [PATCHv2 03/25] meta: dpkg-base: convert "exit" into "return 0" Henning Schild
2019-12-16 11:49 ` [PATCHv2 04/25] base-apt: change the sources.list to also offer deb-src Henning Schild
2019-12-16 11:49 ` [PATCHv2 05/25] base-apt: add deb-src packages as well Henning Schild
2019-12-16 11:49 ` [PATCHv2 06/25] base-apt: do not skip gpg check when it is signed Henning Schild
2019-12-16 11:49 ` [PATCHv2 07/25] ci: conf: add "hello" to the sample config and every build Henning Schild
2019-12-16 11:49 ` [PATCHv2 08/25] meta: split all apt-get invocations into download and execution Henning Schild
2019-12-16 11:49 ` [PATCHv2 09/25] meta: create DL_DIR support for all apt-get downloaded .debs Henning Schild
2020-01-25 14:06   ` vijai kumar
2020-01-28 14:43     ` Henning Schild
2020-01-28 15:41       ` Base-apt rework v3 Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 01/26] repository: new class to deal with repos Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 02/26] dpkg-base: add download caching of apt:// downloads Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 03/26] meta: dpkg-base: convert "exit" into "return 0" Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 04/26] base-apt: change the sources.list to also offer deb-src Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 05/26] base-apt: add deb-src packages as well Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 06/26] base-apt: do not skip gpg check when it is signed Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 07/26] ci: conf: add "hello" to the sample config and every build Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 08/26] meta: split all apt-get invocations into download and execution Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 09/26] meta: create DL_DIR support for all apt-get downloaded .debs Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 10/26] meta: import DL_DIR debs before apt-get download steps Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 11/26] meta: include DL_DIR deb import/export into bootstrap Vijai Kumar K
2020-01-28 15:41         ` [PATCH v3 12/26] base-apt: populate from DEBDIR as well Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 13/26] base-apt: drop the "apt_cache" feature Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 14/26] base-apt: do not copy debs directly out of rootfss anymore Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 15/26] base-apt: rework base-apt population Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 16/26] base-apt: move class "base-apt-helper" into only user Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 17/26] CI: include "isar-disable-apt-cache" into all CI images Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 18/26] CI: include "cowsay" into default build to test dpkg-gbp Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 19/26] CI: set BB_NO_NETWORK for cached rebuild Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 20/26] meta: repository: implement repo_contains_package and use it in base_apt Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 21/26] meta: repository: simplify the matching of packages Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 22/26] meta: deb-dl-dir: do not cache debs from isar-apt Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 23/26] base-apt: pull base-apt population to the front of the build chain Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 24/26] meta: base-apt: remove isar-apt check from population Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 25/26] meta: deb-dl-dir: only export newly downloaded files Vijai Kumar K
2020-01-28 15:42         ` [PATCH v3 26/26] meta/recipes-kernel: Export KBUILD_DEPENDS to cache Vijai Kumar K
2020-01-28 16:52           ` Henning Schild
2020-01-28 19:45             ` vijai kumar
2020-01-28 21:22               ` Henning Schild
2020-01-29  5:29                 ` Vijai Kumar K
2020-01-31  9:27                   ` Henning Schild
2019-12-16 11:49 ` [PATCHv2 10/25] meta: import DL_DIR debs before apt-get download steps Henning Schild
2019-12-16 11:49 ` [PATCHv2 11/25] meta: include DL_DIR deb import/export into bootstrap Henning Schild
2019-12-16 11:49 ` [PATCHv2 12/25] base-apt: populate from DEBDIR as well Henning Schild
2019-12-16 11:49 ` [PATCHv2 13/25] base-apt: drop the "apt_cache" feature Henning Schild
2019-12-16 11:50 ` [PATCHv2 14/25] base-apt: do not copy debs directly out of rootfss anymore Henning Schild
2019-12-16 11:50 ` [PATCHv2 15/25] base-apt: rework base-apt population Henning Schild
2019-12-16 11:50 ` [PATCHv2 16/25] base-apt: move class "base-apt-helper" into only user Henning Schild
2019-12-16 11:50 ` [PATCHv2 17/25] CI: include "isar-disable-apt-cache" into all CI images Henning Schild
2019-12-16 11:50 ` [PATCHv2 18/25] CI: include "cowsay" into default build to test dpkg-gbp Henning Schild
2019-12-16 11:50 ` [PATCHv2 19/25] CI: set BB_NO_NETWORK for cached rebuild Henning Schild
2019-12-16 11:50 ` [PATCHv2 20/25] meta: repository: implement repo_contains_package and use it in base_apt Henning Schild
2019-12-16 11:50 ` [PATCHv2 21/25] meta: repository: simplify the matching of packages Henning Schild
2019-12-16 11:50 ` [PATCHv2 22/25] meta: deb-dl-dir: do not cache debs from isar-apt Henning Schild
2019-12-16 11:50 ` [PATCHv2 23/25] base-apt: pull base-apt population to the front of the build chain Henning Schild
2020-01-21  8:55   ` vijai kumar
2020-01-21  9:54     ` Henning Schild
2020-01-21 13:01       ` vijai kumar
2020-01-27  7:32         ` Henning Schild
2019-12-16 11:50 ` [PATCHv2 24/25] meta: base-apt: remove isar-apt check from population Henning Schild
2019-12-16 11:50 ` [PATCHv2 25/25] meta: deb-dl-dir: only export newly downloaded files Henning Schild
2019-12-19  9:21 ` [PATCHv2 00/25] base-apt-rework Henning Schild

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CALLGG_KKXSxj3SpjHiJL5B99vEOm5W98FRWz7a7tOheQrFrcow@mail.gmail.com \
    --to=vijaikumar.kanagarajan@gmail.com \
    --cc=henning.schild@siemens.com \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox