From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6458972999552860160 X-Received: by 10.28.19.138 with SMTP id 132mr71649wmt.3.1503934208212; Mon, 28 Aug 2017 08:30:08 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.86.73 with SMTP id k70ls58183ljb.5.gmail; Mon, 28 Aug 2017 08:30:07 -0700 (PDT) X-Google-Smtp-Source: ADKCNb51h0s2e/XCgGTT2p2x52zh3TCyV2jNqHz3d5Szqu6FzavQJDNhJpitpwLWb3DV/zyKvS1M X-Received: by 10.46.9.209 with SMTP id 200mr93644ljj.36.1503934207723; Mon, 28 Aug 2017 08:30:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503934207; cv=none; d=google.com; s=arc-20160816; b=jWNo9rAPE6p3SLBqfQBxU+EBXCftMCg4zWeIkqi15FLCV30OseBf2ZULYMqsTozS7r +/Q7zlFJUndRbbRPavnc2U0X8fWO+9q06CdT6cix9+vL4A85bRUI906Xuu/cAtkCC3f9 +osJ4sP0v7hEUU34jw9FOGO1YxcGSQNijb/fPFHoAupQG9FsW+A59d7rWTEivmJW3TgL g9EjR2IQhQkfhXnJ5Mbb2ZrDVaTETTuPdWp3s/1nytzXs9L0qDIdyRNarfsM3hVwb5cp FVHvSV9JURd1D538yiYdrHS8HmyMg+3jaGvE/H0rockmPYWQpRJHkVPpjDWidgYtJ5ry B2sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=NtKTtlFC0N4qr5TRm2ySvfRD2OadF6zW48ZkCuTHByc=; b=RN4mbVyNyH8osZCNiIqt/H9QvE37GnQuC774JIM+79UtBhB7N+TRyEJH0OKDbS9Gkb yICg5+v8KM4fRc11Xh54EAVoCot6PFkx5QFQo25gtyjA+FYNpjXRgpHPIJAX3g2Ar97c nB2eRk/0OysTtBi+mkveJg7xExCRVVcvn592SpEIYyEb+DH67+ADPJ9hOTrnBoU9Co8A LdC7/9XZY3yzTO8veeUAqHbJq64xtCY8V2E9n/V9FuW7TQWPNTYuduIKzJuQYTyuZjIr jIfM/Mi1Re7tJJRkzqX5hJvJrO6c8Toy9BHBWDZthjwTh6pecjyJj/BInkuI5xDA6pfM j36g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id v195si64742wmf.0.2017.08.28.08.30.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Aug 2017 08:30:07 -0700 (PDT) Received-SPF: neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Received: from mail3.siemens.de (mail3.siemens.de [139.25.208.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id v7SFU7jb023322 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 28 Aug 2017 17:30:07 +0200 Received: from md1em3qc ([139.25.68.40]) by mail3.siemens.de (8.15.2/8.15.2) with ESMTP id v7SFU7F4015035; Mon, 28 Aug 2017 17:30:07 +0200 Date: Mon, 28 Aug 2017 17:30:10 +0200 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH 3/6] classes/dpkg: Split install for cache Message-ID: <20170828173010.0a157fed@md1em3qc> In-Reply-To: <20170827151339.12806-4-asmirnov@ilbers.de> References: <20170827151339.12806-1-asmirnov@ilbers.de> <20170827151339.12806-4-asmirnov@ilbers.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 84Va4c3pWGir This is a fixup of 1/6. No need for another patch, just fix 1of6. Henning Am Sun, 27 Aug 2017 18:13:36 +0300 schrieb Alexander Smirnov : > Split install function into two parts: > - install_to_cache: if caching is enabled > - install_to_deploy: if caching is disabled > > This patch brings flexibility to the implementation and makes it > possible to move all the caching implementation code to dedicated > class. The magic behavior depending on the value of DEBCACHE_ENABLED > is now transparent in the bitbake pipeline (can be inspected via > "bitbake -g"). > > Signed-off-by: Alexander Smirnov > --- > meta/classes/dpkg.bbclass | 66 > ++++++++++++++++++++++++++-------------------- > meta/classes/image.bbclass | 2 +- 2 files changed, 38 insertions(+), > 30 deletions(-) > > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > index b1e201d..118ba2f 100644 > --- a/meta/classes/dpkg.bbclass > +++ b/meta/classes/dpkg.bbclass > @@ -56,46 +56,53 @@ do_build() { > } > > # Install package to dedicated deploy directory > -do_binary_deb_install() { > +do_install_to_deploy() { > + # Deb caching is disabled, simply copy all binary packages to > the deploy > + # directory > + install -m 644 "${BUILDROOT}"/*.deb "${DEPLOY_DIR_DEB}/" > +} > + > +addtask install_to_deploy after do_build > +do_install_to_deploy[dirs] = "${DEPLOY_DIR_DEB}" > +do_install_to_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_install_to_deploy[noexec] = "1" > + > +# Install package to dedicated apt repo > +do_install_to_cache() { > readonly DIR_CACHE="${DEBCACHEDIR}/${DISTRO}" > readonly DIR_DB="${DEBDBDIR}/${DISTRO}" > > - if [ "${DEBCACHE_ENABLED}" != "0" ]; then > - # If `bitbake` is running for the first time, the cache > doesn't exist > - # yet and needs to be configured using a `distributions` > file. > - # A template stored in the layer directory is pre-processed > to > - # generate the configuration file, which is then placed in > the > - # appropriate directory. > - if [ ! -e "${DIR_CACHE}/conf/distributions" ]; then > - mkdir -p "${DIR_CACHE}/conf" > - sed -e "s#{DISTRO_NAME}#${DEBDISTRONAME}#g" \ > - "${DEBFILESDIR}/distributions.in" \ > - > "${DIR_CACHE}/conf/distributions" > - fi > - > - # Add binary and source packages to the deb cache > - # If the cache doesn't exist yet, it will be created using > the > - # `distributions` file generated above. > - ls -1 "${BUILDROOT}"/*.deb "${BUILDROOT}"/*.dsc | while read > -r p; do > - reprepro --waitforlock 3 -b "${DIR_CACHE}" --dbdir > "${DIR_DB}" \ > - -C main "include${p##*.}" "${DEBDISTRONAME}" "${p}" > - done > - else > - # Deb caching is disabled, simply copy all binary packages > to the > - # deploy directory > - mkdir -p "${DEPLOY_DIR_DEB}" > - install -m 644 "${BUILDROOT}"/*.deb "${DEPLOY_DIR_DEB}/" > + # If `bitbake` is running for the first time, the cache doesn't > exist > + # yet and needs to be configured using a `distributions` file. > + # A template stored in the layer directory is pre-processed to > + # generate the configuration file, which is then placed in the > + # appropriate directory. > + if [ ! -e "${DIR_CACHE}/conf/distributions" ]; then > + mkdir -p "${DIR_CACHE}/conf" > + sed -e "s#{DISTRO_NAME}#${DEBDISTRONAME}#g" \ > + "${DEBFILESDIR}/distributions.in" \ > + > "${DIR_CACHE}/conf/distributions" > fi > + > + # Add binary and source packages to the deb cache > + # If the cache doesn't exist yet, it will be created using the > + # `distributions` file generated above. > + ls -1 "${BUILDROOT}"/*.deb "${BUILDROOT}"/*.dsc | while read -r > p; do > + reprepro --waitforlock 3 -b "${DIR_CACHE}" --dbdir > "${DIR_DB}" \ > + -C main "include${p##*.}" "${DEBDISTRONAME}" "${p}" > + done > } > > -addtask binary_deb_install after do_build > -do_binary_deb_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +addtask install_to_cache after do_build > +do_install_to_cache[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_install_to_cache[noexec] = "1" > > # Deb caching lambda run during the parsing phase that checks > whether the # current package has to be rebuilt, or taken from the > cache python __anonymous () { > if d.getVar("DEBCACHE_ENABLED", True) == "0": > - # Deb caching is disabled, do nothing > + # Deb caching is disabled > + d.delVarFlag('do_install_to_deploy', 'noexec') > return True > > PN = d.getVar("PN", True) > @@ -108,6 +115,7 @@ python __anonymous () { > path_cache = os.path.join(DEBCACHEDIR, DISTRO) > path_databases = os.path.join(DEBDBDIR, DISTRO) > path_distributions = os.path.join(path_cache, "conf", > "distributions") > + d.delVarFlag('do_install_to_cache', 'noexec') > > # The distributions file is needed by `reprepro` to know what > types # of packages are supported, what the distribution name is, etc. > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index c2ff453..6b1b5eb 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -46,4 +46,4 @@ do_populate() { > } > > addtask populate before do_build > -do_populate[deptask] = "do_install" > +do_populate[deptask] = "do_install_to_cache do_install_to_deploy"