From: Claudius Heine <claudius.heine.ext@siemens.com>
To: "[ext] Jan Kiszka" <jan.kiszka@siemens.com>,
isar-users <isar-users@googlegroups.com>
Subject: Re: [PATCH 04/26] Rework STAMP layout and cleaning
Date: Mon, 26 Aug 2019 10:30:46 +0200 [thread overview]
Message-ID: <0b8426ae-b9e5-9c2c-62ad-302d0f85874e@siemens.com> (raw)
In-Reply-To: <e840208e27ea371b4a518d76589fd990918a607a.1566800787.git.jan.kiszka@siemens.com>
Hi Jan,
On 26/08/2019 08.26, [ext] Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Follow OE more closely by aligning the stamps folder structure to
> workdir. This may resolve false sharing of stamps between recipes of
> common PF but different DISTRO/DISTRO_ARCH without the otherwise
> required stamp-extra-info. That extra-info can now be removed.
>
> As long as recipes mess with PF, we can't use OE's pattern as-is which
> is PN-based.
>
> Furthermore add proper STAMPCLEAN. This ensures that stamp files are
> cleaned when a task is rebuilt. Not providing this pattern likely caused
> a lot of failures when doing partial rebuilds.
>
> As we are at it, switch do_clean to STAMPCLEAN as well.
>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> meta/classes/base.bbclass | 5 ++---
> meta/classes/dpkg-base.bbclass | 6 ------
> meta/classes/dpkg-raw.bbclass | 1 -
> meta/classes/dpkg.bbclass | 1 -
> meta/classes/patch.bbclass | 2 --
> meta/classes/template.bbclass | 1 -
> meta/conf/bitbake.conf | 4 +++-
> meta/recipes-kernel/linux/linux-custom.inc | 1 -
> 8 files changed, 5 insertions(+), 16 deletions(-)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index ca80bf8..1d2852a 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -163,7 +163,6 @@ python do_fetch() {
> addtask fetch before do_build
>
> do_unpack[dirs] = "${WORKDIR}"
> -do_unpack[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>
> # Unpack package and put it into working directory
> python do_unpack() {
> @@ -206,8 +205,8 @@ python do_clean() {
> workdir = d.expand("${WORKDIR}")
> subprocess.check_call(["sudo", "rm", "-rf", workdir])
>
> - stamppath = bb.data.expand(d.getVar('STAMP', False), d)
> - stampdirs = glob.glob(stamppath + ".*")
> + stampclean = bb.data.expand(d.getVar('STAMPCLEAN', False), d)
> + stampdirs = glob.glob(stampclean)
> subprocess.check_call(["sudo", "rm", "-rf"] + stampdirs)
> }
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 39c8acb..2d26418 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -16,7 +16,6 @@ do_adjust_git() {
> }
>
> addtask adjust_git after do_unpack before do_patch
> -do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>
> inherit patch
> addtask patch after do_adjust_git before do_build
> @@ -42,7 +41,6 @@ do_apt_fetch() {
>
> addtask apt_fetch after do_unpack before do_patch
> do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
> -do_apt_fetch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>
> def get_package_srcdir(d):
> s = os.path.abspath(d.getVar("S", True))
> @@ -60,15 +58,12 @@ def get_package_srcdir(d):
> PP = "/home/builder/${PN}"
> PPS ?= "${@get_package_srcdir(d)}"
>
> -do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> -
> # Empty do_prepare_build() implementation, to be overwritten if needed
> do_prepare_build() {
> true
> }
>
> addtask prepare_build after do_patch do_transform_template before do_build
> -do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> # If Isar recipes depend on each other, they typically need the package
> # deployed to isar-apt
> do_prepare_build[deptask] = "do_deploy_deb"
> @@ -132,7 +127,6 @@ do_deploy_deb() {
> }
>
> addtask deploy_deb after do_build
> -do_deploy_deb[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> do_deploy_deb[depends] = "isar-apt:do_cache_config"
> do_deploy_deb[dirs] = "${S}"
> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> index ea03ea4..f8dfb2d 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -16,7 +16,6 @@ do_install() {
> }
>
> do_install[cleandirs] = "${D}"
> -do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> addtask install after do_unpack before do_prepare_build
>
> do_prepare_build[cleandirs] += "${D}/debian"
> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
> index 997f2f3..dd123c3 100644
> --- a/meta/classes/dpkg.bbclass
> +++ b/meta/classes/dpkg.bbclass
> @@ -14,7 +14,6 @@ do_install_builddeps() {
> addtask install_builddeps after do_prepare_build before do_build
> # apt and reprepro may not run in parallel, acquire the Isar lock
> do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
> -do_install_builddeps[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>
> # Build package from sources using build script
> dpkg_runbuild() {
> diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
> index 2662702..c5ba463 100644
> --- a/meta/classes/patch.bbclass
> +++ b/meta/classes/patch.bbclass
> @@ -78,5 +78,3 @@ python do_patch() {
> except bb.fetch2.BBFetchException as e:
> raise bb.build.FuncFailed(e)
> }
> -
> -do_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass
> index f3b5f80..fb9d118 100644
> --- a/meta/classes/template.bbclass
> +++ b/meta/classes/template.bbclass
> @@ -7,7 +7,6 @@ TEMPLATE_FILES ?= ""
> TEMPLATE_VARS ?= "PN PV DESCRIPTION HOMEPAGE MAINTAINER DISTRO_ARCH"
>
> do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}"
> -do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> python do_transform_template() {
> import subprocess, contextlib
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 3d18223..c41f294 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -35,7 +35,9 @@ PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0
> S = "${WORKDIR}/${P}"
> AUTOREV = "${@bb.fetch2.get_autorev(d)}"
> SRC_URI = "file://${FILE}"
> -STAMP = "${TMPDIR}/stamps/${PF}"
> +STAMPS_DIR ?= "${TMPDIR}/stamps"
> +STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PF}/${PV}-${PR}"
Can you explain the issue of the PN/PF usage a bit more. I did not
understand:
> As long as recipes mess with PF, we can't use OE's pattern as-is which
> is PN-based.
The recipes that mess with PF are: buildchroot-*.bb sdkchroot.bb
isar-bootstrap-*.bb and image.bbclass. All of which do not really have a
version.
So I would think that:
STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}/${PF}"
STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}/*-*"
should work. Or am I missing something?
regards,
Claudius
> +STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PF}/*-*"
> SVNDIR = "${DL_DIR}/svn"
> T = "${WORKDIR}/temp"
> TMPDIR = "${TOPDIR}/tmp"
> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
> index ee5f20c..12d8d9c 100644
> --- a/meta/recipes-kernel/linux/linux-custom.inc
> +++ b/meta/recipes-kernel/linux/linux-custom.inc
> @@ -44,7 +44,6 @@ do_install_builddeps() {
> addtask install_builddeps after do_prepare_build before do_build
> # apt and reprepro may not run in parallel, acquire the Isar lock
> do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
> -do_install_builddeps[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>
> dpkg_runbuild() {
> chmod +x ${WORKDIR}/build-kernel.sh
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
next prev parent reply other threads:[~2019-08-26 8:30 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-26 6:26 [PATCH 00/26] Tons of fixes and cleanups Jan Kiszka
2019-08-26 6:26 ` [PATCH 01/26] isar-bootstrap: Consider DISTRO_BOOTSTRAP_KEYS for gnupg installation Jan Kiszka
2019-08-26 6:26 ` [PATCH 02/26] isar-bootstrap: Fix keyring generation Jan Kiszka
2019-09-27 12:03 ` Baurzhan Ismagulov
2019-08-26 6:26 ` [PATCH 03/26] isar-bootstrap: Install DISTRO_BOOTSTRAP_KEYS Jan Kiszka
2019-08-26 6:26 ` [PATCH 04/26] Rework STAMP layout and cleaning Jan Kiszka
2019-08-26 8:30 ` Claudius Heine [this message]
2019-08-26 8:38 ` Jan Kiszka
2019-08-26 6:26 ` [PATCH 05/26] dpkg: Fix shared-locking of build task Jan Kiszka
2019-08-26 6:38 ` Cedric Hombourger
2019-08-26 8:30 ` Claudius Heine
2019-08-26 8:38 ` Jan Kiszka
2019-08-26 6:26 ` [PATCH 06/26] dpkg-base: Free do_build task Jan Kiszka
2019-08-26 6:26 ` [PATCH 07/26] base: Clean up build task Jan Kiszka
2019-08-26 8:30 ` Henning Schild
2019-08-26 8:40 ` Jan Kiszka
2019-08-26 6:26 ` [PATCH 08/26] isar-events: Do not start cleanup early Jan Kiszka
2019-08-26 6:26 ` [PATCH 09/26] meta-isar: Move bananapi to buster, remove bogus example Jan Kiszka
2019-09-04 6:53 ` [PATCH v2 " Jan Kiszka
2019-08-26 6:26 ` [PATCH 10/26] bitbake.conf: Reorder PV definition Jan Kiszka
2019-08-26 6:26 ` [PATCH 11/26] example-module: Fix build with mipsel target Jan Kiszka
2019-08-26 6:26 ` [PATCH 12/26] buildchroot: Ignore ISAR_CROSS_COMPILE for i386 on stretch Jan Kiszka
2019-08-26 6:26 ` [PATCH 13/26] buildchroot: Fix downgrades when installing dependencies Jan Kiszka
2019-08-26 6:26 ` [PATCH 14/26] buildchroot: Properly deploy the build result Jan Kiszka
2019-08-26 6:26 ` [PATCH 15/26] sdk: Use clean-package-cache rather than open-coded cleanup Jan Kiszka
2019-08-26 6:26 ` [PATCH 16/26] sdk: Deploy earlier Jan Kiszka
2019-08-26 6:26 ` [PATCH 17/26] Remove unneeded PF tweakings Jan Kiszka
2019-08-26 6:26 ` [PATCH 18/26] image: Stop changing PF, tune WORKDIR and STAMP directly Jan Kiszka
2019-08-26 8:35 ` Henning Schild
2019-08-26 8:40 ` Jan Kiszka
2019-08-26 6:26 ` [PATCH 19/26] image: Make WORKDIR and STAMPs IMAGE_TYPE-specific Jan Kiszka
2019-08-26 6:26 ` [PATCH 20/26] Adjust STAMP variables to use PN, rather than PF Jan Kiszka
2019-08-26 6:26 ` [PATCH 21/26] Align WORKDIR structure with OE Jan Kiszka
2019-08-26 6:26 ` [PATCH 22/26] isar-bootstrap: Clean up and fix rebuild Jan Kiszka
2019-09-21 8:28 ` Jan Kiszka
2019-08-26 6:26 ` [PATCH 23/26] gitlab-ci: Keep logs as artifacts on failure Jan Kiszka
2019-08-26 6:26 ` [PATCH 24/26] Detect false sharing of recipes Jan Kiszka
2019-08-26 6:26 ` [PATCH 25/26] dpkg-base: Move do_deploy_deb before do_build Jan Kiszka
2019-08-28 7:32 ` Claudius Heine
2019-10-01 11:49 ` Baurzhan Ismagulov
2019-08-26 6:26 ` [PATCH 26/26] base-apt: Move do_cache_config " Jan Kiszka
2019-08-26 21:29 ` [PATCH 27/26] wic-img: Use private tmp dir Jan Kiszka
2019-08-27 15:00 ` [PATCH v2 " Jan Kiszka
2019-10-01 14:43 ` [PATCH 00/26] Tons of fixes and cleanups Baurzhan Ismagulov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b8426ae-b9e5-9c2c-62ad-302d0f85874e@siemens.com \
--to=claudius.heine.ext@siemens.com \
--cc=isar-users@googlegroups.com \
--cc=jan.kiszka@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox