From: Jan Kiszka <jan.kiszka@siemens.com>
To: isar-users <isar-users@googlegroups.com>
Subject: [PATCH v6 12/27] Detect false sharing of recipes
Date: Tue, 24 Sep 2019 20:02:26 +0200 [thread overview]
Message-ID: <c06d4af3-b40a-e3e9-85d6-7121df88a3b8@siemens.com> (raw)
In-Reply-To: <5a2e329b881ec0b392d0f1abd116f1deeee0f66f.1569176231.git.jan.kiszka@siemens.com>
From: Jan Kiszka <jan.kiszka@siemens.com>
When a task is run more than once per build, this indicates a bug,
usually (though not only) related to incorrect multiconfig use. Such
bugs are hard to find in practice, so they could benefit from some
tooling.
This adds a very simple but effective check for such re-run scenarios
by putting a "once.<taskname>" stamp file into each workdir during a
build. If such a file already exists, an error is thrown. The workdirs
are purged after the parsing phase on the beginning of each build so
that no false positives are raised. The task stamp is also deleted after
failures so that no (serialized) retries can cause warnings.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
Changes in v6:
- avoid warnings after build failures of shared tasks
- code refactoring
meta/classes/isar-events.bbclass | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass
index 62eb25d..cad1d57 100644
--- a/meta/classes/isar-events.bbclass
+++ b/meta/classes/isar-events.bbclass
@@ -4,6 +4,40 @@
# Copyright (C) 2015-2017 ilbers GmbH
# Copyright (c) Siemens AG, 2018
+addhandler parse_completed
+
+python parse_completed() {
+ bb.utils.remove(d.getVar('TMPDIR') + "/work/*/*/*/temp/once.*")
+}
+parse_completed[eventmask] = "bb.event.ParseCompleted"
+
+def task_once_stamp(d):
+ return "{temp}/once.{task}".format(temp=d.getVar('T'),
+ task=d.getVar('BB_RUNTASK'))
+
+addhandler task_started
+
+python task_started() {
+ try:
+ f = open(task_once_stamp(d), "x")
+ f.close()
+ except FileExistsError:
+ bb.error("Detect multiple executions of %s in %s" %
+ (d.getVar('BB_RUNTASK'), d.getVar('WORKDIR')))
+ bb.error("Rerun a clean build with empty STAMPCLEAN " \
+ "and compare the sigdata files")
+}
+task_started[eventmask] = "bb.build.TaskStarted"
+
+addhandler task_failed
+
+python task_failed() {
+ # Avoid false positives if a second target depends on this task and retries
+ # the execution after the first failure.
+ os.remove(task_once_stamp(d))
+}
+task_failed[eventmask] = "bb.build.TaskFailed"
+
addhandler build_completed
python build_completed() {
--
2.16.4
next prev parent reply other threads:[~2019-09-24 18:02 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-22 18:16 [PATCH v5 00/27] Pending patches, fix for isar-bootstrap, devshell Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 01/27] buildchroot: Properly deploy the build result Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 02/27] sdk: Use clean-package-cache rather than open-coded cleanup Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 03/27] sdk: Deploy earlier Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 04/27] Remove unneeded PF tweakings Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 05/27] image: Stop changing PF, tune WORKDIR and STAMP directly Jan Kiszka
2019-09-25 12:47 ` [PATCH v6 " Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 06/27] image: Make WORKDIR and STAMPs IMAGE_TYPE-specific Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 07/27] Adjust STAMP variables to use PN, rather than PF Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 08/27] Align WORKDIR structure with OE Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 09/27] isar-bootstrap: Unshare host bootstraps across different target distros Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 10/27] isar-bootstrap: Clean up and fix rebuild Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 11/27] gitlab-ci: Keep logs as artifacts on failure Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 12/27] Detect false sharing of recipes Jan Kiszka
2019-09-24 18:02 ` Jan Kiszka [this message]
2019-09-29 14:57 ` [PATCH v6 " Baurzhan Ismagulov
2019-09-30 6:00 ` Jan Kiszka
2019-09-30 9:56 ` Baurzhan Ismagulov
2019-09-30 10:14 ` Jan Kiszka
2019-09-30 14:53 ` Baurzhan Ismagulov
2019-09-30 15:26 ` Jan Kiszka
2019-09-30 15:39 ` Baurzhan Ismagulov
2019-09-30 16:34 ` Jan Kiszka
2019-09-30 17:20 ` Baurzhan Ismagulov
2019-09-30 17:26 ` Jan Kiszka
2019-09-30 20:29 ` Baurzhan Ismagulov
2019-09-22 18:16 ` [PATCH v5 13/27] dpkg-base: Move do_deploy_deb before do_build Jan Kiszka
2019-10-02 7:09 ` Baurzhan Ismagulov
2019-09-22 18:16 ` [PATCH v5 14/27] base-apt: Move do_cache_config " Jan Kiszka
2019-09-22 18:16 ` [PATCH v5 15/27] wic-img: Use private tmp dir Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 16/27] rootfs: Take isar-apt shared lock during repository operations Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 17/27] Use deb.debian.org mirrors Jan Kiszka
2019-09-23 10:03 ` Baurzhan Ismagulov
2019-09-23 10:09 ` Jan Kiszka
2019-10-02 11:13 ` Baurzhan Ismagulov
2019-09-22 18:17 ` [PATCH v5 18/27] linux-custom: Control linux-libc-dev deployment manually Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 19/27] dpkg-base: Wait for umount to succeed Jan Kiszka
2019-09-30 11:07 ` Baurzhan Ismagulov
2019-09-30 12:22 ` Jan Kiszka
2019-09-30 13:33 ` Baurzhan Ismagulov
2019-10-07 15:48 ` Baurzhan Ismagulov
2019-09-22 18:17 ` [PATCH v5 20/27] dpkg-base: Permit multiple clean runs Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 21/27] buildchroot: Factor out common packages Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 22/27] buildchroot: Do not build cross when there are no arch-specific outputs Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 23/27] Add git-buildpackage support Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 24/27] dbpg-raw: Only list top-level files in <pn>.install Jan Kiszka
2019-09-23 9:52 ` Baurzhan Ismagulov
2019-09-23 10:29 ` Jan Kiszka
2019-09-23 10:30 ` Baurzhan Ismagulov
2019-09-22 18:17 ` [PATCH v5 25/27] Add OE class and library elements for terminal support Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 26/27] dpkg-base: Add devshell target Jan Kiszka
2019-09-22 18:17 ` [PATCH v5 27/27] linux-custom: Add libncurses-dev to KBUILD_DEPENDS Jan Kiszka
2019-09-23 6:33 ` [PATCH v5 00/27] Pending patches, fix for isar-bootstrap, devshell Baurzhan Ismagulov
2019-09-23 7:33 ` Jan Kiszka
2019-09-23 7:50 ` Baurzhan Ismagulov
2019-09-23 9:50 ` Baurzhan Ismagulov
2019-09-23 10:07 ` Jan Kiszka
2019-09-23 10:23 ` Baurzhan Ismagulov
2019-09-23 10:28 ` Jan Kiszka
2019-09-25 8:41 ` Baurzhan Ismagulov
2019-09-25 8:51 ` Jan Kiszka
2019-09-25 9:22 ` Baurzhan Ismagulov
2019-09-25 9:26 ` Jan Kiszka
2019-09-25 9:35 ` Baurzhan Ismagulov
2019-09-25 9:35 ` Jan Kiszka
2019-09-25 10:52 ` Baurzhan Ismagulov
2019-09-25 11:05 ` Jan Kiszka
2019-09-25 12:08 ` Baurzhan Ismagulov
2019-09-25 12:29 ` Jan Kiszka
2019-09-25 12:44 ` Baurzhan Ismagulov
2019-09-23 14:24 ` Henning Schild
2019-09-28 15:46 ` Baurzhan Ismagulov
2019-09-25 10:23 ` Henning Schild
2019-09-25 11:41 ` Jan Kiszka
2019-09-30 9:29 ` Baurzhan Ismagulov
2019-09-30 10:02 ` Jan Kiszka
2019-09-30 13:00 ` Baurzhan Ismagulov
2019-10-01 14:47 ` Baurzhan Ismagulov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c06d4af3-b40a-e3e9-85d6-7121df88a3b8@siemens.com \
--to=jan.kiszka@siemens.com \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox