From: "'Moessbauer, Felix' via isar-users" <isar-users@googlegroups.com>
To: "Kowalsky, Clara" <clara.kowalsky@siemens.com>,
"isar-users@googlegroups.com" <isar-users@googlegroups.com>
Subject: Re: [PATCH 1/8] base: replace BitBake's outdated persist_data
Date: Tue, 10 Nov 2026 11:18:45 +0000 [thread overview]
Message-ID: <ecc2fbf80cc99b3bbc0956f4d65956b650c7d888.camel@siemens.com> (raw)
In-Reply-To: <20261109091802.202521-2-clara.kowalsky@siemens.com>
On Mon, 2026-11-09 at 10:17 +0100, 'Clara Kowalsky' via isar-users
wrote:
> Isar uses a persistent UUID to distinguish build-specific resources,
> including sbuild chroot names.
>
The only user I can find is schroot. This will hopefully go away rather
sooner than later. Given that, I'm wondering if we really need the unit
test.
Anyways, the change is fine.
Felix
> The UUID must remain stable throughout a
> BitBake run, so parallel builds using separate PID namespaces do not
> clash on external mountpoints. It must also be shared across
> multiconfigs and cleared when the build completes.
> Isar currently stores the UUID through bb.persist_data.persist, which
> is removed in BitBake v2.9.2 commit 681a7516 ("persist_data: Remove
> it") from October 2024.
>
> Persist the UUID in a file under PERSISTENT_DIR instead. Use BitBake's
> file-locking helper when reading, creating and clearing it to ensure
> concurrent processes share the same UUID. Clear it on the default
> BuildCompleted event, so the next run receives a fresh UUID.
>
> Add unittests for UUID reuse across calls and multiconfigs, cleanup and
> concurrent access.
>
> Signed-off-by: Clara Kowalsky <clara.kowalsky@siemens.com>
> ---
> meta/classes-global/base.bbclass | 25 +++++--
> meta/classes-global/isar-events.bbclass | 2 +-
> testsuite/unittests/test_build_uuid.py | 89 +++++++++++++++++++++++++
> 3 files changed, 109 insertions(+), 7 deletions(-)
> create mode 100644 testsuite/unittests/test_build_uuid.py
>
> diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
> index 3d5d1c12..c61e59b4 100644
> --- a/meta/classes-global/base.bbclass
> +++ b/meta/classes-global/base.bbclass
> @@ -341,12 +341,25 @@ def base_set_filespath(path, d):
> def calculate_build_uuid(d):
> import uuid
>
> - uuid_data = bb.persist_data.persist('BB_ISAR_UUID_DATA', d)
> - if "uuid" not in uuid_data or not uuid_data["uuid"]:
> - # Generate new UUID
> - uuid_data["uuid"] = str(uuid.uuid4())
> -
> - return uuid_data["uuid"]
> + uuid_file = os.path.join(d.getVar('PERSISTENT_DIR'), 'isar-build-uuid')
> + with bb.utils.fileslocked([uuid_file + '.lock']):
> + try:
> + with open(uuid_file) as file:
> + build_uuid = file.read().strip()
> + except FileNotFoundError:
> + build_uuid = ''
> +
> + if not build_uuid:
> + build_uuid = str(uuid.uuid4())
> + with open(uuid_file, 'w') as file:
> + file.write(build_uuid + '\n')
> +
> + return build_uuid
> +
> +def clear_build_uuid(d):
> + uuid_file = os.path.join(d.getVar('PERSISTENT_DIR'), 'isar-build-uuid')
> + with bb.utils.fileslocked([uuid_file + '.lock']):
> + bb.utils.remove(uuid_file)
>
> # Unique ID for this build, used to avoid name clashes on external mountpoints
> # When running parallel builds in different PID namespaces
> diff --git a/meta/classes-global/isar-events.bbclass b/meta/classes-global/isar-events.bbclass
> index 15bfdb99..cfe6d7dd 100644
> --- a/meta/classes-global/isar-events.bbclass
> +++ b/meta/classes-global/isar-events.bbclass
> @@ -85,7 +85,7 @@ python build_completed() {
> bb.error(str(e))
>
> # Cleanup build UUID, the next bitbake run will generate new one
> - bb.persist_data.persist('BB_ISAR_UUID_DATA', d).clear()
> + bb.utils.exec_flat_python_func('clear_build_uuid', d)
> }
>
> build_completed[eventmask] = "bb.event.BuildCompleted"
> diff --git a/testsuite/unittests/test_build_uuid.py b/testsuite/unittests/test_build_uuid.py
> new file mode 100644
> index 00000000..374cce64
> --- /dev/null
> +++ b/testsuite/unittests/test_build_uuid.py
> @@ -0,0 +1,89 @@
> +# This software is a part of Isar.
> +# Copyright (C) Siemens AG, 2026
> +#
> +# SPDX-License-Identifier: MIT
> +
> +import multiprocessing
> +import os
> +import sys
> +import tempfile
> +import unittest
> +import uuid
> +from pathlib import Path
> +
> +root = Path(__file__).resolve().parents[2]
> +sys.path.extend([
> + str(root / 'bitbake/lib'),
> + str(root / 'meta/lib'),
> +])
> +
> +import bb
> +import bb.data
> +import bb.utils
> +from bb.parse import handle
> +
> +
> +data = bb.data.init()
> +data.setVar('BBPATH', f'{root}/meta:{root}/bitbake')
> +data.setVar('__bbclasstype', 'global')
> +parsed = handle(str(root / 'meta/classes-global/base.bbclass'), data)
> +namespace = {'bb': bb, 'os': os}
> +exec(parsed['calculate_build_uuid'], namespace)
> +exec(parsed['clear_build_uuid'], namespace)
> +calculate_build_uuid = namespace['calculate_build_uuid']
> +clear_build_uuid = namespace['clear_build_uuid']
> +
> +
> +def calculate_in_worker(persistent_dir, start, results):
> + data = bb.data.init()
> + data.setVar('PERSISTENT_DIR', persistent_dir)
> + start.wait()
> + results.put(calculate_build_uuid(data))
> +
> +
> +class TestBuildUUID(unittest.TestCase):
> + def setUp(self):
> + directory = tempfile.TemporaryDirectory()
> + self.addCleanup(directory.cleanup)
> + self.persistent_dir = os.path.join(directory.name, 'cache')
> + self.data = bb.data.init()
> + self.data.setVar('PERSISTENT_DIR', self.persistent_dir)
> +
> + def test_uuid_is_reused_across_calls_and_multiconfigs(self):
> + build_uuid = calculate_build_uuid(self.data)
> + self.assertEqual(str(uuid.UUID(build_uuid)), build_uuid)
> + self.assertEqual(calculate_build_uuid(self.data), build_uuid)
> + other_config = self.data.createCopy()
> + other_config.setVar('BB_CURRENT_MC', 'other')
> + self.assertEqual(calculate_build_uuid(other_config), build_uuid)
> + clear_build_uuid(self.data)
> + self.assertNotEqual(calculate_build_uuid(self.data), build_uuid)
> +
> + def test_concurrent_workers_share_uuid(self):
> + context = multiprocessing.get_context('fork')
> + start = context.Event()
> + results = context.Queue()
> + workers = [
> + context.Process(
> + target=calculate_in_worker,
> + args=(self.persistent_dir, start, results),
> + )
> + for _ in range(8)
> + ]
> + try:
> + for worker in workers:
> + worker.start()
> + start.set()
> + values = [results.get(timeout=15) for worker in workers]
> + for worker in workers:
> + worker.join(timeout=15)
> + self.assertEqual(worker.exitcode, 0)
> + self.assertEqual(len(set(values)), 1)
> + self.assertEqual(values[0], calculate_build_uuid(self.data))
> + finally:
> + for worker in workers:
> + if worker.is_alive():
> + worker.terminate()
> + worker.join(timeout=15)
> + results.close()
> + results.join_thread()
> --
> 2.56.0
>
> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/20261109091802.202521-2-clara.kowalsky%40siemens.com.
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/ecc2fbf80cc99b3bbc0956f4d65956b650c7d888.camel%40siemens.com.
next prev parent reply other threads:[~2026-11-10 11:18 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-09 9:17 [PATCH 0/8] Update BitBake in Isar to v2.20.0 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 1/8] base: replace BitBake's outdated persist_data 'Clara Kowalsky' via isar-users
2026-11-10 11:18 ` 'Moessbauer, Felix' via isar-users [this message]
2026-11-11 14:37 ` 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 2/8] isar-events: preserve cleanup for default multiconfig 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 3/8] isar-events: close /proc/mounts after cleanup 'Clara Kowalsky' via isar-users
2026-11-10 11:22 ` 'Moessbauer, Felix' via isar-users
2026-11-11 14:38 ` 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 4/8] meta: migrate fetcher references to bb.fetch 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 5/8] wic: run the Python driver outside the imaging chroot 'Clara Kowalsky' via isar-users
2026-11-10 11:39 ` 'Moessbauer, Felix' via isar-users
2026-11-11 14:39 ` 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 6/8] bitbake.conf: explicitly exclude export and func from signatures 'Clara Kowalsky' via isar-users
2026-11-09 9:18 ` [PATCH 7/8] meta: add missing whitespace around assignment operators 'Clara Kowalsky' via isar-users
2026-11-09 9:18 ` [PATCH 8/8] bitbake: update to v2.20.0 'Clara Kowalsky' via isar-users
2026-11-13 14:41 ` [PATCH 0/8] Update BitBake in Isar " Anton Mikanovich
2026-11-13 17:19 ` 'Clara Kowalsky' via isar-users
2026-11-16 8:46 ` 'Moessbauer, Felix' via isar-users
2026-11-16 10:44 ` 'Clara Kowalsky' via isar-users
2026-11-16 12:12 ` 'Moessbauer, Felix' via isar-users
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ecc2fbf80cc99b3bbc0956f4d65956b650c7d888.camel@siemens.com \
--to=isar-users@googlegroups.com \
--cc=clara.kowalsky@siemens.com \
--cc=felix.moessbauer@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox