From: "'Clara Kowalsky' via isar-users" <isar-users@googlegroups.com>
To: "Moessbauer,
Felix (FT RPD CPC OES-DE)" <felix.moessbauer@siemens.com>,
"isar-users@googlegroups.com" <isar-users@googlegroups.com>
Subject: Re: [PATCH 1/8] base: replace BitBake's outdated persist_data
Date: Wed, 11 Nov 2026 15:37:45 +0100 [thread overview]
Message-ID: <0d7832ad-aa96-49c0-8e2d-502c5be233be@siemens.com> (raw)
In-Reply-To: <ecc2fbf80cc99b3bbc0956f4d65956b650c7d888.camel@siemens.com>
On 11/10/26 12:18, Moessbauer, Felix (FT RPD CPC OES-DE) wrote:
> On Mon, 2026-11-09 at 10:17 +0100, 'Clara Kowalsky' via isar-users
> wrote:
>> Isar uses a persistent UUID to distinguish build-specific resources,
>> including sbuild chroot names.
>>
>
> The only user I can find is schroot. This will hopefully go away rather
> sooner than later. Given that, I'm wondering if we really need the unit
> test.
>
> Anyways, the change is fine.
>
> Felix
Yes, schroot is currently the only consumer. We can remove these bits
when schroot support is dropped.
Clara
>
>> The UUID must remain stable throughout a
>> BitBake run, so parallel builds using separate PID namespaces do not
>> clash on external mountpoints. It must also be shared across
>> multiconfigs and cleared when the build completes.
>> Isar currently stores the UUID through bb.persist_data.persist, which
>> is removed in BitBake v2.9.2 commit 681a7516 ("persist_data: Remove
>> it") from October 2024.
>>
>> Persist the UUID in a file under PERSISTENT_DIR instead. Use BitBake's
>> file-locking helper when reading, creating and clearing it to ensure
>> concurrent processes share the same UUID. Clear it on the default
>> BuildCompleted event, so the next run receives a fresh UUID.
>>
>> Add unittests for UUID reuse across calls and multiconfigs, cleanup and
>> concurrent access.
>>
>> Signed-off-by: Clara Kowalsky <clara.kowalsky@siemens.com>
>> ---
>> meta/classes-global/base.bbclass | 25 +++++--
>> meta/classes-global/isar-events.bbclass | 2 +-
>> testsuite/unittests/test_build_uuid.py | 89 +++++++++++++++++++++++++
>> 3 files changed, 109 insertions(+), 7 deletions(-)
>> create mode 100644 testsuite/unittests/test_build_uuid.py
>>
>> diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
>> index 3d5d1c12..c61e59b4 100644
>> --- a/meta/classes-global/base.bbclass
>> +++ b/meta/classes-global/base.bbclass
>> @@ -341,12 +341,25 @@ def base_set_filespath(path, d):
>> def calculate_build_uuid(d):
>> import uuid
>>
>> - uuid_data = bb.persist_data.persist('BB_ISAR_UUID_DATA', d)
>> - if "uuid" not in uuid_data or not uuid_data["uuid"]:
>> - # Generate new UUID
>> - uuid_data["uuid"] = str(uuid.uuid4())
>> -
>> - return uuid_data["uuid"]
>> + uuid_file = os.path.join(d.getVar('PERSISTENT_DIR'), 'isar-build-uuid')
>> + with bb.utils.fileslocked([uuid_file + '.lock']):
>> + try:
>> + with open(uuid_file) as file:
>> + build_uuid = file.read().strip()
>> + except FileNotFoundError:
>> + build_uuid = ''
>> +
>> + if not build_uuid:
>> + build_uuid = str(uuid.uuid4())
>> + with open(uuid_file, 'w') as file:
>> + file.write(build_uuid + '\n')
>> +
>> + return build_uuid
>> +
>> +def clear_build_uuid(d):
>> + uuid_file = os.path.join(d.getVar('PERSISTENT_DIR'), 'isar-build-uuid')
>> + with bb.utils.fileslocked([uuid_file + '.lock']):
>> + bb.utils.remove(uuid_file)
>>
>> # Unique ID for this build, used to avoid name clashes on external mountpoints
>> # When running parallel builds in different PID namespaces
>> diff --git a/meta/classes-global/isar-events.bbclass b/meta/classes-global/isar-events.bbclass
>> index 15bfdb99..cfe6d7dd 100644
>> --- a/meta/classes-global/isar-events.bbclass
>> +++ b/meta/classes-global/isar-events.bbclass
>> @@ -85,7 +85,7 @@ python build_completed() {
>> bb.error(str(e))
>>
>> # Cleanup build UUID, the next bitbake run will generate new one
>> - bb.persist_data.persist('BB_ISAR_UUID_DATA', d).clear()
>> + bb.utils.exec_flat_python_func('clear_build_uuid', d)
>> }
>>
>> build_completed[eventmask] = "bb.event.BuildCompleted"
>> diff --git a/testsuite/unittests/test_build_uuid.py b/testsuite/unittests/test_build_uuid.py
>> new file mode 100644
>> index 00000000..374cce64
>> --- /dev/null
>> +++ b/testsuite/unittests/test_build_uuid.py
>> @@ -0,0 +1,89 @@
>> +# This software is a part of Isar.
>> +# Copyright (C) Siemens AG, 2026
>> +#
>> +# SPDX-License-Identifier: MIT
>> +
>> +import multiprocessing
>> +import os
>> +import sys
>> +import tempfile
>> +import unittest
>> +import uuid
>> +from pathlib import Path
>> +
>> +root = Path(__file__).resolve().parents[2]
>> +sys.path.extend([
>> + str(root / 'bitbake/lib'),
>> + str(root / 'meta/lib'),
>> +])
>> +
>> +import bb
>> +import bb.data
>> +import bb.utils
>> +from bb.parse import handle
>> +
>> +
>> +data = bb.data.init()
>> +data.setVar('BBPATH', f'{root}/meta:{root}/bitbake')
>> +data.setVar('__bbclasstype', 'global')
>> +parsed = handle(str(root / 'meta/classes-global/base.bbclass'), data)
>> +namespace = {'bb': bb, 'os': os}
>> +exec(parsed['calculate_build_uuid'], namespace)
>> +exec(parsed['clear_build_uuid'], namespace)
>> +calculate_build_uuid = namespace['calculate_build_uuid']
>> +clear_build_uuid = namespace['clear_build_uuid']
>> +
>> +
>> +def calculate_in_worker(persistent_dir, start, results):
>> + data = bb.data.init()
>> + data.setVar('PERSISTENT_DIR', persistent_dir)
>> + start.wait()
>> + results.put(calculate_build_uuid(data))
>> +
>> +
>> +class TestBuildUUID(unittest.TestCase):
>> + def setUp(self):
>> + directory = tempfile.TemporaryDirectory()
>> + self.addCleanup(directory.cleanup)
>> + self.persistent_dir = os.path.join(directory.name, 'cache')
>> + self.data = bb.data.init()
>> + self.data.setVar('PERSISTENT_DIR', self.persistent_dir)
>> +
>> + def test_uuid_is_reused_across_calls_and_multiconfigs(self):
>> + build_uuid = calculate_build_uuid(self.data)
>> + self.assertEqual(str(uuid.UUID(build_uuid)), build_uuid)
>> + self.assertEqual(calculate_build_uuid(self.data), build_uuid)
>> + other_config = self.data.createCopy()
>> + other_config.setVar('BB_CURRENT_MC', 'other')
>> + self.assertEqual(calculate_build_uuid(other_config), build_uuid)
>> + clear_build_uuid(self.data)
>> + self.assertNotEqual(calculate_build_uuid(self.data), build_uuid)
>> +
>> + def test_concurrent_workers_share_uuid(self):
>> + context = multiprocessing.get_context('fork')
>> + start = context.Event()
>> + results = context.Queue()
>> + workers = [
>> + context.Process(
>> + target=calculate_in_worker,
>> + args=(self.persistent_dir, start, results),
>> + )
>> + for _ in range(8)
>> + ]
>> + try:
>> + for worker in workers:
>> + worker.start()
>> + start.set()
>> + values = [results.get(timeout=15) for worker in workers]
>> + for worker in workers:
>> + worker.join(timeout=15)
>> + self.assertEqual(worker.exitcode, 0)
>> + self.assertEqual(len(set(values)), 1)
>> + self.assertEqual(values[0], calculate_build_uuid(self.data))
>> + finally:
>> + for worker in workers:
>> + if worker.is_alive():
>> + worker.terminate()
>> + worker.join(timeout=15)
>> + results.close()
>> + results.join_thread()
>> --
>> 2.56.0
>>
>> --
>> You received this message because you are subscribed to the Google Groups "isar-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
>> To view this discussion visit https://groups.google.com/d/msgid/isar-users/20261109091802.202521-2-clara.kowalsky%40siemens.com.
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/0d7832ad-aa96-49c0-8e2d-502c5be233be%40siemens.com.
next prev parent reply other threads:[~2026-11-11 14:38 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-09 9:17 [PATCH 0/8] Update BitBake in Isar to v2.20.0 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 1/8] base: replace BitBake's outdated persist_data 'Clara Kowalsky' via isar-users
2026-11-10 11:18 ` 'Moessbauer, Felix' via isar-users
2026-11-11 14:37 ` 'Clara Kowalsky' via isar-users [this message]
2026-11-09 9:17 ` [PATCH 2/8] isar-events: preserve cleanup for default multiconfig 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 3/8] isar-events: close /proc/mounts after cleanup 'Clara Kowalsky' via isar-users
2026-11-10 11:22 ` 'Moessbauer, Felix' via isar-users
2026-11-11 14:38 ` 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 4/8] meta: migrate fetcher references to bb.fetch 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 5/8] wic: run the Python driver outside the imaging chroot 'Clara Kowalsky' via isar-users
2026-11-10 11:39 ` 'Moessbauer, Felix' via isar-users
2026-11-11 14:39 ` 'Clara Kowalsky' via isar-users
2026-11-09 9:17 ` [PATCH 6/8] bitbake.conf: explicitly exclude export and func from signatures 'Clara Kowalsky' via isar-users
2026-11-09 9:18 ` [PATCH 7/8] meta: add missing whitespace around assignment operators 'Clara Kowalsky' via isar-users
2026-11-09 9:18 ` [PATCH 8/8] bitbake: update to v2.20.0 'Clara Kowalsky' via isar-users
2026-11-13 14:41 ` [PATCH 0/8] Update BitBake in Isar " Anton Mikanovich
2026-11-13 17:19 ` 'Clara Kowalsky' via isar-users
2026-11-16 8:46 ` 'Moessbauer, Felix' via isar-users
2026-11-16 10:44 ` 'Clara Kowalsky' via isar-users
2026-11-16 12:12 ` 'Moessbauer, Felix' via isar-users
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0d7832ad-aa96-49c0-8e2d-502c5be233be@siemens.com \
--to=isar-users@googlegroups.com \
--cc=clara.kowalsky@siemens.com \
--cc=felix.moessbauer@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox