[PATCH v3 0/1] add uuid to schroot folder

[PATCH v3 0/1] add uuid to schroot folder

[Felix Moessbauer]

Changes since v2:

- re-add PID component
- remove PN component to shorten file name (was added as part of v1)

Changes since v1:

- Add ISAR_BUILD_UUID to BB_HASHBASE_WHITELIST to exclude from cache signature

Felix Moessbauer (1):
  add uuid to schroot folder

 meta/classes/sbuild.bbclass    | 2 +-
 meta/conf/bitbake.conf         | 7 ++++++-
 scripts/isar-buildenv-internal | 4 +++-
 3 files changed, 10 insertions(+), 3 deletions(-)

-- 
2.30.2

[PATCH v3 1/1] add uuid to schroot folder

[Felix Moessbauer]

PIDs are not unique across containers.
When running the build in a container (e.g. the kas container),
the PID of bitbake is likely the same across multiple simultaneously
running builds.

This is especially the case for CI runners, where it is common that
multiple jobs run in parallel.

This patch adds an additional UUID component that is injected
from the ISAR environment.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/sbuild.bbclass    | 2 +-
 meta/conf/bitbake.conf         | 7 ++++++-
 scripts/isar-buildenv-internal | 4 +++-
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index f9bfaaf6..7c399543 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -23,7 +23,7 @@ python __anonymous() {
     d.setVar('SCHROOT_DEP', dep)
 }
 
-SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${@os.getpid()}"
+SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}"
 
 SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}"
 SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6451cb59..31e11569 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -104,7 +104,7 @@ BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI
     WORKDIR STAMPCLEAN PKGDATA_DIR BUILD_ARCH SSTATE_PKGARCH \
     BB_WORKERCONTEXT BB_LIMITEDDEPS DEPLOY_DIR BUILDCHROOT_DIR \
     REPO_ISAR_DIR REPO_ISAR_DB_DIR REPO_BASE_DIR REPO_BASE_DB_DIR LAYERDIR_core \
-    SCRIPTSDIR TOPDIR"
+    SCRIPTSDIR TOPDIR ISAR_BUILD_UUID"
 BB_HASHCONFIG_WHITELIST ?= "${BB_HASHBASE_WHITELIST} DATE TIME SSH_AGENT_PID \
     SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_EXTRAWHITE DISABLE_SANITY_CHECKS \
     BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \
@@ -134,6 +134,11 @@ USE_CCACHE ??= "0"
 CCACHE_TOP_DIR ?= "${TMPDIR}/ccache"
 CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${DISTRO}-${DISTRO_ARCH}"
 
+# Unique ID for this build, used to avoid name clashes on external mountpoints
+# When running parallel builds in different PID namespaces, overwrite
+# this with a globally unique value (e.g. by injecting a UUID from the environment)
+ISAR_BUILD_UUID ??= "none"
+
 include conf/local.conf
 include conf/multiconfig/${BB_CURRENT_MC}.conf
 include conf/machine/${MACHINE}.conf
diff --git a/scripts/isar-buildenv-internal b/scripts/isar-buildenv-internal
index ec8e4377..e5cb5364 100755
--- a/scripts/isar-buildenv-internal
+++ b/scripts/isar-buildenv-internal
@@ -62,6 +62,8 @@ export BUILDDIR
 export BITBAKEDIR="${ISARROOT}/bitbake"
 export SCRIPTSDIR="${ISARROOT}/scripts"
 export TESTSUITEDIR="${ISARROOT}/testsuite"
+# use builtin python uuid to avoid dep to uuidgen
+export ISAR_BUILD_UUID="$(python3 -c 'import uuid; print(str(uuid.uuid4()))')"
 
 for newpath in "$BITBAKEDIR/bin" "$SCRIPTSDIR"; do
     # Remove any existences of $newpath from $PATH
@@ -77,5 +79,5 @@ export PATH
 BBPATH="${BUILDDIR}"
 export BBPATH
 
-BB_ENV_EXTRAWHITE="BITBAKEDIR SCRIPTSDIR TESTSUITEDIR http_proxy https_proxy ftp_proxy no_proxy GNUPGHOME"
+BB_ENV_EXTRAWHITE="BITBAKEDIR SCRIPTSDIR TESTSUITEDIR http_proxy https_proxy ftp_proxy no_proxy GNUPGHOME ISAR_BUILD_UUID"
 export BB_ENV_EXTRAWHITE
-- 
2.30.2

Re: [PATCH v3 1/1] add uuid to schroot folder

[Anton Mikanovich]

04.07.2022 12:15, Felix Moessbauer wrote:
> PIDs are not unique across containers.
> When running the build in a container (e.g. the kas container),
> the PID of bitbake is likely the same across multiple simultaneously
> running builds.
>
> This is especially the case for CI runners, where it is common that
> multiple jobs run in parallel.
>
> This patch adds an additional UUID component that is injected
> from the ISAR environment.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
I'm still think placing ISAR_BUILD_UUID generaion outside bitbake is not 
really
good idea. This is ok for single run inside container but not so 
suitable for
running several builds in one build environment (when we source
isar-init-build-env only once and then execute bitbake over and over again).
Current implementation is more env-unique but not build-unique.

RE: [PATCH v3 1/1] add uuid to schroot folder

[Moessbauer, Felix]

> -----Original Message-----
> From: Anton Mikanovich <amikan@ilbers.de>
> Sent: Friday, July 8, 2022 2:47 PM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar-
> users@googlegroups.com
> Cc: Bezdeka, Florian (T CED SES-DE) <florian.bezdeka@siemens.com>; Schild,
> Henning (T CED SES-DE) <henning.schild@siemens.com>
> Subject: Re: [PATCH v3 1/1] add uuid to schroot folder
> 
> 04.07.2022 12:15, Felix Moessbauer wrote:
> > PIDs are not unique across containers.
> > When running the build in a container (e.g. the kas container), the
> > PID of bitbake is likely the same across multiple simultaneously
> > running builds.
> >
> > This is especially the case for CI runners, where it is common that
> > multiple jobs run in parallel.
> >
> > This patch adds an additional UUID component that is injected from the
> > ISAR environment.
> >
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> I'm still think placing ISAR_BUILD_UUID generaion outside bitbake is not really
> good idea. This is ok for single run inside container but not so suitable for
> running several builds in one build environment (when we source isar-init-build-
> env only once and then execute bitbake over and over again).
> Current implementation is more env-unique but not build-unique.

We need both, and that is exactly what is implemented in this patch.
We need an external UUID to solve the non-unique PID problem when running in multiple containers simultaneously (like in the gitlab-ci), and we need the PID to support multiple builds in the same dir.

In the future we might want to use the unshare backend and get rid of the mountpoint altogether, but currently this is not mature enough. Hence, we need a solution to fix the sporadic breaks of the CI by PID collisions in the schroot mountpoint.

Felix