[PATCH v2 0/4] Fix ccache issues

[PATCH v2 0/4] Fix ccache issues

[Anton Mikanovich]

This patchset just fixes some ccache-related issues which prevented us
from enabling ccache globally.

Changes since v1:
- Rebased on next
- Addressed previous feedback

Anton Mikanovich (4):
  ccache: fix dirs creation
  ccache: fix CCACHE_DIR owner for sbuild
  meta: unify cross selection
  ccache: separate cache based on cross compile

 doc/user_manual.md               |  3 ++-
 meta/classes/buildchroot.bbclass | 22 +++++----------------
 meta/classes/crossvars.bbclass   | 33 ++++++++++++++++++++++++++++++++
 meta/classes/dpkg-base.bbclass   |  2 +-
 meta/classes/dpkg.bbclass        |  2 +-
 meta/classes/sbuild.bbclass      | 19 ++----------------
 meta/classes/sdk.bbclass         | 17 +++-------------
 meta/conf/bitbake.conf           |  2 +-
 8 files changed, 48 insertions(+), 52 deletions(-)
 create mode 100644 meta/classes/crossvars.bbclass

-- 
2.34.1

[PATCH v2 1/4] ccache: fix dirs creation

[Anton Mikanovich]

Create ccache debug directory only if CCACHE_DEBUG is enabled.
Also restore CCACHE_DIR creation if not exist.

Fixes 7956c4c ("add flag to enable ccache debug mode")

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/buildchroot.bbclass | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass
index 5abd533..3d68d29 100644
--- a/meta/classes/buildchroot.bbclass
+++ b/meta/classes/buildchroot.bbclass
@@ -24,7 +24,10 @@ MOUNT_LOCKFILE = "${BUILDCHROOT_DIR}.lock"
 
 buildchroot_do_mounts() {
     if [ "${USE_CCACHE}" = "1" ]; then
-        mkdir -p ${CCACHE_DIR}/debug
+        mkdir -p "${CCACHE_DIR}"
+        if [ "${CCACHE_DEBUG}" = "1" ]; then
+            mkdir -p "${CCACHE_DIR}/debug"
+        fi
     fi
 
     sudo -s <<'EOSUDO'
-- 
2.34.1

[PATCH v2 2/4] ccache: fix CCACHE_DIR owner for sbuild

[Anton Mikanovich]

If CCACHE_DIR do not exist before sbuild execution it is created with root
privileges. Previous 'install' implementation did not work for some reason.
This results in permissions issues during the next usage by the imager.
Create CCACHE_DIR before sbuild to fix that.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/sbuild.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index 6e3c790..ea3fb34 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -128,12 +128,12 @@ EOSUDO
 }
 
 schroot_configure_ccache() {
+    mkdir -p "${CCACHE_DIR}"
     sudo -s <<'EOSUDO'
         set -e
 
         sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
 
-        install --group=sbuild --mode=2775 -d ${CCACHE_DIR}
         fstab_ccachedir="${CCACHE_DIR} /ccache none rw,bind 0 0"
         grep -qxF "${fstab_ccachedir}" ${sbuild_fstab} || echo "${fstab_ccachedir}" >> ${sbuild_fstab}
 
-- 
2.34.1

[PATCH v2 3/4] meta: unify cross selection

[Anton Mikanovich]

Unify and move ISAR_CROSS_COMPILE related logic into one location.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/buildchroot.bbclass | 17 +---------------
 meta/classes/crossvars.bbclass   | 33 ++++++++++++++++++++++++++++++++
 meta/classes/dpkg-base.bbclass   |  2 +-
 meta/classes/dpkg.bbclass        |  2 +-
 meta/classes/sbuild.bbclass      | 17 +---------------
 meta/classes/sdk.bbclass         | 17 +++-------------
 6 files changed, 40 insertions(+), 48 deletions(-)
 create mode 100644 meta/classes/crossvars.bbclass

diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass
index 3d68d29..e4a7a57 100644
--- a/meta/classes/buildchroot.bbclass
+++ b/meta/classes/buildchroot.bbclass
@@ -3,22 +3,7 @@
 #
 # SPDX-License-Identifier: MIT
 
-ISAR_CROSS_COMPILE ??= "0"
-
-# Choose the correct buildchroot: host or target
-python __anonymous() {
-    mode = d.getVar('ISAR_CROSS_COMPILE', True)
-    distro_arch = d.getVar('DISTRO_ARCH')
-    if mode == "0" or d.getVar('HOST_ARCH') ==  distro_arch:
-        dep = "buildchroot-target:do_build"
-        rootfs = d.getVar('BUILDCHROOT_TARGET_DIR', False)
-    else:
-        dep = "buildchroot-host:do_build"
-        rootfs = d.getVar('BUILDCHROOT_HOST_DIR', False)
-
-    d.setVar('BUILDCHROOT_DEP', dep)
-    d.setVar('BUILDCHROOT_DIR', rootfs)
-}
+inherit crossvars
 
 MOUNT_LOCKFILE = "${BUILDCHROOT_DIR}.lock"
 
diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
new file mode 100644
index 0000000..3686dbe
--- /dev/null
+++ b/meta/classes/crossvars.bbclass
@@ -0,0 +1,33 @@
+# This software is a part of ISAR.
+# Copyright (C) 2022 ilbers GmbH
+
+ISAR_CROSS_COMPILE ??= "0"
+
+python __anonymous() {
+    import pwd
+    d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
+
+    mode = d.getVar('ISAR_CROSS_COMPILE')
+    distro_arch = d.getVar('DISTRO_ARCH')
+    if mode == "0" or d.getVar('HOST_ARCH') ==  distro_arch:
+        d.setVar('BUILD_HOST_ARCH', distro_arch)
+        schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
+        sbuild_dep = "sbuild-chroot-target:do_build"
+        buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False)
+        buildchroot_dep = "buildchroot-target:do_build"
+        sdk_toolchain = "build-essential"
+    else:
+        d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
+        schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
+        sbuild_dep = "sbuild-chroot-host:do_build"
+        buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False)
+        buildchroot_dep = "buildchroot-host:do_build"
+        sdk_toolchain = "crossbuild-essential-" + distro_arch
+    d.setVar('SCHROOT_DIR', schroot_dir)
+    d.setVar('SCHROOT_DEP', sbuild_dep)
+    d.setVar('BUILDCHROOT_DIR', buildchroot_dir)
+    d.setVar('BUILDCHROOT_DEP', buildchroot_dep)
+    if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == "1":
+        sdk_toolchain += " crossbuild-essential-" + d.getVar('COMPAT_DISTRO_ARCH')
+    d.setVar('TOOLCHAIN', sdk_toolchain)
+}
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index ad28f7b..4c0fdc7 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -295,7 +295,7 @@ python do_devshell() {
     schroot = d.getVar('SBUILD_CHROOT')
     isar_apt = d.getVar('ISAR_APT_REPO')
     pkg_arch = d.getVar('PACKAGE_ARCH', True)
-    build_arch = d.getVar('SBUILD_HOST_ARCH', True)
+    build_arch = d.getVar('BUILD_HOST_ARCH', True)
     pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
     debdistroname = d.getVar('DEBDISTRONAME')
 
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index d528503..52c200d 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -102,7 +102,7 @@ dpkg_runbuild() {
     DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -print)
 
     sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
-        --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} ${profiles} \
+        --host=${PACKAGE_ARCH} --build=${BUILD_HOST_ARCH} ${profiles} \
         --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
         --no-apt-update \
         --chroot-setup-commands="echo \"Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt" \
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index ea3fb34..995f7a5 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -5,22 +5,7 @@ SCHROOT_CONF ?= "/etc/schroot"
 
 SCHROOT_MOUNTS ?= ""
 
-python __anonymous() {
-    import pwd
-    d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
-
-    mode = d.getVar('ISAR_CROSS_COMPILE', True)
-    distro_arch = d.getVar('DISTRO_ARCH')
-    if mode == "0" or d.getVar('HOST_ARCH') ==  distro_arch:
-        d.setVar('SBUILD_HOST_ARCH', distro_arch)
-        d.setVar('SCHROOT_DIR', d.getVar('SCHROOT_TARGET_DIR', False))
-        dep = "sbuild-chroot-target:do_build"
-    else:
-        d.setVar('SBUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
-        d.setVar('SCHROOT_DIR', d.getVar('SCHROOT_HOST_DIR', False))
-        dep = "sbuild-chroot-host:do_build"
-    d.setVar('SCHROOT_DEP', dep)
-}
+inherit crossvars
 
 SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}"
 
diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
index 0a98ea0..d7add82 100644
--- a/meta/classes/sdk.bbclass
+++ b/meta/classes/sdk.bbclass
@@ -5,6 +5,8 @@
 #
 # This class extends the image.bbclass to supply the creation of a sdk
 
+inherit crossvars
+
 # hook up the -sdk image variant
 BBCLASSEXTEND = "sdk"
 BPN = "${PN}"
@@ -42,23 +44,10 @@ SDK_PREINSTALL += " \
     devscripts \
     equivs"
 
-# Choose the correct toolchain: cross or native
-python __anonymous() {
-    mode = d.getVar('ISAR_CROSS_COMPILE', True)
-    distro_arch = d.getVar('DISTRO_ARCH')
-    if mode == "0" or d.getVar('HOST_ARCH') ==  distro_arch:
-        toolchain = "build-essential"
-    else:
-        toolchain = "crossbuild-essential-" + distro_arch
-    if d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == "1":
-        toolchain += " crossbuild-essential-" + d.getVar('COMPAT_DISTRO_ARCH')
-    d.setVar('TOOLCHAIN', toolchain)
-}
-
 # rootfs/image overrides for the SDK
 ROOTFS_ARCH:class-sdk = "${HOST_ARCH}"
 ROOTFS_DISTRO:class-sdk = "${HOST_DISTRO}"
-ROOTFS_PACKAGES:class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${SDK_INSTALL}"
+ROOTFS_PACKAGES:class-sdk = "sdk-files ${SDK_TOOLCHAIN} ${SDK_PREINSTALL} ${SDK_INSTALL}"
 ROOTFS_FEATURES:append:class-sdk = " clean-package-cache generate-manifest export-dpkg-status"
 ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
 ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
-- 
2.34.1

[PATCH v2 4/4] ccache: separate cache based on cross compile

[Anton Mikanovich]

Mixing ccache generated by cross and native compilers makes the second
run to stuck all the build.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 doc/user_manual.md     | 3 ++-
 meta/conf/bitbake.conf | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index b9a0bb6..61d1390 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -1008,7 +1008,8 @@ to `local.conf`. If some package requires ccache to be always disabled,
 
 By default, ccache directory is created inside `TMPDIR`, but it can be
 adjusted by `CCACHE_TOP_DIR` variable in `local.conf`. Ccache directory
-`CCACHE_DIR` default value is `"${CCACHE_TOP_DIR}/${DISTRO}-${DISTRO_ARCH}"`,
+`CCACHE_DIR` default value is
+`"${CCACHE_TOP_DIR}/${DISTRO}-${DISTRO_ARCH}-${BUILD_HOST_ARCH}"`,
 that means caches for different distros and architectures are not overlapped.
 
 The ccache debug mode can be enabled by setting `CCACHE_DEBUG = "1"`
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 522241a..e6956c3 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -153,7 +153,7 @@ INHERIT += "${@'buildstats' if d.getVar('USE_BUILDSTATS') == '1' else ''}"
 # Default values for ccache
 USE_CCACHE ??= "0"
 CCACHE_TOP_DIR ?= "${TMPDIR}/ccache"
-CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${DISTRO}-${DISTRO_ARCH}"
+CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${DISTRO}-${DISTRO_ARCH}-${BUILD_HOST_ARCH}"
 CCACHE_DEBUG ?= "0"
 
 # Variables for tasks marking
-- 
2.34.1

Re: [PATCH v2 0/4] Fix ccache issues

[Anton Mikanovich]

13/02/2023 09:56, Anton Mikanovich wrote:
> This patchset just fixes some ccache-related issues which prevented us
> from enabling ccache globally.
>
> Changes since v1:
> - Rebased on next
> - Addressed previous feedback
>
There are some ccache related issues still not fixed in this version of the
patchset. This is why USE_CCACHE still did not enabled by default.
We will continue investigation but it will probably takes some time. So 
going
one step closer to clean colution looks also ok for now.

Re: [PATCH v2 0/4] Fix ccache issues

[Uladzimir Bely]

In mail from Monday, 13 February 2023 10:56:07 +03 user Anton Mikanovich 
wrote:
> This patchset just fixes some ccache-related issues which prevented us
> from enabling ccache globally.
> 
> Changes since v1:
> - Rebased on next
> - Addressed previous feedback
> 
> Anton Mikanovich (4):
>   ccache: fix dirs creation
>   ccache: fix CCACHE_DIR owner for sbuild
>   meta: unify cross selection
>   ccache: separate cache based on cross compile
> 
>  doc/user_manual.md               |  3 ++-
>  meta/classes/buildchroot.bbclass | 22 +++++----------------
>  meta/classes/crossvars.bbclass   | 33 ++++++++++++++++++++++++++++++++
>  meta/classes/dpkg-base.bbclass   |  2 +-
>  meta/classes/dpkg.bbclass        |  2 +-
>  meta/classes/sbuild.bbclass      | 19 ++----------------
>  meta/classes/sdk.bbclass         | 17 +++-------------
>  meta/conf/bitbake.conf           |  2 +-
>  8 files changed, 48 insertions(+), 52 deletions(-)
>  create mode 100644 meta/classes/crossvars.bbclass

Rebased and applied to next.

Re: [PATCH v2 0/4] Fix ccache issues

[Moessbauer, Felix]

On Mon, 2023-03-27 at 08:27 +0300, Uladzimir Bely wrote:
> In mail from Monday, 13 February 2023 10:56:07 +03 user Anton
> Mikanovich 
> wrote:
> > This patchset just fixes some ccache-related issues which prevented
> > us
> > from enabling ccache globally.

Hi,

since this commit, I again see ccache issues when mixing cross and non-
cross compilation for arm64 targets:

ccache: error: Failed to create directory /ccache/a/7: Not a directory

The error happened in the non-cross recipe.

I'm not 100% sure if this is related, but with ISAR f95f69d3 I did not
observe these on the same layer.

Felix

> > 
> > Changes since v1:
> > - Rebased on next
> > - Addressed previous feedback
> > 
> > Anton Mikanovich (4):
> >   ccache: fix dirs creation
> >   ccache: fix CCACHE_DIR owner for sbuild
> >   meta: unify cross selection
> >   ccache: separate cache based on cross compile
> > 
> >  doc/user_manual.md               |  3 ++-
> >  meta/classes/buildchroot.bbclass | 22 +++++----------------
> >  meta/classes/crossvars.bbclass   | 33
> > ++++++++++++++++++++++++++++++++
> >  meta/classes/dpkg-base.bbclass   |  2 +-
> >  meta/classes/dpkg.bbclass        |  2 +-
> >  meta/classes/sbuild.bbclass      | 19 ++----------------
> >  meta/classes/sdk.bbclass         | 17 +++-------------
> >  meta/conf/bitbake.conf           |  2 +-
> >  8 files changed, 48 insertions(+), 52 deletions(-)
> >  create mode 100644 meta/classes/crossvars.bbclass
> 
> Rebased and applied to next.
> 
> 

Re: [PATCH v2 0/4] Fix ccache issues

[Uladzimir Bely]

In mail from Tuesday, 28 March 2023 18:51:58 +03 user Moessbauer, Felix wrote:
> On Mon, 2023-03-27 at 08:27 +0300, Uladzimir Bely wrote:
> 
> > In mail from Monday, 13 February 2023 10:56:07 +03 user Anton
> > Mikanovich 
> > wrote:
> > 
> > > This patchset just fixes some ccache-related issues which prevented
> > > us
> > > from enabling ccache globally.
> 
> 
> Hi,
> 
> since this commit, I again see ccache issues when mixing cross and non-
> cross compilation for arm64 targets:
> 
> ccache: error: Failed to create directory /ccache/a/7: Not a directory
> 
> The error happened in the non-cross recipe.
> 
> I'm not 100% sure if this is related, but with ISAR f95f69d3 I did not
> observe these on the same layer.
> 
> Felix
> 

Probably, it's related to patch2.

You might have /ccache/a/ created earlier by 'root', and not it can't create a 
subdir as sbuild user.

Re: [PATCH v2 0/4] Fix ccache issues

[Moessbauer, Felix]

On Tue, 2023-03-28 at 18:56 +0300, Uladzimir Bely wrote:
> In mail from Tuesday, 28 March 2023 18:51:58 +03 user Moessbauer,
> Felix wrote:
> > On Mon, 2023-03-27 at 08:27 +0300, Uladzimir Bely wrote:
> > 
> > > In mail from Monday, 13 February 2023 10:56:07 +03 user Anton
> > > Mikanovich 
> > > wrote:
> > > 
> > > > This patchset just fixes some ccache-related issues which
> > > > prevented
> > > > us
> > > > from enabling ccache globally.
> > 
> > 
> > Hi,
> > 
> > since this commit, I again see ccache issues when mixing cross and
> > non-
> > cross compilation for arm64 targets:
> > 
> > ccache: error: Failed to create directory /ccache/a/7: Not a
> > directory
> > 
> > The error happened in the non-cross recipe.
> > 
> > I'm not 100% sure if this is related, but with ISAR f95f69d3 I did
> > not
> > observe these on the same layer.
> > 
> > Felix
> > 
> 
> Probably, it's related to patch2.
> 
> You might have /ccache/a/ created earlier by 'root', and not it can't
> create a 
> subdir as sbuild user.

Hi, this could very well be the case. However, when running in a CI on
a project with multiple branches (e.g. during feature integration),
this situation cannot be avoided. The cache is shared across all builds
of a single project, but these builds might use different ISAR
versions.

This technically means, we need to update all branches of the project
in a lockstep to be able to re-enable the ccache.

It would be good if we could find a way to avoid this situation.

Felix


> 
> 

Re: [PATCH v2 0/4] Fix ccache issues

[Uladzimir Bely]

In mail from Wednesday, 29 March 2023 15:43:10 +03 user Moessbauer, Felix 
wrote:
> On Tue, 2023-03-28 at 18:56 +0300, Uladzimir Bely wrote:
> 
> > In mail from Tuesday, 28 March 2023 18:51:58 +03 user Moessbauer,
> > Felix wrote:
> > 
> > > On Mon, 2023-03-27 at 08:27 +0300, Uladzimir Bely wrote:
> > > 
> > > 
> > > > In mail from Monday, 13 February 2023 10:56:07 +03 user Anton
> > > > Mikanovich 
> > > > wrote:
> > > > 
> > > > 
> > > > > This patchset just fixes some ccache-related issues which
> > > > > prevented
> > > > > us
> > > > > from enabling ccache globally.
> > > 
> > > 
> > > 
> > > Hi,
> > > 
> > > since this commit, I again see ccache issues when mixing cross and
> > > non-
> > > cross compilation for arm64 targets:
> > > 
> > > ccache: error: Failed to create directory /ccache/a/7: Not a
> > > directory
> > > 
> > > The error happened in the non-cross recipe.
> > > 
> > > I'm not 100% sure if this is related, but with ISAR f95f69d3 I did
> > > not
> > > observe these on the same layer.
> > > 
> > > Felix
> > > 
> > 
> > 
> > Probably, it's related to patch2.
> > 
> > You might have /ccache/a/ created earlier by 'root', and not it can't
> > create a 
> > subdir as sbuild user.
> 
> 
> Hi, this could very well be the case. However, when running in a CI on
> a project with multiple branches (e.g. during feature integration),
> this situation cannot be avoided. The cache is shared across all builds
> of a single project, but these builds might use different ISAR
> versions.
> 

Yes, it sounds a bit challenging. 

Returning back to root-owned ccache would not be a good solution. But if there 
are no other ways to deal with it, we might have to do it.

Alternatively, kas-based downstreams using old isar could apply patch2 as isar 
patch, until updated.

> This technically means, we need to update all branches of the project
> in a lockstep to be able to re-enable the ccache.
> 
> It would be good if we could find a way to avoid this situation.
> 
> Felix
> 
> 
> 
> > 
> > 
> 
> 

Re: [PATCH v2 0/4] Fix ccache issues

[Baurzhan Ismagulov]

On 2023-03-29 12:43, Moessbauer, Felix wrote:
> > You might have /ccache/a/ created earlier by 'root', and not it can't
> > create a 
> > subdir as sbuild user.
> 
> Hi, this could very well be the case. However, when running in a CI on
> a project with multiple branches (e.g. during feature integration),
> this situation cannot be avoided. The cache is shared across all builds
> of a single project, but these builds might use different ISAR
> versions.
> 
> This technically means, we need to update all branches of the project
> in a lockstep to be able to re-enable the ccache.
> 
> It would be good if we could find a way to avoid this situation.

What about sudo addgroup ccache; sudo adduser build ccache; sudo chgrp -R
ccache /ccache; sudo chmod -R g+w /ccache; find /ccache -type d |sudo xargs
chmod g+s? Maybe also sudo adduser root ccache.

With kind regards,
Baurzhan

Re: [PATCH v2 0/4] Fix ccache issues

[Moessbauer, Felix]

On Sun, 2023-04-02 at 20:44 +0200, Baurzhan Ismagulov wrote:
> On 2023-03-29 12:43, Moessbauer, Felix wrote:
> > > You might have /ccache/a/ created earlier by 'root', and not it
> > > can't
> > > create a 
> > > subdir as sbuild user.
> > 
> > Hi, this could very well be the case. However, when running in a CI
> > on
> > a project with multiple branches (e.g. during feature integration),
> > this situation cannot be avoided. The cache is shared across all
> > builds
> > of a single project, but these builds might use different ISAR
> > versions.
> > 
> > This technically means, we need to update all branches of the
> > project
> > in a lockstep to be able to re-enable the ccache.
> > 
> > It would be good if we could find a way to avoid this situation.
> 
> What about sudo addgroup ccache; sudo adduser build ccache; sudo
> chgrp -R
> ccache /ccache; sudo chmod -R g+w /ccache; find /ccache -type d |sudo
> xargs
> chmod g+s? Maybe also sudo adduser root ccache.

This would solve the permission issues at the cost of always touching a
lot of files. Especially in case the cache is on a NFS, this could be
problematic. All that boils down to the following root causes:

- shared caches across ISAR versions in the same CI
- inconsistent users creating / accessing the cache in ISAR (e.g.
sbuilder as builder, devshell as root)

Also some custom tasks might accidentally use the ccache and break the
build for the following (possibly unrelated) tasks. I don't know which
solution is best. Ideas are highly appreciated.

Felix

> 
> With kind regards,
> Baurzhan
> 

Re: [PATCH v2 0/4] Fix ccache issues

[Baurzhan Ismagulov]

On 2023-04-05 11:56, Moessbauer, Felix wrote:
> > What about sudo addgroup ccache; sudo adduser build ccache; sudo
> > chgrp -R
> > ccache /ccache; sudo chmod -R g+w /ccache; find /ccache -type d |sudo
> > xargs
> > chmod g+s? Maybe also sudo adduser root ccache.
> 
> This would solve the permission issues at the cost of always touching a
> lot of files.

Hmm, why always a lot? This is meant as a one-time action during setup. Set the
directory group to ccache, set the mode to at least 2770 -- anything that is
created under it should be readable and writable by anyone in the group. I used
this in the past, but didn't test with root and NFS.

With kind regards,
Baurzhan

Re: [PATCH v2 0/4] Fix ccache issues

[Moessbauer, Felix]

On Wed, 2023-04-05 at 15:28 +0200, Baurzhan Ismagulov wrote:
> On 2023-04-05 11:56, Moessbauer, Felix wrote:
> > > What about sudo addgroup ccache; sudo adduser build ccache; sudo
> > > chgrp -R
> > > ccache /ccache; sudo chmod -R g+w /ccache; find /ccache -type d
> > > |sudo
> > > xargs
> > > chmod g+s? Maybe also sudo adduser root ccache.
> > 
> > This would solve the permission issues at the cost of always
> > touching a
> > lot of files.
> 
> Hmm, why always a lot? This is meant as a one-time action during
> setup. Set the
> directory group to ccache, set the mode to at least 2770 -- anything
> that is
> created under it should be readable and writable by anyone in the
> group. I used
> this in the past, but didn't test with root and NFS.

Is this a one-time action per dpkg-build task, or how often is that
called? I'm also a bit unsure how that works together with other ccache
instances from older ISAR versions that simultaneously work on the same
cache, but using root. The automatic internal cleanup logic of the
ccache as builder might then want to delete ccache entries which have
been created by a ccache running as root, but after the chmod:

An example sequence is the following (A: old, B:new):

B: run chmod
A: create cache entries
B: write to cache, fill it up and delete entries created by A

Do I miss something?

Best regards,
Felix

> 
> With kind regards,
> Baurzhan
>