RE: ISAR schroot mountpoints when running in container

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

> -----Original Message-----
> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>
> Sent: Friday, July 1, 2022 12:44 PM
> To: Bezdeka, Florian (T CED SES-DE) <florian.bezdeka@siemens.com>;
> ubely@ilbers.de; isar-users@googlegroups.com; Moessbauer, Felix (T CED SES-
> DE) <felix.moessbauer@siemens.com>
> Subject: Re: ISAR schroot mountpoints when running in container
> 
> On 01.07.22 12:30, Bezdeka, Florian wrote:
> > On Fri, 2022-07-01 at 13:23 +0300, Uladzimir Bely wrote:
> >> In the email from Friday, 1 July 2022 12:11:42 +03 user Moessbauer,
> >> Felix
> >> wrote:
> >>> Hi,
> >>>
> >>> as we now have sbuild in ISAR next, first users stumble upon issues
> >>> when running the kas docker image in the gitlab-ci. This requires
> >>> additional mountpoints, as the schroot itself uses overlayfs and
> >>> stacking two overlayfs is not possible. By that, you have to mount
> >>> the overlay from the host.
> >>>
> >>> The kas-container script already has support for that, but it lacks
> >>> documentation on how to configure this manually. In short: The
> >>> following mountpoint has to be added (as RW):
> >>> /var/lib/schroot/union/overlay
> >>>
> >>> As this is neither an ISAR issue, nor a KAS issue per-se, I send
> >>> this to both lists.
> >>>
> >>
> >> Hello.
> >>
> >> Yes, this overlayfs-over-overlayfs issue can be solved by something
> >> like
> >>
> >> `volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"]`
> >>
> >> placed to `/etc/gitlab-runner/config.toml`, as mentioned in 'sbuild'
> >> series cover-letter
> >
> > Oh no. That's going to kill a lot of gitlab-ci setups. Even kas-
> > container based environments (often used as local development
> > environment) will need adjustments. Uncool.
> >
> > Anyway, I will try to look into the kas-container script. Let's hope
> > there is a simple solution for adding one more mount.
> >
> 
> When designing that, we should also have an eye on the optimization of
> mapping build/tmp/ onto a tmpfs mount inside the container.

Wait a minute: We are mixing things up here.

1. kas-container (script) already provides this mountpoint when enabling isar mode. By that, local deployments are not affected
2. When running the kas container image directly (via docker, podman or gitlab-ci), the mount point is missing. There is not much we can do about, except to document it for the user.
3. Schroot ID collisions: When running two kas containers in parallel, that also use the same mountpoints on the host, we likely get name collisions (as PIDs are no longer unique). This scenario is common for gitlab-ci.
4. Using tmpfs: With Schroot, we could easily put just the schroot onto a tmpfs. This has the further advantage, that the upper-dir of the overlayfs can be a tmpfs which itself can be mounted inside the container. By that, no bind-mount from the host is required. But this might only work for machines with a lot of memory.

Required changes:

In ISAR, we have to make the name of the Schroot folder more unique. But as BB requires recipes to be deterministic (per-build), we have to inject the ID from the outside. This could happen either via local.conf or via an env-var. This env-var has to be provided by KAS, with an fallback in ISAR to use the PID of the bitbake process if not provided.

A probably better strategy would be to get a per-bitbake invocation constant UUID directly from Bitbake. Don't know if that already exists in BB.
Putting Adriaan in CC. 

Felix

> 
> Jan
> 
> --
> Siemens AG, Technology
> Competence Center Embedded Linux