Re: ISAR schroot mountpoints when running in container

[Henning Schild <henning.schild@siemens.com>]

Am Fri, 1 Jul 2022 11:30:39 +0000
schrieb "Moessbauer, Felix" <felix.moessbauer@siemens.com>:

> > -----Original Message-----
> > From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>
> > Sent: Friday, July 1, 2022 12:44 PM
> > To: Bezdeka, Florian (T CED SES-DE) <florian.bezdeka@siemens.com>;
> > ubely@ilbers.de; isar-users@googlegroups.com; Moessbauer, Felix (T
> > CED SES- DE) <felix.moessbauer@siemens.com>
> > Subject: Re: ISAR schroot mountpoints when running in container
> > 
> > On 01.07.22 12:30, Bezdeka, Florian wrote:  
> > > On Fri, 2022-07-01 at 13:23 +0300, Uladzimir Bely wrote:  
> > >> In the email from Friday, 1 July 2022 12:11:42 +03 user
> > >> Moessbauer, Felix
> > >> wrote:  
> > >>> Hi,
> > >>>
> > >>> as we now have sbuild in ISAR next, first users stumble upon
> > >>> issues when running the kas docker image in the gitlab-ci. This
> > >>> requires additional mountpoints, as the schroot itself uses
> > >>> overlayfs and stacking two overlayfs is not possible. By that,
> > >>> you have to mount the overlay from the host.
> > >>>
> > >>> The kas-container script already has support for that, but it
> > >>> lacks documentation on how to configure this manually. In
> > >>> short: The following mountpoint has to be added (as RW):
> > >>> /var/lib/schroot/union/overlay
> > >>>
> > >>> As this is neither an ISAR issue, nor a KAS issue per-se, I send
> > >>> this to both lists.
> > >>>  
> > >>
> > >> Hello.
> > >>
> > >> Yes, this overlayfs-over-overlayfs issue can be solved by
> > >> something like
> > >>
> > >> `volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"]`
> > >>
> > >> placed to `/etc/gitlab-runner/config.toml`, as mentioned in
> > >> 'sbuild' series cover-letter  
> > >
> > > Oh no. That's going to kill a lot of gitlab-ci setups. Even kas-
> > > container based environments (often used as local development
> > > environment) will need adjustments. Uncool.
> > >
> > > Anyway, I will try to look into the kas-container script. Let's
> > > hope there is a simple solution for adding one more mount.
> > >  
> > 
> > When designing that, we should also have an eye on the optimization
> > of mapping build/tmp/ onto a tmpfs mount inside the container.  
> 
> Wait a minute: We are mixing things up here.
> 
> 1. kas-container (script) already provides this mountpoint when
> enabling isar mode. By that, local deployments are not affected 2.
> When running the kas container image directly (via docker, podman or
> gitlab-ci), the mount point is missing. There is not much we can do
> about, except to document it for the user. 3. Schroot ID collisions:
> When running two kas containers in parallel, that also use the same
> mountpoints on the host, we likely get name collisions (as PIDs are
> no longer unique). This scenario is common for gitlab-ci. 4. Using
> tmpfs: With Schroot, we could easily put just the schroot onto a
> tmpfs. This has the further advantage, that the upper-dir of the
> overlayfs can be a tmpfs which itself can be mounted inside the
> container. By that, no bind-mount from the host is required. But this
> might only work for machines with a lot of memory.

What if the runner had btrfs? I guess that would be a good
suggestion which would lead to no overlay at the base, not needing a
volume mount in the config.

Henning

> Required changes:
> 
> In ISAR, we have to make the name of the Schroot folder more unique.
> But as BB requires recipes to be deterministic (per-build), we have
> to inject the ID from the outside. This could happen either via
> local.conf or via an env-var. This env-var has to be provided by KAS,
> with an fallback in ISAR to use the PID of the bitbake process if not
> provided.
> 
> A probably better strategy would be to get a per-bitbake invocation
> constant UUID directly from Bitbake. Don't know if that already
> exists in BB. Putting Adriaan in CC. 
> 
> Felix
> 
> > 
> > Jan
> > 
> > --
> > Siemens AG, Technology
> > Competence Center Embedded Linux  
>