Re: [PATCH v6 17/17] run-tests: add support for isar-rootless mode

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Mon, 2026-06-22 at 21:53 +0200, Jan Kiszka wrote:
> On 22.06.26 20:55, 'Jan Kiszka' via isar-users wrote:
> > On 22.06.26 18:25, Moessbauer, Felix (FT RPD CED OES-DE) wrote:
> > > On Fri, 2026-06-19 at 16:32 +0200, Zhihang Wei wrote:
> > > > On 6/19/26 12:03, MOESSBAUER, Felix wrote:
> > > > > On Fri, 2026-06-19 at 11:08 +0200, Zhihang Wei wrote:
> > > > > > On 6/15/26 11:24, 'Felix Moessbauer' via isar-users wrote:
> > > > > > > If the testsuite is started in rootless mode (-p rootless=1), then start
> > > > > > > the container in rootless mode as well. By that, we can seamlessly
> > > > > > > switch between rootfull and rootless mode when testing.
> > > > > > > 
> > > > > > > Note, that this requires a recent enough kas-container version (>5.3).
> > > > > > > 
> > > > > > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > > > > > > ---
> > > > > > >    scripts/run-tests.sh | 7 ++++++-
> > > > > > >    1 file changed, 6 insertions(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/scripts/run-tests.sh b/scripts/run-tests.sh
> > > > > > > index bce10d70..ed373af7 100755
> > > > > > > --- a/scripts/run-tests.sh
> > > > > > > +++ b/scripts/run-tests.sh
> > > > > > > @@ -15,4 +15,9 @@ TEST_CONTAINER_VERSION=$(cat ${ISAR_DIR}/testsuite/dockerdata/version)
> > > > > > >    export KAS_CONTAINER_IMAGE_DISTRO="container:$TEST_CONTAINER_VERSION"
> > > > > > >    export KAS_CONTAINER_IMAGE=${CONTAINER_BASENAME:-ghcr.io/ilbers/isar}/test
> > > > > > >    
> > > > > > > -${ISAR_DIR}/kas/kas-container --isar --repo-ro shell -c "$*"
> > > > > > > +ISAR_FLAG="--isar"
> > > > > > > +case "$*" in
> > > > > > > +    *"-p rootless=1"*) ISAR_FLAG="--isar-rootless" ;;
> > > > > > > +esac
> > > > > > > +
> > > > > > > +${ISAR_DIR}/kas/kas-container ${ISAR_FLAG} --repo-ro shell -c "$*"
> > > > > > "-p rootless=1" is also passed to avocado, seems unintended.
> > > > > This is intended as we manually need to tell isar to build in rootless
> > > > > mode. Please check p16:
> > > > > 
> > > > > --- a/testsuite/cibuilder.py
> > > > > +++ b/testsuite/cibuilder.py
> > > > > @@ -128,6 +128,7 @@ class CIBuilder(Test):
> > > > >           customizations=None,
> > > > >           generate_sbom=False,
> > > > >           lines=None,
> > > > > +        rootless=False,
> > > > >           **kwargs,
> > > > >       ):
> > > > >           # write configuration file and set bitbake_args
> > > > > @@ -140,6 +141,9 @@ class CIBuilder(Test):
> > > > >           if not sstate:
> > > > >               sstate = bool(int(self.params.get('sstate', default=0)))
> > > > >   
> > > > > +        if not rootless:
> > > > > +            rootless = bool(int(self.params.get('rootless',
> > > > > default=0)))
> > > > 
> > > > Understand.
> > > > 
> > > > > > isar$ scripts/run-tests.sh -t dev -p rootless=1
> > > > > > + avocado run --max-parallel-tasks=1 --disable-sysinfo -t dev -p
> > > > > > rootless=1 testsuite/citest.py
> > > > > This will not work, as this starts the container in rootless mode, but
> > > > > the build in privileged mode (which also explains the error pattern
> > > > > from the other mail).
> > > > 
> > > > But why would this not work? I can see from the Avocado test log:
> > > > 
> > > > [stdlog] 2026-06-1916:17:36,287avocado.testparameters L0141 DEBUG| 
> > > > PARAMS (key=rootless, path=*, default=0) => '1'
> > > > [stdlog] 2026-06-1916:17:36,287avocado.testcibuilder L0164 INFO| 
> > > > ===================================================
> > > > ......
> > > > [stdlog] rootless = True
> > > > ......
> > > > 
> > > > So the build is done in rootless mode.
> > > > 
> > > > Or could you share how you are testing it? Thanks.
> > > 
> > > Hi, I did the following to prepare the container and then run the tests
> > > (on a 7.0.10+deb14-amd64 kernel):
> > > 
> > > podman pull ghcr.io/siemens/kas/kas-isar:next
> > > podman build --network=host --build-arg KAS_VERSION=next --tag
> > > ghcr.io/ilbers/isar/test-container:0 --file
> > > testsuite/dockerdata/Dockerfile .
> > > # get latest vesion of kas-container, paste into kas/kas-container
> > > 
> > > # Run dev tests, passes
> > > ./scripts/run-tests.sh -t dev -p rootless=1
> > > 
> > > JOB ID     : 3189ecd790d9825a8402c61fdd8b20768f18c2d8
> > > JOB LOG    : /work/build/testsuite/job-results/job-2026-06-22T15.41-
> > > 3189ecd/job.log
> > >  (1/6) testsuite/citest.py:DevTest.test_dev: STARTED
> > >  (1/6) testsuite/citest.py:DevTest.test_dev: PASS (1190.69 s)
> > >  (2/6) testsuite/citest.py:DevTest.test_dev_apps: STARTED
> > >  (2/6) testsuite/citest.py:DevTest.test_dev_apps: PASS (1427.20 s)
> > >  (3/6) testsuite/citest.py:DevTest.test_dev_rebuild: STARTED
> > >  (3/6) testsuite/citest.py:DevTest.test_dev_rebuild: PASS (373.13 s)
> > >  (4/6) testsuite/citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
> > >  (4/6) testsuite/citest.py:DevTest.test_dev_run_amd64_bookworm: PASS
> > > (96.29 s)
> > >  (5/6) testsuite/citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
> > >  (5/6) testsuite/citest.py:DevTest.test_dev_run_arm64_bookworm: PASS
> > > (49.01 s)
> > >  (6/6) testsuite/citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
> > >  (6/6) testsuite/citest.py:DevTest.test_dev_run_arm_bookworm: PASS
> > > (45.41 s)
> > > 
> > > # Run nocross test (which failed on your side)
> > > ./scripts/run-tests.sh testsuite/citest.py:NoCrossTest.test_nocross -p
> > > rootless=1
> > > + avocado run --max-parallel-tasks=1 --disable-sysinfo
> > > testsuite/citest.py:NoCrossTest.test_nocross -p rootless=1
> > > JOB ID     : 801920baf29862426207f9abf8a22ab2fd69807d
> > > JOB LOG    : /work/build/testsuite/job-results/job-2026-06-22T18.12-
> > > 801920b/job.log
> > >  (1/6) testsuite/citest.py:NoCrossTest.test_nocross: STARTED
> > > [...]
> > > 
> > > Could it be, that your kernel is too old and does not support binfmt-
> > > misc namespacing, which was added in Linux 6.7.
> > > 
> > 
> > That reminds me that we wanted to add some more feature qualifications
> > to kas-container before or at least after spawning a rootless session.
> > Checking for binfmt namespace support would be a candidate. Is there
> > more that we need from the kernel which may not be present?

We need support for unprivileged user namespaces, which is tricky to
test (that's why I explicitly decided against this test after talking
to our container experts). But it immediately fails when starting the
container. And it usually is available.

> > 
> > Regarding binfmt_misc, there is likely no way to discover its support
> > via /proc or /sys, is there? Then we should test at least once how the
> > error mode looks like when trying to mount or configure binfmt_misc
> > rootless on a, say 6.1 kernel.
> > 
> 
> mount: /proc/sys/fs/binfmt_misc: permission denied.
>        dmesg(1) may have more information after failed mount system call.
> sh: 1: cannot create /proc/sys/fs/binfmt_misc/register: Directory nonexistent
> sh: 1: cannot create /proc/sys/fs/binfmt_misc/register: Directory nonexistent
> [...]
> sh: 1: cannot create /proc/sys/fs/binfmt_misc/register: Directory nonexistent
> 2026-06-22 19:50:38 - INFO     - kas 5.3 started on Debian GNU/Linux trixie
> 2026-06-22 19:50:38 - INFO     - Using /repo as root for repository isar
> 2026-06-22 19:50:38 - INFO     - To start the default build, run: bitbake -c build isar-image-base
> builder@f35b2a0cfa71:/work/build$ 

That looks familiar and similar to the one observed by Zhihang Wei.

We probably want to emit a warning from the kas-container entrypoint if
support for binfmt_misc is missing (if the fs cannot be mounted).

> 
> This is what you get in bookworm debvm trying to start a rootless cross-
> build. We need to improve kas here to detect the error properly and bail 
> out. Good that I didn't release 5.4 yet.

I'm not sure if we really want to bail out in this case. If the build
is native, there is no reason to bail out. Instead, we need to check
for the fs in isar-mmdebstrap-target.

How about the following:

- emit a warning in the container-entrypoint if binfmt_misc cannot be
mounted
- add a pre-func to do_bootstrap on isar-mmdebstrap-target that checks
if binfmt_misc is mounted

Felix

> 
> Jan
> 
> -- 
> Siemens AG, Foundational Technologies
> Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/c52265fe33ae4087f3cbaf598969004bcd82ef79.camel%40siemens.com.