[PATCH 0/9] Improve testsuite executability, basic GitHub CI

[PATCH 0/9] Improve testsuite executability, basic GitHub CI

['Jan Kiszka' via isar-users]

There were several proposals for wrapping the testsuite execution into
some container. There is even docker-isar already, though this one was
not yet used for a wrapper.

This new approach enhances the existing container to "test-container", a
kas-based environment that contains all required testsuite dependencies
and can run them both as CI base container as well as via a script that
is kas-container-like (actually using it). So, all the setup steps are
history now, just call

    scripts/run-tests.sh -t dev

to execute the minimal testsuite locally. Or just run

    scripts/run-tests.sh testsuite/citest.py:test_dev_apps

to execute a specific test (avocado command line interface).

Along that, kas is updated consistently to 5.2, using the trixie
container now. The testsuite loses some obsolete KFAIL tags. And,
finally, some basic CI step is added as GitHub action.

The latter can be considered an RFC for now. It's unclear how far we can
go on public CI by parallelizing some of the test sets. Avocado lacks
support for test dependencies [1], thus this would require additional
maintenance and may still run into the limitations of public GitHub
runners.

A full run of this series can be found at [2]. This also allows to try
out the container before it was built officially, just set this prior to
calling run-tests.sh:

    CONTAINER_BASENAME=ghcr.io/siemens/isar

Jan

[1] https://github.com/avocado-framework/avocado/issues/6131#issuecomment-2694016818
[2] https://github.com/siemens/isar/actions/runs/23426024062

Jan Kiszka (9):
  Update kas-container to 5.2
  testsuite: Provide a new kas-based test container and wrapper script
  testsuite: Install avocado-framework-plugin-varianter-yaml-to-mux in
    test-container
  testsuite: Add squid to test-container
  ci: Add github workflow for building and deploying test-container
  ci: Switch gitlab-ci to new test-container
  CONTRIBUTING: Switch to new run-tests.sh
  testsuite: Drop KFAIL from trixie tests
  ci: Add github test job to github workflow

 .github/workflows/main.yml                    | 101 ++++++++++++++++++
 .gitlab-ci.yml                                |  22 ++--
 CONTRIBUTING.md                               |  72 +++----------
 kas/kas-container                             |   6 +-
 scripts/ci_setup.sh                           |  32 ------
 scripts/run-tests.sh                          |  18 ++++
 testsuite/citest.py                           |  15 +--
 testsuite/dockerdata/Dockerfile               |  19 +++-
 testsuite/dockerdata/README.md                |  20 ----
 testsuite/dockerdata/build.sh                 |  14 +++
 .../dockerdata/test-container-entrypoint      |  99 +++++++++++++++++
 testsuite/dockerdata/version                  |   1 +
 12 files changed, 280 insertions(+), 139 deletions(-)
 create mode 100644 .github/workflows/main.yml
 delete mode 100755 scripts/ci_setup.sh
 create mode 100755 scripts/run-tests.sh
 delete mode 100644 testsuite/dockerdata/README.md
 create mode 100755 testsuite/dockerdata/build.sh
 create mode 100755 testsuite/dockerdata/test-container-entrypoint
 create mode 100644 testsuite/dockerdata/version

-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/cover.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 1/9] Update kas-container to 5.2

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

This allows to move to the trixie container because the sbuild issue
has been fixed upstream and was picked up by the 5.2 release.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 kas/kas-container | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kas/kas-container b/kas/kas-container
index b46682e4..993f0bb8 100755
--- a/kas/kas-container
+++ b/kas/kas-container
@@ -27,9 +27,9 @@
 
 set -e
 
-KAS_CONTAINER_SCRIPT_VERSION="5.1"
+KAS_CONTAINER_SCRIPT_VERSION="5.2"
 KAS_IMAGE_VERSION_DEFAULT="${KAS_CONTAINER_SCRIPT_VERSION}"
-KAS_CONTAINER_IMAGE_DISTRO_DEFAULT="debian-bookworm"
+KAS_CONTAINER_IMAGE_DISTRO_DEFAULT=""
 KAS_CONTAINER_IMAGE_PATH_DEFAULT="ghcr.io/siemens/kas"
 KAS_CONTAINER_IMAGE_NAME_DEFAULT="kas"
 KAS_CONTAINER_SELF_NAME="$(basename "$0")"
@@ -80,6 +80,8 @@ usage()
 		    "container.\n"
 	printf "%b" "--ssh-agent\t\tForward ssh-agent socket to the container.\n"
 	printf "%b" "--aws-dir\t\tDirectory containing AWScli configuration.\n"
+	printf "%b" "\t\t\tAvoid \$HOME/.aws unless you fully trust the " \
+		    "container.\n"
 	printf "%b" "--git-credential-store\tFile path to the git credential " \
 		    "store\n"
 	printf "%b" "--no-proxy-from-env\tDo not inherit proxy settings from " \
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/0a6be9b89cd05ae53ba30307fb9939f3fe99cf03.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 2/9] testsuite: Provide a new kas-based test container and wrapper script

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

This reworks the current docker-isar container to test-container.
Besides its new path and revisioning scheme, this container comes with
its own entrypoint script. The script still builds on top of kas'
container-entrypoint but avoids calling kas itself, thus entering a
bitbake env in shell mode.

When invoked via "kas-container shell", the new entrypoint builds up an
environment to run the avocado testsuite against the Isar repo that is
passed in, but that without modifying it. All testsuite results are
directed to the build/testsuite folder, including all modifications
which are kept in an overlay. This ensure a clean execution context for
the testsuite.

If the container is started without arguments, in continues to act as a
basis for gitlab-ci job executions.

The kas version used as baseline for test-container is now derived from
the one used for interactive building as well. This avoid surprises due
to the different versions.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 scripts/run-tests.sh                          | 18 ++++
 testsuite/dockerdata/Dockerfile               |  8 +-
 testsuite/dockerdata/README.md                | 12 ++-
 testsuite/dockerdata/build.sh                 | 14 +++
 .../dockerdata/test-container-entrypoint      | 95 +++++++++++++++++++
 testsuite/dockerdata/version                  |  1 +
 6 files changed, 142 insertions(+), 6 deletions(-)
 create mode 100755 scripts/run-tests.sh
 create mode 100755 testsuite/dockerdata/build.sh
 create mode 100755 testsuite/dockerdata/test-container-entrypoint
 create mode 100644 testsuite/dockerdata/version

diff --git a/scripts/run-tests.sh b/scripts/run-tests.sh
new file mode 100755
index 00000000..bce10d70
--- /dev/null
+++ b/scripts/run-tests.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# Script to run testsuite inside the official test-container
+#
+# Copyright (c) Siemens AG, 2026
+# SPDX-License-Identifier: MIT
+
+ISAR_DIR=$(readlink -f $(dirname $0)/..)
+
+TEST_CONTAINER_VERSION=$(cat ${ISAR_DIR}/testsuite/dockerdata/version)
+
+# The way to do this after kas 5.2:
+# export KAS_CONTAINER_IMAGE="${CONTAINER_BASENAME:-ghcr.io/ilbers/isar}/test-container:$TEST_CONTAINER_VERSION"
+#
+# For now:
+export KAS_CONTAINER_IMAGE_DISTRO="container:$TEST_CONTAINER_VERSION"
+export KAS_CONTAINER_IMAGE=${CONTAINER_BASENAME:-ghcr.io/ilbers/isar}/test
+
+${ISAR_DIR}/kas/kas-container --isar --repo-ro shell -c "$*"
diff --git a/testsuite/dockerdata/Dockerfile b/testsuite/dockerdata/Dockerfile
index 2a77798a..7b9cdaf6 100644
--- a/testsuite/dockerdata/Dockerfile
+++ b/testsuite/dockerdata/Dockerfile
@@ -1,4 +1,6 @@
-FROM ghcr.io/siemens/kas/kas-isar:<version>
+ARG KAS_VERSION=0
+
+FROM ghcr.io/siemens/kas/kas-isar:${KAS_VERSION}
 
 ARG TARGETPLATFORM
 ARG DEBIAN_FRONTEND=noninteractive
@@ -10,3 +12,7 @@ RUN sudo pip3 --proxy=$https_proxy install --break-system-packages avocado-frame
     sudo apt-get install -y -f --no-install-recommends python3-apt && \
     sudo apt-get clean && \
     sudo rm -rf $(pip3 cache dir) /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
+COPY testsuite/dockerdata/test-container-entrypoint /
+
+ENTRYPOINT ["/test-container-entrypoint"]
diff --git a/testsuite/dockerdata/README.md b/testsuite/dockerdata/README.md
index e6b76d25..54a78187 100644
--- a/testsuite/dockerdata/README.md
+++ b/testsuite/dockerdata/README.md
@@ -1,9 +1,12 @@
 # Creating image
 
+- Make sure `testsuite/dockerdata/version` is bumped for new images, also
+  after updating `kas/kas-container`.
+
+- Run:
+
 ```
-export version="4.4"
-cd <isar_dir>/testsuite/dockerdata
-sed "s/:<version>/:${version}/" Dockerfile | docker build -t ghcr.io/ilbers/docker-isar:${version} -
+testsuite/dockerdata/build.sh
 ```
 
 # Pushing the image to docker hub
@@ -13,8 +16,7 @@ sed "s/:<version>/:${version}/" Dockerfile | docker build -t ghcr.io/ilbers/dock
 - Use it for uploading docker image:
 
 ```
-export version="4.4"
-docker push ghcr.io/ilbers/docker-isar:${version}
+docker push ghcr.io/ilbers/isar/test-container:$(cat testsuite/dockerdata/version)
 ```
 
 - Make the uploaded package public
diff --git a/testsuite/dockerdata/build.sh b/testsuite/dockerdata/build.sh
new file mode 100755
index 00000000..7880d19f
--- /dev/null
+++ b/testsuite/dockerdata/build.sh
@@ -0,0 +1,14 @@
+#!/bin/sh
+# Build kas-based test-container
+#
+# Copyright (c) Siemens AG, 2026
+# SPDX-License-Identifier: MIT
+
+ISAR_DIR=$(readlink -f $(dirname $0)/../..)
+
+eval $(grep "^KAS_CONTAINER_SCRIPT_VERSION=" ${ISAR_DIR}/kas/kas-container)
+TEST_CONTAINER_VERSION=$(cat ${ISAR_DIR}/testsuite/dockerdata/version)
+
+docker build --file ${ISAR_DIR}/testsuite/dockerdata/Dockerfile \
+    --build-arg KAS_VERSION=$KAS_CONTAINER_SCRIPT_VERSION \
+    --tag ${CONTAINER_BASENAME:-ghcr.io/ilbers/isar}/test-container:$TEST_CONTAINER_VERSION ${ISAR_DIR}
diff --git a/testsuite/dockerdata/test-container-entrypoint b/testsuite/dockerdata/test-container-entrypoint
new file mode 100755
index 00000000..16f74124
--- /dev/null
+++ b/testsuite/dockerdata/test-container-entrypoint
@@ -0,0 +1,95 @@
+#!/bin/sh
+# Alternative entrypoint for the kas-based test-container
+#
+# Copyright (c) Siemens AG, 2026
+# SPDX-License-Identifier: MIT
+
+if [ $# -eq 0 ]; then
+	exec /container-entrypoint
+fi
+
+set -e
+
+# skip "shell [-c]"
+if [ $# -gt 1 ]; then
+	shift 2
+else
+	shift 1
+fi
+
+export args="--max-parallel-tasks=1 --disable-sysinfo"
+for arg in $*; do
+	case "$arg" in
+	--clean)
+		rm -rf /work/build/testsuite
+		;;
+	--debug)
+		export showopt="--show=app,test"
+		;;
+	--shell)
+		export start_shell=1
+		;;
+	--help)
+		cat <<EOF
+Usage: run-tests.sh [params] ...
+
+Supported parameters:
+  --clean		Purge results of previous test runs before starting.
+  --debug		Use '--show=app,test' log settings for avocado.
+  --shell		Drop into shell rather than starting tests.
+  --help		Show this help message.
+
+Any other parameters are passed to "avocado run". Its usage is:
+
+EOF
+		avocado run --help
+		exit 0
+		;;
+	*)
+		args="$args $arg"
+		;;
+	esac
+done
+
+case "$args" in
+*testsuite/citest.py*)
+	;;
+*)
+	args="$args testsuite/citest.py"
+	;;
+esac
+
+mkdir /isar
+
+# Reuse the existing entrypoint script, exploiting that entry without
+# parameters drops us into a shell, rather then calling kas.
+echo exit | /container-entrypoint
+
+gosu builder sh -c '
+set -e
+
+base_dir=/work/build/testsuite
+
+mkdir -p ${base_dir}/overlay/upper
+mkdir -p ${base_dir}/overlay/work
+sudo mount -t overlay overlay -o lowerdir=/repo,upperdir=${base_dir}/overlay/upper,workdir=${base_dir}/overlay/work /isar
+sudo mount -o bind ${base_dir} /isar/build
+
+mkdir -p "${HOME}"/.config/avocado
+cat <<EOF > "${HOME}"/.config/avocado/avocado.conf
+[datadir.paths]
+base_dir = ${base_dir}
+test_dir = ${base_dir}/tests
+data_dir = ${base_dir}/data
+logs_dir = ${base_dir}/job-results
+EOF
+
+cd /isar
+
+if [ "${start_shell}" = 1 ]; then
+	bash
+else
+	set -x
+	avocado ${showopt} run ${args}
+fi
+'
diff --git a/testsuite/dockerdata/version b/testsuite/dockerdata/version
new file mode 100644
index 00000000..d00491fd
--- /dev/null
+++ b/testsuite/dockerdata/version
@@ -0,0 +1 @@
+1
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/c8706d0bf1c6b023dcc63af39b12c1649100b8c4.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 3/9] testsuite: Install avocado-framework-plugin-varianter-yaml-to-mux in test-container

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

The ci_build.sh script does that for a while, update the test-container
accordingly

Fixes: 1f5ed58980a7 ("CI: Add make world test case")
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 testsuite/dockerdata/Dockerfile | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/testsuite/dockerdata/Dockerfile b/testsuite/dockerdata/Dockerfile
index 7b9cdaf6..ef11ba95 100644
--- a/testsuite/dockerdata/Dockerfile
+++ b/testsuite/dockerdata/Dockerfile
@@ -4,9 +4,12 @@ FROM ghcr.io/siemens/kas/kas-isar:${KAS_VERSION}
 
 ARG TARGETPLATFORM
 ARG DEBIAN_FRONTEND=noninteractive
+ARG AVOCADO_VERSION=103.0
 
 # Isar testsuite dependencies
-RUN sudo pip3 --proxy=$https_proxy install --break-system-packages avocado-framework==103.0 && \
+RUN sudo pip3 --proxy=$https_proxy install --break-system-packages \
+              avocado-framework==${AVOCADO_VERSION} \
+              avocado-framework-plugin-varianter-yaml-to-mux==${AVOCADO_VERSION} && \
     sudo apt-get update && \
     sudo apt-get install -y -f --no-install-recommends qemu-system ovmf && \
     sudo apt-get install -y -f --no-install-recommends python3-apt && \
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/d0dd51a44b63baa1c29986db4395d7711131ea3e.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 4/9] testsuite: Add squid to test-container

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

This is supposed to accelerate certain tests by caching downloads in
RAM.

We need special tuning for docker, see also
https://bugs.launchpad.net/ubuntu-docker-images/+bug/1978272

Start a minimal syslogd to avoid that squid chats to /dev/console.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 testsuite/dockerdata/Dockerfile                | 6 +++---
 testsuite/dockerdata/test-container-entrypoint | 4 ++++
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/testsuite/dockerdata/Dockerfile b/testsuite/dockerdata/Dockerfile
index ef11ba95..d4698aea 100644
--- a/testsuite/dockerdata/Dockerfile
+++ b/testsuite/dockerdata/Dockerfile
@@ -11,10 +11,10 @@ RUN sudo pip3 --proxy=$https_proxy install --break-system-packages \
               avocado-framework==${AVOCADO_VERSION} \
               avocado-framework-plugin-varianter-yaml-to-mux==${AVOCADO_VERSION} && \
     sudo apt-get update && \
-    sudo apt-get install -y -f --no-install-recommends qemu-system ovmf && \
-    sudo apt-get install -y -f --no-install-recommends python3-apt && \
+    sudo apt-get install -y -f --no-install-recommends qemu-system ovmf python3-apt busybox squid && \
     sudo apt-get clean && \
-    sudo rm -rf $(pip3 cache dir) /var/lib/apt/lists/* /tmp/* /var/tmp/*
+    sudo rm -rf $(pip3 cache dir) /var/lib/apt/lists/* /tmp/* /var/tmp/* && \
+    sudo sh -c 'echo "max_filedescriptors 1048576" > /etc/squid/conf.d/docker.conf'
 
 COPY testsuite/dockerdata/test-container-entrypoint /
 
diff --git a/testsuite/dockerdata/test-container-entrypoint b/testsuite/dockerdata/test-container-entrypoint
index 16f74124..e4714942 100755
--- a/testsuite/dockerdata/test-container-entrypoint
+++ b/testsuite/dockerdata/test-container-entrypoint
@@ -61,6 +61,10 @@ esac
 
 mkdir /isar
 
+busybox syslogd
+squid
+export http_proxy=http://localhost:3128
+
 # Reuse the existing entrypoint script, exploiting that entry without
 # parameters drops us into a shell, rather then calling kas.
 echo exit | /container-entrypoint
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/64fec185fc007870ccd9aa120ae5b3bd18b3f015.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 5/9] ci: Add github workflow for building and deploying test-container

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

Trigger a container build if the registery does not yet contain the
version of the test-container described by testsuite/dockerdata/version.

This obsoletes the need for manual build and deployment. Drop the
related README.md.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .github/workflows/main.yml     | 72 ++++++++++++++++++++++++++++++++++
 testsuite/dockerdata/README.md | 22 -----------
 2 files changed, 72 insertions(+), 22 deletions(-)
 create mode 100644 .github/workflows/main.yml
 delete mode 100644 testsuite/dockerdata/README.md

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
new file mode 100644
index 00000000..e9dd039e
--- /dev/null
+++ b/.github/workflows/main.yml
@@ -0,0 +1,72 @@
+# Copyright (c) Siemens AG, 2026
+# SPDX-License-Identifier: MIT
+
+name: CI
+
+on: [push]
+
+env:
+  CONTAINER_BASENAME: ${{ vars.CONTAINER_BASENAME || 'ghcr.io/ilbers/isar' }}
+
+jobs:
+  container:
+    name: Refresh test-container
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+      packages: write
+      contents: read
+      attestations: write
+      artifact-metadata: write
+    if: github.ref == 'refs/heads/next'
+    steps:
+      - name: Check out repo
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd  # v6.0.2
+
+      - name: Check for pre-existing container version
+        run: |
+          TEST_CONTAINER_VERSION=$(cat testsuite/dockerdata/version)
+          echo "TEST_CONTAINER_VERSION=$TEST_CONTAINER_VERSION" >> $GITHUB_ENV
+          if ! docker manifest inspect ${CONTAINER_BASENAME}/test-container:$TEST_CONTAINER_VERSION >/dev/null 2>&1; then
+              eval $(grep "^KAS_CONTAINER_SCRIPT_VERSION=" kas/kas-container)
+              echo "KAS_VERSION=$KAS_CONTAINER_SCRIPT_VERSION" >> $GITHUB_ENV
+              echo "BUILD_CONTAINER=true" >> $GITHUB_ENV
+          fi
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a  # v4.0.0
+        with:
+          platforms: linux/amd64,linux/arm64
+        if: ${{ env.BUILD_CONTAINER }}
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd  # v4.0.0
+        if: ${{ env.BUILD_CONTAINER }}
+      - name: Login to ghcr.io
+        uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2  # v4.0.0
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+        if: ${{ env.BUILD_CONTAINER }}
+
+      - name: Build and deploy container
+        uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294  #v7.0.0
+        id: push
+        with:
+          platforms: linux/amd64,linux/arm64
+          file: testsuite/dockerdata/Dockerfile
+          build-args: KAS_VERSION=${{ env.KAS_VERSION }}
+          provenance: false
+          outputs: type=registry
+          tags: |
+            ${{ env.CONTAINER_BASENAME }}/test-container:latest
+            ${{ env.CONTAINER_BASENAME }}/test-container:${{ env.TEST_CONTAINER_VERSION }}
+          annotations: ${{ env.DOCKER_METADATA_OUTPUT_ANNOTATIONS }}
+        if: ${{ env.BUILD_CONTAINER }}
+      - name: Attest container image
+        uses: actions/attest@59d89421af93a897026c735860bf21b6eb4f7b26  # v4.1.0
+        with:
+          subject-name: ${{ env.CONTAINER_BASENAME }}/test-container
+          subject-digest: ${{ steps.push.outputs.digest }}
+          push-to-registry: true
+        if: ${{ env.BUILD_CONTAINER }}
diff --git a/testsuite/dockerdata/README.md b/testsuite/dockerdata/README.md
deleted file mode 100644
index 54a78187..00000000
--- a/testsuite/dockerdata/README.md
+++ /dev/null
@@ -1,22 +0,0 @@
-# Creating image
-
-- Make sure `testsuite/dockerdata/version` is bumped for new images, also
-  after updating `kas/kas-container`.
-
-- Run:
-
-```
-testsuite/dockerdata/build.sh
-```
-
-# Pushing the image to docker hub
-
-- Configure github token (classic) with `write:packages` permissions.
-
-- Use it for uploading docker image:
-
-```
-docker push ghcr.io/ilbers/isar/test-container:$(cat testsuite/dockerdata/version)
-```
-
-- Make the uploaded package public
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/0d31a55008c43a72c8afcba35319ddb894c49012.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 6/9] ci: Switch gitlab-ci to new test-container

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

Should act as the old one but will receive the updates from now on.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .gitlab-ci.yml | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ce12e26c..d997f73c 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -35,8 +35,8 @@ variables:
     expire_in: 1 week
 
 
-.docker-isar: &docker-isar
-  image: ghcr.io/ilbers/docker-isar:4.8.2
+.test-container: &test-container
+  image: ghcr.io/ilbers/isar/test-container:1
 
 
 .use-default-image:
@@ -45,10 +45,10 @@ variables:
       if: $IMAGE
       when: never
 
-.use-docker-isar-image:
+.use-test-image:
   rules:
-    - &use-docker-isar-image
-      if: $IMAGE != 'docker-isar'
+    - &use-test-image
+      if: $IMAGE != 'test-container'
       when: never
 
 
@@ -87,28 +87,28 @@ full-ci:
 
 
 dev-ci-isar:
-  <<: *docker-isar
+  <<: *test-container
   <<: *common-build
   rules:
-    - *use-docker-isar-image
+    - *use-test-image
     - if: $TESTSUITE == 'dev'
   script:
     - scripts/ci_build.sh --sstate 1 -T dev
 
 fast-ci-isar:
-  <<: *docker-isar
+  <<: *test-container
   <<: *common-build
   rules:
-    - *use-docker-isar-image
+    - *use-test-image
     - if: $TESTSUITE == 'fast'
   script:
     - scripts/ci_build.sh --sstate 1 -T fast
 
 full-ci-isar:
-  <<: *docker-isar
+  <<: *test-container
   <<: *common-build
   rules:
-    - *use-docker-isar-image
+    - *use-test-image
     - if: $TESTSUITE == 'full'
   script:
     - scripts/ci_build.sh --sstate 1 -T full
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/65b9e614e97ae0f27bb5b6e37a44b4ababd7419b.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 7/9] CONTRIBUTING: Switch to new run-tests.sh

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

This massively simplifies the way to invoke the testsuite.

It also obsoletes the old ci_setup.sh script, so drop it.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 CONTRIBUTING.md     | 72 ++++++++++-----------------------------------
 scripts/ci_setup.sh | 32 --------------------
 2 files changed, 15 insertions(+), 89 deletions(-)
 delete mode 100755 scripts/ci_setup.sh

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 07621232..71edabde 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -77,63 +77,21 @@ Plan merges to `master` so that both fit the two-week window; short extensions s
 
    * Seemingly unaffected boards still build.
 
-   * It's highly suggested to test your patchset before submitting it to the mailing
-     by launching CI tests scripts. The procedure is described below:
-
-    ```
-    git clone https://github.com/siemens/kas
-    cat > kas.yml <<EOF
-    header:
-      version: 14
-    build_system: isar
-    defaults:
-      repos:
-        patches:
-          repo: isar
-    repos:
-      isar:
-        url: "http://github.com:/ilbers/isar"
-        branch: next
-        layers:
-          meta:
-          meta-isar:
-    EOF
-    kas/kas-container shell --command /work/isar/scripts/ci_setup.sh kas.yml
-    ```
-
-    In kas shell:
-
-    ```
-    cd /work/isar/testsuite
-    avocado run citest.py -t dev --max-parallel-tasks=1
-    ```
-
-    Your git-formatpatches may be listed in the `kas.yml` file as illustrated below:
-
-    ```
-    ...
-    repos:
-      isar:
-        url: "http://github.com:/ilbers/isar"
-        branch: next
-	patches:
-          0001:
-            path: /work/0001-my-contribution-to-isar.patch
-        layers:
-          meta:
-          meta-isar:
-    ```
-
-    Perform the above steps from a clean directory for your CI run to be as close as
-    possible to the environment that our project maintainers will be using. That
-    directory would contain: *.patch isar/ kas/ kas.yml
-
-    Be also mindful of community-provided resources such as deb.debian.org or
-    snapshot.debian.org and consider using a caching proxy in your setup to
-    reduce traffic as much as possible.
-
-    Active developers may request from maintainers an account on isar-build.org
-    to analyze CI logs or to launch their own CI builds there.
+   * It's highly suggested to test your patchset before submitting it to the
+     mailing list by launching CI tests scripts:
+
+     ```
+     scripts/run-tests -T dev
+     ```
+
+     Perform the above steps over the branch that contain your patches.
+
+     Be also mindful of community-provided resources such as deb.debian.org or
+     snapshot.debian.org and consider using a caching proxy in your setup to
+     reduce traffic as much as possible.
+
+     Active developers may request from maintainers an account on isar-build.org
+     to analyze CI logs or to launch their own CI builds there.
 
 4. Structure patches logically, in small increments.
 
diff --git a/scripts/ci_setup.sh b/scripts/ci_setup.sh
deleted file mode 100755
index f373d1c0..00000000
--- a/scripts/ci_setup.sh
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/bin/sh
-# Script to setup a container for CI builds
-#
-# Cedric Hombourger <cedric.hombourger@siemens.com>
-# Copyright (c) Siemens AG, 2025
-# SPDX-License-Identifier: MIT
-
-gpg_key=/etc/apt/trusted.gpg.d/debian-isar.gpg
-[ -f "${gpg_key}" ] || {
-    wget -q http://deb.isar-build.org/debian-isar.key -O- \
-    | gpg --dearmor \
-    | sudo dd of="${gpg_key}"
-}
-
-list=/etc/apt/sources.list.d/10-isar_build.list
-[ -f "${list}" ] || {
-    echo "deb [signed-by=/etc/apt/trusted.gpg.d/debian-isar.gpg] \
-        http://deb.isar-build.org/debian-isar bookworm-isar main" \
-    | sudo tee /etc/apt/sources.list.d/10-isar_build.list
-}
-
-tools="avocado qemu-system-aarch64 qemu-system-arm qemu-system-i386 qemu-system-x86_64"
-need_install=0
-for tool in ${tools}; do
-    which "${tool}" || need_install=1
-done
-[ "${need_install}" = "0" ] || {
-    sudo apt-get update
-    sudo apt-get install -y avocado qemu-system-arm qemu-system-x86
-}
-
-exec /container-entrypoint ${*}
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/0707415b2a9875c8101e609e215af9f2f20bdb53.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 8/9] testsuite: Drop KFAIL from trixie tests

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

Neither builds nor executions of trixie images must fail at this point.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 testsuite/citest.py | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index ab28b68e..b5cb75db 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -360,10 +360,7 @@ class CrossTest(CIBaseTest):
         ]
 
         self.init()
-        try:
-            self.perform_build_test(targets, cross=False)
-        except exceptions.TestFail:
-            self.cancel('KFAIL')
+        self.perform_build_test(targets, cross=False)
 
     def test_run_amd64_trixie(self):
         """
@@ -963,20 +960,14 @@ class NoCrossTest(CIBaseTest):
             'mc:starfive-visionfive2-trixie:isar-image-base',
         ]
 
-        try:
-            self.perform_build_test(targets, cross=False)
-        except exceptions.TestFail:
-            self.cancel('KFAIL')
+        self.perform_build_test(targets, cross=False)
 
     def test_run_arm_trixie(self):
         """
         :avocado: tags=startvm
         """
         self.init()
-        try:
-            self.vm_start('arm', 'trixie')
-        except exceptions.TestFail:
-            self.cancel('KFAIL')
+        self.vm_start('arm', 'trixie')
 
     def test_run_arm64_trixie(self):
         """
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/b4257505099d891bb2b571c8db64fcd0026af08e.1774254639.git.jan.kiszka%40siemens.com.

[PATCH 9/9] ci: Add github test job to github workflow

['Jan Kiszka' via isar-users]

From: Jan Kiszka <jan.kiszka@siemens.com>

Decompose the dev-tagged tests into two sets that can be run
independently and execute them as public github actions. This exercise
might be expendable to fast-tagged tests in the future, though likely
not to the full set due to limits of the public runners.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .github/workflows/main.yml | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index e9dd039e..1c54eba2 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -18,10 +18,10 @@ jobs:
       contents: read
       attestations: write
       artifact-metadata: write
-    if: github.ref == 'refs/heads/next'
     steps:
       - name: Check out repo
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd  # v6.0.2
+        if: github.ref == 'refs/heads/next'
 
       - name: Check for pre-existing container version
         run: |
@@ -32,6 +32,7 @@ jobs:
               echo "KAS_VERSION=$KAS_CONTAINER_SCRIPT_VERSION" >> $GITHUB_ENV
               echo "BUILD_CONTAINER=true" >> $GITHUB_ENV
           fi
+        if: github.ref == 'refs/heads/next'
 
       - name: Set up QEMU
         uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a  # v4.0.0
@@ -70,3 +71,31 @@ jobs:
           subject-digest: ${{ steps.push.outputs.digest }}
           push-to-registry: true
         if: ${{ env.BUILD_CONTAINER }}
+
+  test:
+    name: Run tests
+    runs-on: ubuntu-latest
+    needs: container
+    strategy:
+      matrix:
+        tests: [
+          {name: 'test_dev',
+           spec: 'testsuite/citest.py:test_dev$ testsuite/citest.py:test_dev_run_arm_'},
+          {name: 'test_dev_apps',
+           spec: 'testsuite/citest.py:test_dev_apps testsuite/citest.py:test_dev_rebuild testsuite/citest.py:test_dev_run_amd64_ testsuite/citest.py:test_dev_run_arm64_'}
+        ]
+    steps:
+      - name: Check out repo
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd  # v6.0.2
+      - name: Execute run-tests.sh
+        run: |
+          scripts/run-tests.sh --debug ${{ matrix.tests.spec }}
+      - name: Upload logs
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f  # v7.0.0
+        if: always()
+        with:
+          name: logs-${{ matrix.tests.name }}
+          path: |
+            build/testsuite/tmp*/work/*/*/*/temp
+            build/testsuite/job-results
+            build/testsuite/vm_start
-- 
2.47.3

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/d528838790f311583bbd3724d3e6b86e01e38e3f.1774254639.git.jan.kiszka%40siemens.com.

Re: [PATCH 7/9] CONTRIBUTING: Switch to new run-tests.sh

[Zhihang Wei]

On 3/23/26 09:30, 'Jan Kiszka' via isar-users wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> This massively simplifies the way to invoke the testsuite.
>
> It also obsoletes the old ci_setup.sh script, so drop it.
>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>   CONTRIBUTING.md     | 72 ++++++++++-----------------------------------
>   scripts/ci_setup.sh | 32 --------------------
>   2 files changed, 15 insertions(+), 89 deletions(-)
>   delete mode 100755 scripts/ci_setup.sh
>
> diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
> index 07621232..71edabde 100644
> --- a/CONTRIBUTING.md
> +++ b/CONTRIBUTING.md
> @@ -77,63 +77,21 @@ Plan merges to `master` so that both fit the two-week window; short extensions s
>   
>      * Seemingly unaffected boards still build.
>   
> -   * It's highly suggested to test your patchset before submitting it to the mailing
<snip>
> -    Active developers may request from maintainers an account on isar-build.org
> -    to analyze CI logs or to launch their own CI builds there.
> +   * It's highly suggested to test your patchset before submitting it to the
> +     mailing list by launching CI tests scripts:
> +
> +     ```
> +     scripts/run-tests -T dev
> +     ```
Here it should be "-t dev".

 From my point of view, it would be better to expose the avocado command
to users. They should be aware that avocado is used here and can more
easily understand the commands sent later to reproduce issues on CI.

Zhihang

> +     Perform the above steps over the branch that contain your patches.
> +
> +     Be also mindful of community-provided resources such as deb.debian.org or
> +     snapshot.debian.org and consider using a caching proxy in your setup to
> +     reduce traffic as much as possible.
> +
> +     Active developers may request from maintainers an account on isar-build.org
> +     to analyze CI logs or to launch their own CI builds there.
>   
>   4. Structure patches logically, in small increments.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/4ae04a9d-ba56-4036-b431-d851064ba893%40ilbers.de.

Re: [PATCH 7/9] CONTRIBUTING: Switch to new run-tests.sh

['Jan Kiszka' via isar-users]

On 26.03.26 18:01, Zhihang Wei wrote:
> 
> 
> On 3/23/26 09:30, 'Jan Kiszka' via isar-users wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> This massively simplifies the way to invoke the testsuite.
>>
>> It also obsoletes the old ci_setup.sh script, so drop it.
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>>   CONTRIBUTING.md     | 72 ++++++++++-----------------------------------
>>   scripts/ci_setup.sh | 32 --------------------
>>   2 files changed, 15 insertions(+), 89 deletions(-)
>>   delete mode 100755 scripts/ci_setup.sh
>>
>> diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
>> index 07621232..71edabde 100644
>> --- a/CONTRIBUTING.md
>> +++ b/CONTRIBUTING.md
>> @@ -77,63 +77,21 @@ Plan merges to `master` so that both fit the two-
>> week window; short extensions s
>>        * Seemingly unaffected boards still build.
>>   -   * It's highly suggested to test your patchset before submitting
>> it to the mailing
> <snip>
>> -    Active developers may request from maintainers an account on
>> isar-build.org
>> -    to analyze CI logs or to launch their own CI builds there.
>> +   * It's highly suggested to test your patchset before submitting it
>> to the
>> +     mailing list by launching CI tests scripts:
>> +
>> +     ```
>> +     scripts/run-tests -T dev
>> +     ```
> Here it should be "-t dev".
> 

Upps, yes. I originally exposed ci_build.sh API, but the realized that
avocado is the better interface. And the only way to run individual tests.

> From my point of view, it would be better to expose the avocado command
> to users. They should be aware that avocado is used here and can more
> easily understand the commands sent later to reproduce issues on CI.

That's what we do, see help. It's just that some required switches for
avocado are preconfigured, specifically the unfortunately serialization.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/8817cccb-c763-4190-a283-310483cef09e%40siemens.com.