(no subject)

[Uladzimir Bely]

Subject: [PATCH v3 0/5] Use `docker-isar` image for maintainer test jobs

This patchset allows to use own `docker-isar` image for maintainer
test jobs that better fits CI needs:

 - Avocado testsuite preinstalled;
 - Qemu-system packages for run tests preinstalled;
 - Potentially, avocado can be run directly, without ci_build.sh.

By default, `kas-isar` image is still used. To use `docker-isar` one,
IMAGE="docker-isar" variable should be set in the schedule settings.

Also, by specifying "TESTSUITE" variable it's now possible to filter
jobs (e.g. select "fast" or "full" or "dev") to be run in the pipeline.

Changes since v2:
- Rebased on latest next.
- Don't use confusing `latest` tag as ancor for image generation command.

Changes since v1:
- Rename patchset from `Switch to own docker-isar image`
- Update to the recent version of `kas-isar` images.
- `docker-isar` image is now based on `kas-isar` one.
- Use `docker-isar` naming in commit messages (in some places
'isar-docker' was used).
- Introduce TESTSUITE and IMAGE variables to control schedules.
Existing GitLab installations that don't use these variables continue
using `kas-isar`.


Uladzimir Bely (5):
  testsuite: Dockerfile for docker-isar image
  testsuite: Add README for docker-isar
  gitlab-ci: Switch to `rules` instead of `except` and `only`
  gitlab-ci.yml: Run 'fast' and 'full' only if specific variable set
  gitlab-ci.yml: Add jobs that run with custom docker-isar image

 .gitlab-ci.yml                     | 65 ++++++++++++++++++++++++++----
 testsuite/dockerdata/.dockerignore |  3 ++
 testsuite/dockerdata/Dockerfile    | 11 +++++
 testsuite/dockerdata/README.md     | 20 +++++++++
 4 files changed, 92 insertions(+), 7 deletions(-)
 create mode 100644 testsuite/dockerdata/.dockerignore
 create mode 100644 testsuite/dockerdata/Dockerfile
 create mode 100644 testsuite/dockerdata/README.md

-- 
2.20.1

[PATCH v3 1/5] testsuite: Dockerfile for docker-isar image

[Uladzimir Bely]

Add dockerfile for generating docker-isar image supposed to be used
in docker-based CI systems.

This image better fits testing when compared with 'kas-isar' one:
 - Avocado test framework preinstalled.
 - Qemu-system packages for running execution testcases preinstalled.
 - Direct avocado execution (without ci_build.sh) when necessary
   (e.g., for manual testing).

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 testsuite/dockerdata/.dockerignore |  2 ++
 testsuite/dockerdata/Dockerfile    | 11 +++++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 testsuite/dockerdata/.dockerignore
 create mode 100644 testsuite/dockerdata/Dockerfile

diff --git a/testsuite/dockerdata/.dockerignore b/testsuite/dockerdata/.dockerignore
new file mode 100644
index 00000000..6e19512a
--- /dev/null
+++ b/testsuite/dockerdata/.dockerignore
@@ -0,0 +1,2 @@
+.dockerignore
+Dockerfile
diff --git a/testsuite/dockerdata/Dockerfile b/testsuite/dockerdata/Dockerfile
new file mode 100644
index 00000000..9dd392de
--- /dev/null
+++ b/testsuite/dockerdata/Dockerfile
@@ -0,0 +1,11 @@
+FROM ghcr.io/siemens/kas/kas-isar:<version>
+
+ARG TARGETPLATFORM
+ARG DEBIAN_FRONTEND=noninteractive
+
+# Isar testsuite dependencies
+RUN sudo pip3 --proxy=$https_proxy install avocado-framework==100.1 && \
+    sudo apt-get update && \
+    sudo apt-get install -y -f --no-install-recommends qemu-system ovmf && \
+    sudo apt-get clean && \
+    sudo rm -rf $(pip3 cache dir) /var/lib/apt/lists/* /tmp/* /var/tmp/* 
-- 
2.20.1

[PATCH v3 2/5] testsuite: Add README for docker-isar

[Uladzimir Bely]

Document `docker-isar` image prepare and publish steps.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 testsuite/dockerdata/.dockerignore |  1 +
 testsuite/dockerdata/README.md     | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+)
 create mode 100644 testsuite/dockerdata/README.md

diff --git a/testsuite/dockerdata/.dockerignore b/testsuite/dockerdata/.dockerignore
index 6e19512a..d06a06ab 100644
--- a/testsuite/dockerdata/.dockerignore
+++ b/testsuite/dockerdata/.dockerignore
@@ -1,2 +1,3 @@
 .dockerignore
 Dockerfile
+README.md
diff --git a/testsuite/dockerdata/README.md b/testsuite/dockerdata/README.md
new file mode 100644
index 00000000..49d2d6eb
--- /dev/null
+++ b/testsuite/dockerdata/README.md
@@ -0,0 +1,20 @@
+# Creating image
+
+```
+export version="3.2.3"
+cd <isar_dir>/testsuite/dockerdata
+sed "s/:<version>/:${version}/" Dockerfile | docker build -t ghcr.io/ilbers/docker-isar:${version} -
+```
+
+# Pushing the image to docker hub
+
+- Configure github token (classic) with `write:packages` permissions.
+
+- Use it for uploading docker image:
+
+```
+export version="3.2.3"
+docker push ghcr.io/ilbers/docker-isar:${version}
+```
+
+- Make the uploaded package public 
-- 
2.20.1

[PATCH v3 3/5] gitlab-ci: Switch to `rules` instead of `except` and `only`

[Uladzimir Bely]

Filters that use `rules` are more configurable and allow using
several rules in the chain.

Also, put `image` under `default` section instead of deprecated
global declaration, with image version update to the latest one.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 .gitlab-ci.yml | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ded8469b..b700015c 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,4 +1,5 @@
-image: ghcr.io/siemens/kas/kas-isar:3.2.3
+default:
+  image: ghcr.io/siemens/kas/kas-isar:3.2.3
 
 variables:
   GIT_STRATEGY: clone
@@ -24,22 +25,22 @@ variables:
 
 dev-ci:
   <<: *common-build
-  except:
-    - schedules
+  rules:
+    - if: $CI_PIPELINE_SOURCE != 'schedule'
   script:
     - scripts/ci_build.sh -T dev
 
 fast-ci:
   <<: *common-build
-  only:
-    - schedules
+  rules:
+    - if: $CI_PIPELINE_SOURCE == 'schedule'
   script:
     - scripts/ci_build.sh -T fast
 
 full-ci:
   <<: *common-build
-  only:
-    - schedules
+  rules:
+    - if: $CI_PIPELINE_SOURCE == 'schedule'
   script:
     - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
     - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
-- 
2.20.1

[PATCH v3 4/5] gitlab-ci.yml: Run 'fast' and 'full' only if specific variable set

[Uladzimir Bely]

Before the patch, any schedule run both 'fast' and 'full' jobs.
Now, to filter the required job, we should now specify `TESTSUITE`
variable. If not set, schedule will run only 'full' for compatibility
with current CI setups.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 .gitlab-ci.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b700015c..5117519a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -26,21 +26,22 @@ variables:
 dev-ci:
   <<: *common-build
   rules:
-    - if: $CI_PIPELINE_SOURCE != 'schedule'
+    - if: $TESTSUITE == 'dev' || $CI_PIPELINE_SOURCE != 'schedule'
   script:
     - scripts/ci_build.sh -T dev
 
 fast-ci:
   <<: *common-build
   rules:
-    - if: $CI_PIPELINE_SOURCE == 'schedule'
+    - if: $TESTSUITE == 'fast'
   script:
     - scripts/ci_build.sh -T fast
 
 full-ci:
   <<: *common-build
   rules:
-    - if: $CI_PIPELINE_SOURCE == 'schedule'
+    - if: $TESTSUITE == null && $CI_PIPELINE_SOURCE == 'schedule'
+    - if: $TESTSUITE == 'full'
   script:
     - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
     - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
-- 
2.20.1

[PATCH v3 5/5] gitlab-ci.yml: Add jobs that run with custom docker-isar image

[Uladzimir Bely]

This adds possibility to select between `kas-isar` image for building
the project and `docker-isar` one.

By default, if variable `IMAGE` is not set in schedule properties, we
continue using `kas-isar`. Specifying `IMAGE = docker-isar` swtiches CI
to `docker-isar` image that has "avocado" and "qemu-system" packages
preinstalled.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 .gitlab-ci.yml | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5117519a..7e28a234 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -23,9 +23,28 @@ variables:
     when: always
     expire_in: 1 week
 
+
+.docker-isar: &docker-isar
+  image: ghcr.io/ilbers/docker-isar:3.2.3
+
+
+.use-default-image:
+  rules:
+    - &use-default-image
+      if: $IMAGE
+      when: never
+
+.use-docker-isar-image:
+  rules:
+    - &use-docker-isar-image
+      if: $IMAGE != 'docker-isar'
+      when: never
+
+
 dev-ci:
   <<: *common-build
   rules:
+    - *use-default-image
     - if: $TESTSUITE == 'dev' || $CI_PIPELINE_SOURCE != 'schedule'
   script:
     - scripts/ci_build.sh -T dev
@@ -33,6 +52,7 @@ dev-ci:
 fast-ci:
   <<: *common-build
   rules:
+    - *use-default-image
     - if: $TESTSUITE == 'fast'
   script:
     - scripts/ci_build.sh -T fast
@@ -40,6 +60,7 @@ fast-ci:
 full-ci:
   <<: *common-build
   rules:
+    - *use-default-image
     - if: $TESTSUITE == null && $CI_PIPELINE_SOURCE == 'schedule'
     - if: $TESTSUITE == 'full'
   script:
@@ -52,3 +73,31 @@ full-ci:
     key: "$CI_COMMIT_REF_SLUG"
     paths:
       - .CI_COMMIT_SHA
+
+
+dev-ci-isar:
+  <<: *docker-isar
+  <<: *common-build
+  rules:
+    - *use-docker-isar-image
+    - if: $TESTSUITE == 'dev'
+  script:
+    - scripts/ci_build.sh -T dev
+
+fast-ci-isar:
+  <<: *docker-isar
+  <<: *common-build
+  rules:
+    - *use-docker-isar-image
+    - if: $TESTSUITE == 'fast'
+  script:
+    - scripts/ci_build.sh -T fast
+
+full-ci-isar:
+  <<: *docker-isar
+  <<: *common-build
+  rules:
+    - *use-docker-isar-image
+    - if: $TESTSUITE == 'full'
+  script:
+    - scripts/ci_build.sh -T full
-- 
2.20.1

Re:

[Uladzimir Bely]

On Fri, 2023-05-26 at 07:23 +0200, Uladzimir Bely wrote:
> Subject: [PATCH v3 0/5] Use `docker-isar` image for maintainer test
> jobs

Sorry, moved e-mail subject the body by mistake...

Re: [PATCH v3 0/5] Use `docker-isar` image for maintainer test jobs

[Uladzimir Bely]

On Fri, 2023-05-26 at 07:23 +0200, Uladzimir Bely wrote:
> Subject: [PATCH v3 0/5] Use `docker-isar` image for maintainer test
> jobs
> 
> This patchset allows to use own `docker-isar` image for maintainer
> test jobs that better fits CI needs:
> 
>  - Avocado testsuite preinstalled;
>  - Qemu-system packages for run tests preinstalled;
>  - Potentially, avocado can be run directly, without ci_build.sh.
> 
> By default, `kas-isar` image is still used. To use `docker-isar` one,
> IMAGE="docker-isar" variable should be set in the schedule settings.
> 
> Also, by specifying "TESTSUITE" variable it's now possible to filter
> jobs (e.g. select "fast" or "full" or "dev") to be run in the
> pipeline.
> 
> Changes since v2:
> - Rebased on latest next.
> - Don't use confusing `latest` tag as ancor for image generation
> command.
> 
> Changes since v1:
> - Rename patchset from `Switch to own docker-isar image`
> - Update to the recent version of `kas-isar` images.
> - `docker-isar` image is now based on `kas-isar` one.
> - Use `docker-isar` naming in commit messages (in some places
> 'isar-docker' was used).
> - Introduce TESTSUITE and IMAGE variables to control schedules.
> Existing GitLab installations that don't use these variables continue
> using `kas-isar`.
> 
> 
> Uladzimir Bely (5):
>   testsuite: Dockerfile for docker-isar image
>   testsuite: Add README for docker-isar
>   gitlab-ci: Switch to `rules` instead of `except` and `only`
>   gitlab-ci.yml: Run 'fast' and 'full' only if specific variable set
>   gitlab-ci.yml: Add jobs that run with custom docker-isar image
> 
>  .gitlab-ci.yml                     | 65 ++++++++++++++++++++++++++--
> --
>  testsuite/dockerdata/.dockerignore |  3 ++
>  testsuite/dockerdata/Dockerfile    | 11 +++++
>  testsuite/dockerdata/README.md     | 20 +++++++++
>  4 files changed, 92 insertions(+), 7 deletions(-)
>  create mode 100644 testsuite/dockerdata/.dockerignore
>  create mode 100644 testsuite/dockerdata/Dockerfile
>  create mode 100644 testsuite/dockerdata/README.md
> 
> -- 
> 2.20.1
> 

Applied to next.