[PATCH v2 1/2] CI: install avocado in venv if host does not have it

[PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Anton Mikanovich]

From: Henning Schild <henning.schild@siemens.com>

Avocado packages are currently not available in Debian. Use virtualenv
and pip for sites that do not provide avocado as part of execution
environment. We expect the execution environment to be Debian-based and
to allow us to sudo (true for kas-isar container).

Use version 88.0 as the latest working one from pip.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 scripts/ci_build.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 4534957..32054c4 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -14,6 +14,17 @@ export PATH=$PATH:/sbin
 # Go to Isar root
 cd "$(dirname "$0")/.."
 
+# install avocado in virtualenv in case it is not there already
+if ! command -v avocado > /dev/null; then
+    sudo apt-get update -qq
+    sudo apt-get install -y virtualenv
+    rm -rf /tmp/avocado_venv
+    virtualenv --python python3 /tmp/avocado_venv
+    source /tmp/avocado_venv/bin/activate
+    # The latest working version of Avocado framework
+    pip install avocado-framework==88.0
+fi
+
 # Get Avocado build tests path
 BUILD_TEST_DIR="$(pwd)/testsuite/build_test"
 
-- 
2.7.4

[PATCH v2 2/2] CI: export avocado junit as gitlab report

[Anton Mikanovich]

From: Henning Schild <henning.schild@siemens.com>

Having the artifact as "report" makes sure we get nice rendering in the
gitlab web UI. For that to really work we want the artifacts no matter what
the return will be.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 .gitlab-ci.yml | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fad4210..1693a8a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -16,7 +16,10 @@ variables:
     paths:
       - build/tmp/work/*/*/*/temp
       - build/job-results
-    when: on_failure
+    reports:
+      junit:
+        - build/job-results/job*/results.xml
+    when: always
     expire_in: 1 week
 
 fast-ci:
-- 
2.7.4

Re: [PATCH v2 2/2] CI: export avocado junit as gitlab report

[Anton Mikanovich]

On 20.10.21 13:03, Anton Mikanovich wrote:
> From: Henning Schild <henning.schild@siemens.com>
>
> Having the artifact as "report" makes sure we get nice rendering in the
> gitlab web UI. For that to really work we want the artifacts no matter what
> the return will be.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   .gitlab-ci.yml | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fad4210..1693a8a 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -16,7 +16,10 @@ variables:
>       paths:
>         - build/tmp/work/*/*/*/temp
>         - build/job-results
> -    when: on_failure
> +    reports:
> +      junit:
> +        - build/job-results/job*/results.xml
> +    when: always
>       expire_in: 1 week
>   
>   fast-ci:

Applied to next.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Anton Mikanovich]

On 20.10.21 13:03, Anton Mikanovich wrote:
> From: Henning Schild <henning.schild@siemens.com>
>
> Avocado packages are currently not available in Debian. Use virtualenv
> and pip for sites that do not provide avocado as part of execution
> environment. We expect the execution environment to be Debian-based and
> to allow us to sudo (true for kas-isar container).
>
> Use version 88.0 as the latest working one from pip.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>

Packages subprocess32 and avocado-framework-plugin-varianter-yaml-to-mux 
are not used in the current implementation, so we need to install only 
avocado itself.
The FileNotFoundError: whiteboard issue is caused by the new avocado 
version provided by pip, the latest working one is 88.0. Deeper 
investigation is in progress.

We are going to merge this today.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Anton Mikanovich]

On 20.10.21 13:03, Anton Mikanovich wrote:
> From: Henning Schild <henning.schild@siemens.com>
>
> Avocado packages are currently not available in Debian. Use virtualenv
> and pip for sites that do not provide avocado as part of execution
> environment. We expect the execution environment to be Debian-based and
> to allow us to sudo (true for kas-isar container).
>
> Use version 88.0 as the latest working one from pip.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>   scripts/ci_build.sh | 11 +++++++++++
>   1 file changed, 11 insertions(+)
>
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index 4534957..32054c4 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -14,6 +14,17 @@ export PATH=$PATH:/sbin
>   # Go to Isar root
>   cd "$(dirname "$0")/.."
>   
> +# install avocado in virtualenv in case it is not there already
> +if ! command -v avocado > /dev/null; then
> +    sudo apt-get update -qq
> +    sudo apt-get install -y virtualenv
> +    rm -rf /tmp/avocado_venv
> +    virtualenv --python python3 /tmp/avocado_venv
> +    source /tmp/avocado_venv/bin/activate
> +    # The latest working version of Avocado framework
> +    pip install avocado-framework==88.0
> +fi
> +
>   # Get Avocado build tests path
>   BUILD_TEST_DIR="$(pwd)/testsuite/build_test"
>   

Applied to next.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Henning Schild]

Am Wed, 20 Oct 2021 19:10:48 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> On 20.10.21 13:03, Anton Mikanovich wrote:
> > From: Henning Schild <henning.schild@siemens.com>
> >
> > Avocado packages are currently not available in Debian. Use
> > virtualenv and pip for sites that do not provide avocado as part of
> > execution environment. We expect the execution environment to be
> > Debian-based and to allow us to sudo (true for kas-isar container).
> >
> > Use version 88.0 as the latest working one from pip.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> > ---
> >   scripts/ci_build.sh | 11 +++++++++++
> >   1 file changed, 11 insertions(+)
> >
> > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> > index 4534957..32054c4 100755
> > --- a/scripts/ci_build.sh
> > +++ b/scripts/ci_build.sh
> > @@ -14,6 +14,17 @@ export PATH=$PATH:/sbin
> >   # Go to Isar root
> >   cd "$(dirname "$0")/.."
> >   
> > +# install avocado in virtualenv in case it is not there already
> > +if ! command -v avocado > /dev/null; then
> > +    sudo apt-get update -qq
> > +    sudo apt-get install -y virtualenv
> > +    rm -rf /tmp/avocado_venv
> > +    virtualenv --python python3 /tmp/avocado_venv
> > +    source /tmp/avocado_venv/bin/activate
> > +    # The latest working version of Avocado framework
> > +    pip install avocado-framework==88.0
> > +fi
> > +
> >   # Get Avocado build tests path
> >   BUILD_TEST_DIR="$(pwd)/testsuite/build_test"
> >     
> 
> Applied to next.


So the fix was to pin down the version? That will cause maint headache
... quality tool and package manager!

Henning

Re: [PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Henning Schild]

Am Wed, 20 Oct 2021 18:32:37 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Am Wed, 20 Oct 2021 19:10:48 +0300
> schrieb Anton Mikanovich <amikan@ilbers.de>:
> 
> > On 20.10.21 13:03, Anton Mikanovich wrote:  
> > > From: Henning Schild <henning.schild@siemens.com>
> > >
> > > Avocado packages are currently not available in Debian. Use
> > > virtualenv and pip for sites that do not provide avocado as part
> > > of execution environment. We expect the execution environment to
> > > be Debian-based and to allow us to sudo (true for kas-isar
> > > container).
> > >
> > > Use version 88.0 as the latest working one from pip.
> > >
> > > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > > Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> > > ---
> > >   scripts/ci_build.sh | 11 +++++++++++
> > >   1 file changed, 11 insertions(+)
> > >
> > > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> > > index 4534957..32054c4 100755
> > > --- a/scripts/ci_build.sh
> > > +++ b/scripts/ci_build.sh
> > > @@ -14,6 +14,17 @@ export PATH=$PATH:/sbin
> > >   # Go to Isar root
> > >   cd "$(dirname "$0")/.."
> > >   
> > > +# install avocado in virtualenv in case it is not there already
> > > +if ! command -v avocado > /dev/null; then
> > > +    sudo apt-get update -qq
> > > +    sudo apt-get install -y virtualenv
> > > +    rm -rf /tmp/avocado_venv
> > > +    virtualenv --python python3 /tmp/avocado_venv
> > > +    source /tmp/avocado_venv/bin/activate
> > > +    # The latest working version of Avocado framework
> > > +    pip install avocado-framework==88.0
> > > +fi
> > > +
> > >   # Get Avocado build tests path
> > >   BUILD_TEST_DIR="$(pwd)/testsuite/build_test"
> > >       
> > 
> > Applied to next.  
> 
> 
> So the fix was to pin down the version? That will cause maint headache
> ... quality tool and package manager!

But thanks for the quick fix! I gave this to our gitlab to try and
digest. Having a testsuite is a win for sure, even if i still tend to
rant about the choice ... bats or pytest would have been OK as well and
readily available via debian.

Henning

> Henning
> 

Re: [PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Baurzhan Ismagulov]

On Wed, Oct 20, 2021 at 06:38:01PM +0200, Henning Schild wrote:
> > So the fix was to pin down the version? That will cause maint headache
> > ... quality tool and package manager!
> 
> But thanks for the quick fix! I gave this to our gitlab to try and
> digest. Having a testsuite is a win for sure, even if i still tend to
> rant about the choice ... bats or pytest would have been OK as well and
> readily available via debian.

Welcome Henning,

Regarding the pinning, Anton has already mentioned that we are analyzing the
issue. The plan is to update to the latest stable version regularly. We still
want pinning to avoid surprises, just like with bitbake and wic.

Regarding pip, it isn't our choice :) . Debian packaging is available from
http://deb.isar-build.org/. We have submitted Debian Policy updates to Avocado
and are in contact with Debian developers regarding inclusion in upstream
Debian.

Regarding Avocado itself, we had discussed that in [1]. Pytest is good for unit
testing; Avocado is more suitable for our scenarios. E.g., RedHat uses it for
Fedora. Also, people want to migrate to Avocado citing flexibility limitations
with pytest.

All in all, I think Avocado is a good choice and is worth the hassle of
upstreaming into Debian.

1. https://groups.google.com/g/isar-users/c/o3tJjEhe4m4/m/D7gA0SmJAQAJ

With kind regards,
Baurzhan.

Re: [PATCH v2 1/2] CI: install avocado in venv if host does not have it

[Henning Schild]

Am Thu, 21 Oct 2021 12:39:30 +0200
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Wed, Oct 20, 2021 at 06:38:01PM +0200, Henning Schild wrote:
> > > So the fix was to pin down the version? That will cause maint
> > > headache ... quality tool and package manager!  
> > 
> > But thanks for the quick fix! I gave this to our gitlab to try and
> > digest. Having a testsuite is a win for sure, even if i still tend
> > to rant about the choice ... bats or pytest would have been OK as
> > well and readily available via debian.  
> 
> Welcome Henning,
> 
> Regarding the pinning, Anton has already mentioned that we are
> analyzing the issue. The plan is to update to the latest stable
> version regularly. We still want pinning to avoid surprises, just
> like with bitbake and wic.
> 
> Regarding pip, it isn't our choice :) . Debian packaging is available
> from http://deb.isar-build.org/. We have submitted Debian Policy
> updates to Avocado and are in contact with Debian developers
> regarding inclusion in upstream Debian.

The virtualenv code already assumes a debian host, so it could be
changed to in fact use that package and not a venv. That would take
care of the pinning to only one stream to deliver. Should i write a
patch?

> Regarding Avocado itself, we had discussed that in [1]. Pytest is
> good for unit testing; Avocado is more suitable for our scenarios.
> E.g., RedHat uses it for Fedora. Also, people want to migrate to
> Avocado citing flexibility limitations with pytest.
> 
> All in all, I think Avocado is a good choice and is worth the hassle
> of upstreaming into Debian.
> 
> 1. https://groups.google.com/g/isar-users/c/o3tJjEhe4m4/m/D7gA0SmJAQAJ

It is a chicken-egg problem. But if we want to have avocado and push
for it to become part of debian that is fine with me. Did not look into
the benefits after seeing the problems. Plus i kind of still feel the
choice was never discussed on the list, but maybe i did not follow
close enough.

I will stop the avocado-rants and focus future rants on multiconfig ;)

regards,
Henning

> With kind regards,
> Baurzhan.
>