[PATCH] CI: Fix paths for the latest Avocado version

[PATCH] CI: Fix paths for the latest Avocado version

[Anton Mikanovich]

We can't use relative path for logdir in Avocado 89.0+ because it fails with:

FileNotFoundError: [Errno 2] No such file or directory: '.../whiteboard'

Move to absolute paths to be able to use the latest version.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 scripts/ci_build.sh | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 32054c4a..5ff455e3 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -21,8 +21,7 @@ if ! command -v avocado > /dev/null; then
     rm -rf /tmp/avocado_venv
     virtualenv --python python3 /tmp/avocado_venv
     source /tmp/avocado_venv/bin/activate
-    # The latest working version of Avocado framework
-    pip install avocado-framework==88.0
+    pip install avocado-framework
 fi
 
 # Get Avocado build tests path
@@ -121,10 +120,10 @@ set -x
 mkdir -p .config/avocado
 cat <<EOF > .config/avocado/avocado.conf
 [datadir.paths]
-base_dir = $BUILD_DIR/
-test_dir = $BUILD_DIR/tests
-data_dir = $BUILD_DIR/data
-logs_dir = $BUILD_DIR/job-results
+base_dir = $(realpath $BUILD_DIR)/
+test_dir = $(realpath $BUILD_DIR)/tests
+data_dir = $(realpath $BUILD_DIR)/data
+logs_dir = $(realpath $BUILD_DIR)/job-results
 EOF
 export VIRTUAL_ENV="./"
 
-- 
2.20.1

[PATCH] CI: install avocado debian packages from isar-build.org

[Henning Schild]

Might be better than doing it with pip. So we only need to maintain one
path while waiting for upsteam debian to ship packages.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 scripts/ci_build.sh | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 5ff455e3d77b..ddec9e38cb87 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -14,14 +14,16 @@ export PATH=$PATH:/sbin
 # Go to Isar root
 cd "$(dirname "$0")/.."
 
-# install avocado in virtualenv in case it is not there already
+# install avocado in from deb.isar-build.org in case it is not there already
 if ! command -v avocado > /dev/null; then
     sudo apt-get update -qq
-    sudo apt-get install -y virtualenv
-    rm -rf /tmp/avocado_venv
-    virtualenv --python python3 /tmp/avocado_venv
-    source /tmp/avocado_venv/bin/activate
-    pip install avocado-framework
+    sudo apt-get install gnupg2 lsb-release curl -y
+    codename=$( lsb_release -cs )
+    curl http://deb.isar-build.org/debian-isar.key --output /tmp/isar-gpg.pub
+    sudo apt-key add /tmp/isar-gpg.pub
+    echo "deb http://deb.isar-build.org/debian-isar $codename-isar main" | sudo tee /etc/apt/sources.list.d/isar.list > /dev/null
+    sudo apt-get update -qq
+    sudo apt-get install -y avocado
 fi
 
 # Get Avocado build tests path
-- 
2.32.0

Re: [PATCH] CI: install avocado debian packages from isar-build.org

[Henning Schild]

This patch did not work on next, but will likely work on top of
"CI: Fix paths for the latest Avocado version"

Seems like the ilbers runner does not use that package?

Henning

Am Fri, 22 Oct 2021 09:57:37 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Might be better than doing it with pip. So we only need to maintain
> one path while waiting for upsteam debian to ship packages.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  scripts/ci_build.sh | 14 ++++++++------
>  1 file changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index 5ff455e3d77b..ddec9e38cb87 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -14,14 +14,16 @@ export PATH=$PATH:/sbin
>  # Go to Isar root
>  cd "$(dirname "$0")/.."
>  
> -# install avocado in virtualenv in case it is not there already
> +# install avocado in from deb.isar-build.org in case it is not there
> already if ! command -v avocado > /dev/null; then
>      sudo apt-get update -qq
> -    sudo apt-get install -y virtualenv
> -    rm -rf /tmp/avocado_venv
> -    virtualenv --python python3 /tmp/avocado_venv
> -    source /tmp/avocado_venv/bin/activate
> -    pip install avocado-framework
> +    sudo apt-get install gnupg2 lsb-release curl -y
> +    codename=$( lsb_release -cs )
> +    curl http://deb.isar-build.org/debian-isar.key --output
> /tmp/isar-gpg.pub
> +    sudo apt-key add /tmp/isar-gpg.pub
> +    echo "deb http://deb.isar-build.org/debian-isar $codename-isar
> main" | sudo tee /etc/apt/sources.list.d/isar.list > /dev/null
> +    sudo apt-get update -qq
> +    sudo apt-get install -y avocado
>  fi
>  
>  # Get Avocado build tests path

Re: [PATCH] CI: install avocado debian packages from isar-build.org

[Henning Schild]

Am Fri, 22 Oct 2021 09:57:37 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Might be better than doing it with pip. So we only need to maintain
> one path while waiting for upsteam debian to ship packages.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  scripts/ci_build.sh | 14 ++++++++------
>  1 file changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index 5ff455e3d77b..ddec9e38cb87 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -14,14 +14,16 @@ export PATH=$PATH:/sbin
>  # Go to Isar root
>  cd "$(dirname "$0")/.."
>  
> -# install avocado in virtualenv in case it is not there already
> +# install avocado in from deb.isar-build.org in case it is not there
> already if ! command -v avocado > /dev/null; then
>      sudo apt-get update -qq
> -    sudo apt-get install -y virtualenv
> -    rm -rf /tmp/avocado_venv
> -    virtualenv --python python3 /tmp/avocado_venv
> -    source /tmp/avocado_venv/bin/activate
> -    pip install avocado-framework
> +    sudo apt-get install gnupg2 lsb-release curl -y
> +    codename=$( lsb_release -cs )
> +    curl http://deb.isar-build.org/debian-isar.key --output
> /tmp/isar-gpg.pub
> +    sudo apt-key add /tmp/isar-gpg.pub
> +    echo "deb http://deb.isar-build.org/debian-isar $codename-isar
> main" | sudo tee /etc/apt/sources.list.d/isar.list > /dev/null
> +    sudo apt-get update -qq
> +    sudo apt-get install -y avocado

Isar can also be used to build a repo with a bunch of packages, not a
full image. If we had a recipe for avocado we could in fact build the
avocado package on demand and not fetch it from that mirror.
Would make a nice demo of building packages with isar, but would also
end up slow and complicated in a way.

Henning

>  fi
>  
>  # Get Avocado build tests path

Re: [PATCH] CI: install avocado debian packages from isar-build.org

[Anton Mikanovich]

22.10.2021 10:57, Henning Schild wrote:
> Might be better than doing it with pip. So we only need to maintain one
> path while waiting for upsteam debian to ship packages.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   scripts/ci_build.sh | 14 ++++++++------
>   1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index 5ff455e3d77b..ddec9e38cb87 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -14,14 +14,16 @@ export PATH=$PATH:/sbin
>   # Go to Isar root
>   cd "$(dirname "$0")/.."
>   
> -# install avocado in virtualenv in case it is not there already
> +# install avocado in from deb.isar-build.org in case it is not there already
>   if ! command -v avocado > /dev/null; then
>       sudo apt-get update -qq
> -    sudo apt-get install -y virtualenv
> -    rm -rf /tmp/avocado_venv
> -    virtualenv --python python3 /tmp/avocado_venv
> -    source /tmp/avocado_venv/bin/activate
> -    pip install avocado-framework
> +    sudo apt-get install gnupg2 lsb-release curl -y
> +    codename=$( lsb_release -cs )
> +    curl http://deb.isar-build.org/debian-isar.key --output /tmp/isar-gpg.pub
> +    sudo apt-key add /tmp/isar-gpg.pub
> +    echo "deb http://deb.isar-build.org/debian-isar $codename-isar main" | sudo tee /etc/apt/sources.list.d/isar.list > /dev/null
> +    sudo apt-get update -qq
> +    sudo apt-get install -y avocado
>   fi
>   
>   # Get Avocado build tests path

What is the main difference of this patch and `[PATCH v10 17/17] 
ci_build: Install Avocado if needed` (besides distro detection and apt-get)?
Looks pretty much like the way we proposed originally.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH] CI: install avocado debian packages from isar-build.org

[Henning Schild]

Am Tue, 26 Oct 2021 10:37:00 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> 22.10.2021 10:57, Henning Schild wrote:
> > Might be better than doing it with pip. So we only need to maintain
> > one path while waiting for upsteam debian to ship packages.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >   scripts/ci_build.sh | 14 ++++++++------
> >   1 file changed, 8 insertions(+), 6 deletions(-)
> >
> > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> > index 5ff455e3d77b..ddec9e38cb87 100755
> > --- a/scripts/ci_build.sh
> > +++ b/scripts/ci_build.sh
> > @@ -14,14 +14,16 @@ export PATH=$PATH:/sbin
> >   # Go to Isar root
> >   cd "$(dirname "$0")/.."
> >   
> > -# install avocado in virtualenv in case it is not there already
> > +# install avocado in from deb.isar-build.org in case it is not
> > there already if ! command -v avocado > /dev/null; then
> >       sudo apt-get update -qq
> > -    sudo apt-get install -y virtualenv
> > -    rm -rf /tmp/avocado_venv
> > -    virtualenv --python python3 /tmp/avocado_venv
> > -    source /tmp/avocado_venv/bin/activate
> > -    pip install avocado-framework
> > +    sudo apt-get install gnupg2 lsb-release curl -y
> > +    codename=$( lsb_release -cs )
> > +    curl http://deb.isar-build.org/debian-isar.key --output
> > /tmp/isar-gpg.pub
> > +    sudo apt-key add /tmp/isar-gpg.pub
> > +    echo "deb http://deb.isar-build.org/debian-isar $codename-isar
> > main" | sudo tee /etc/apt/sources.list.d/isar.list > /dev/null
> > +    sudo apt-get update -qq
> > +    sudo apt-get install -y avocado
> >   fi
> >   
> >   # Get Avocado build tests path  
> 
> What is the main difference of this patch and `[PATCH v10 17/17] 
> ci_build: Install Avocado if needed` (besides distro detection and
> apt-get)? Looks pretty much like the way we proposed originally.

I think that sums it up, it added distro-detection and uses apt-get.

Not sure which you will find easier to maintain. I guess pip might end
up being less work, because you can drop the maint of that debian
mirror (there is no trace of it in the docs, maybe on your jenkins).

While pip is moving "out of control" it will also serve as a detection
mechanism if avocado breaks again. I think i would prefer pip after
all, seeing the risk also as a chance to keep up to date with little
effort.

But feel free to take the patch if you want to go the debian route and
maintain that repo. (in which case it might be a good idea to write in
testsuite/README.md where that deb might be coming from)

Henning

Re: [PATCH] CI: Fix paths for the latest Avocado version

[Anton Mikanovich]

22.10.2021 09:51, Anton Mikanovich wrote:
> We can't use relative path for logdir in Avocado 89.0+ because it fails with:
>
> FileNotFoundError: [Errno 2] No such file or directory: '.../whiteboard'
>
> Move to absolute paths to be able to use the latest version.
>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>

Applied to next.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov