[PATCH 0/2] Adopt CI to Avocado 97.0

[PATCH 0/2] Adopt CI to Avocado 97.0

[Anton Mikanovich]

Avocado package version was fixed due to dropped support of legacy
runner in the newest version. Move to the new runner to be able to use
the latest Avocado version.

Anton Mikanovich (2):
  CI: switch to the new runner
  Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

 scripts/ci_build.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
2.17.1

[PATCH 1/2] CI: switch to the new runner

[Anton Mikanovich]

Switch CI from legacy runner removed in Avocado 97.0 to the nrunner.
To make it run serialized limit maximum task count to 1.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 scripts/ci_build.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index b1ec691..fa7208c 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -143,5 +143,5 @@ export VIRTUAL_ENV="./"
 set -x
 
 avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
-    -t $TAGS --test-runner=runner --disable-sysinfo \
+    -t $TAGS --nrunner-max-parallel-tasks=1 --disable-sysinfo \
     -p quiet=$QUIET -p cross=$CROSS_BUILD -p time_to_wait=$TIMEOUT
-- 
2.17.1

[PATCH 2/2] Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

[Anton Mikanovich]

No more Avocado version limit needed after switching to nrunner.

This reverts commit 9aec8ba92912c682f7732b015f1919c5308f40a2.
---
 scripts/ci_build.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index fa7208c..debb386 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
     rm -rf /tmp/avocado_venv
     virtualenv --python python3 /tmp/avocado_venv
     source /tmp/avocado_venv/bin/activate
-    pip install avocado-framework==96.0
+    pip install avocado-framework
 fi
 
 # Get Avocado build tests path
-- 
2.17.1

Re: [PATCH 2/2] Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

[Jan Kiszka]

On 12.07.22 19:32, Anton Mikanovich wrote:
> No more Avocado version limit needed after switching to nrunner.
> 
> This reverts commit 9aec8ba92912c682f7732b015f1919c5308f40a2.
> ---
>  scripts/ci_build.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index fa7208c..debb386 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
>      rm -rf /tmp/avocado_venv
>      virtualenv --python python3 /tmp/avocado_venv
>      source /tmp/avocado_venv/bin/activate
> -    pip install avocado-framework==96.0
> +    pip install avocado-framework
>  fi
>  
>  # Get Avocado build tests path

Bad idea as you can't predict future API changes of Avocado.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [PATCH 1/2] CI: switch to the new runner

[Henning Schild]

Am Tue, 12 Jul 2022 20:32:36 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> Switch CI from legacy runner removed in Avocado 97.0 to the nrunner.
> To make it run serialized limit maximum task count to 1.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  scripts/ci_build.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index b1ec691..fa7208c 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -143,5 +143,5 @@ export VIRTUAL_ENV="./"
>  set -x
>  
>  avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
> -    -t $TAGS --test-runner=runner --disable-sysinfo \
> +    -t $TAGS --nrunner-max-parallel-tasks=1 --disable-sysinfo \

Why 1 and not 42 or nproc? Because our tests are written in a way that
they build onto each other, so we lack modularization and are pretty
much still on a monolith like back in the shell age?

Henning

>      -p quiet=$QUIET -p cross=$CROSS_BUILD -p time_to_wait=$TIMEOUT

Re: [PATCH 1/2] CI: switch to the new runner

[Anton Mikanovich]

13.07.2022 14:20, Henning Schild wrote:
> Am Tue, 12 Jul 2022 20:32:36 +0300
> schrieb Anton Mikanovich <amikan@ilbers.de>:
>
>> Switch CI from legacy runner removed in Avocado 97.0 to the nrunner.
>> To make it run serialized limit maximum task count to 1.
>>
>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>> ---
>>   scripts/ci_build.sh | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
>> index b1ec691..fa7208c 100755
>> --- a/scripts/ci_build.sh
>> +++ b/scripts/ci_build.sh
>> @@ -143,5 +143,5 @@ export VIRTUAL_ENV="./"
>>   set -x
>>   
>>   avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
>> -    -t $TAGS --test-runner=runner --disable-sysinfo \
>> +    -t $TAGS --nrunner-max-parallel-tasks=1 --disable-sysinfo \
> Why 1 and not 42 or nproc? Because our tests are written in a way that
> they build onto each other, so we lack modularization and are pretty
> much still on a monolith like back in the shell age?
>
> Henning
>
>>       -p quiet=$QUIET -p cross=$CROSS_BUILD -p time_to_wait=$TIMEOUT

We need this fix to make nrunner work like previous legacy runner until 
tests
dependencies will be integrated into Avocado so we can rebuild our test 
cases
to be executed in parallel.
See: https://github.com/avocado-framework/avocado/issues/5205

Re: [PATCH 2/2] Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

[Gylstorff Quirin]

On 7/13/22 07:19, Jan Kiszka wrote:
> On 12.07.22 19:32, Anton Mikanovich wrote:
>> No more Avocado version limit needed after switching to nrunner.
>>
>> This reverts commit 9aec8ba92912c682f7732b015f1919c5308f40a2.
>> ---
>>   scripts/ci_build.sh | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
>> index fa7208c..debb386 100755
>> --- a/scripts/ci_build.sh
>> +++ b/scripts/ci_build.sh
>> @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
>>       rm -rf /tmp/avocado_venv
>>       virtualenv --python python3 /tmp/avocado_venv
>>       source /tmp/avocado_venv/bin/activate
>> -    pip install avocado-framework==96.0
>> +    pip install avocado-framework
>>   fi
>>   
>>   # Get Avocado build tests path
> 
> Bad idea as you can't predict future API changes of Avocado.
> 

We should use a defined version for the avocado framework. This 
simplifies the testing for new developers.

As alternative we can use a requirements.txt.

Quirin
> Jan
> 

Re: [PATCH 2/2] Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

[Anton Mikanovich]

13.07.2022 16:12, Gylstorff Quirin wrote:
>
>
> On 7/13/22 07:19, Jan Kiszka wrote:
>> On 12.07.22 19:32, Anton Mikanovich wrote:
>>> No more Avocado version limit needed after switching to nrunner.
>>>
>>> This reverts commit 9aec8ba92912c682f7732b015f1919c5308f40a2.
>>> ---
>>>   scripts/ci_build.sh | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
>>> index fa7208c..debb386 100755
>>> --- a/scripts/ci_build.sh
>>> +++ b/scripts/ci_build.sh
>>> @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
>>>       rm -rf /tmp/avocado_venv
>>>       virtualenv --python python3 /tmp/avocado_venv
>>>       source /tmp/avocado_venv/bin/activate
>>> -    pip install avocado-framework==96.0
>>> +    pip install avocado-framework
>>>   fi
>>>     # Get Avocado build tests path
>>
>> Bad idea as you can't predict future API changes of Avocado.
>>
>
> We should use a defined version for the avocado framework. This 
> simplifies the testing for new developers.
>
> As alternative we can use a requirements.txt.
>
> Quirin
>> Jan
>>
>
Ok. Let's just fix it to 97.0 for now and move next after manual testing 
only.

Re: [PATCH 2/2] Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

[Jan Kiszka]

On 13.07.22 15:21, Anton Mikanovich wrote:
> 13.07.2022 16:12, Gylstorff Quirin wrote:
>>
>>
>> On 7/13/22 07:19, Jan Kiszka wrote:
>>> On 12.07.22 19:32, Anton Mikanovich wrote:
>>>> No more Avocado version limit needed after switching to nrunner.
>>>>
>>>> This reverts commit 9aec8ba92912c682f7732b015f1919c5308f40a2.
>>>> ---
>>>>   scripts/ci_build.sh | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
>>>> index fa7208c..debb386 100755
>>>> --- a/scripts/ci_build.sh
>>>> +++ b/scripts/ci_build.sh
>>>> @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
>>>>       rm -rf /tmp/avocado_venv
>>>>       virtualenv --python python3 /tmp/avocado_venv
>>>>       source /tmp/avocado_venv/bin/activate
>>>> -    pip install avocado-framework==96.0
>>>> +    pip install avocado-framework
>>>>   fi
>>>>     # Get Avocado build tests path
>>>
>>> Bad idea as you can't predict future API changes of Avocado.
>>>
>>
>> We should use a defined version for the avocado framework. This
>> simplifies the testing for new developers.
>>
>> As alternative we can use a requirements.txt.
>>
>> Quirin
>>> Jan
>>>
>>
> Ok. Let's just fix it to 97.0 for now and move next after manual testing
> only.
> 

You can have an additionally scheduled pipeline doing "latest" testing
every week or so and report issues early this way.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [PATCH 2/2] Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"

[Henning Schild]

Am Thu, 14 Jul 2022 11:19:59 +0200
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> On 13.07.22 15:21, Anton Mikanovich wrote:
> > 13.07.2022 16:12, Gylstorff Quirin wrote:  
> >>
> >>
> >> On 7/13/22 07:19, Jan Kiszka wrote:  
> >>> On 12.07.22 19:32, Anton Mikanovich wrote:  
> >>>> No more Avocado version limit needed after switching to nrunner.
> >>>>
> >>>> This reverts commit 9aec8ba92912c682f7732b015f1919c5308f40a2.
> >>>> ---
> >>>>   scripts/ci_build.sh | 2 +-
> >>>>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> >>>> index fa7208c..debb386 100755
> >>>> --- a/scripts/ci_build.sh
> >>>> +++ b/scripts/ci_build.sh
> >>>> @@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
> >>>>       rm -rf /tmp/avocado_venv
> >>>>       virtualenv --python python3 /tmp/avocado_venv
> >>>>       source /tmp/avocado_venv/bin/activate
> >>>> -    pip install avocado-framework==96.0
> >>>> +    pip install avocado-framework
> >>>>   fi
> >>>>     # Get Avocado build tests path  
> >>>
> >>> Bad idea as you can't predict future API changes of Avocado.
> >>>  
> >>
> >> We should use a defined version for the avocado framework. This
> >> simplifies the testing for new developers.
> >>
> >> As alternative we can use a requirements.txt.
> >>
> >> Quirin  
> >>> Jan
> >>>  
> >>  
> > Ok. Let's just fix it to 97.0 for now and move next after manual
> > testing only.
> >   
> 
> You can have an additionally scheduled pipeline doing "latest" testing
> every week or so and report issues early this way.

Very good idea! But i am afraid that jenkins setup we use for upstream
CI does already have avocado installed, likely even in another version
than we pin here.

Henning

> Jan
> 

Re: [PATCH 0/2] Adopt CI to Avocado 97.0

[Anton Mikanovich]

12.07.2022 20:32, Anton Mikanovich wrote:
> Avocado package version was fixed due to dropped support of legacy
> runner in the newest version. Move to the new runner to be able to use
> the latest Avocado version.
>
> Anton Mikanovich (2):
>    CI: switch to the new runner
>    Revert "scripts/ci_build.sh: Set avocado-framework version to 96.0"
>
>   scripts/ci_build.sh | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
Patch p1 applied to next, p2 rejected.