* [PATCH 00/11] Next avocado improvements
@ 2023-02-23 15:41 Anton Mikanovich
2023-02-23 15:41 ` [PATCH 01/11] CI: Update Avocado version Anton Mikanovich
` (10 more replies)
0 siblings, 11 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
There are several ways of improving Isar testsuite currently:
1) Simplify testsuite configuring. Beside of making CI more comfortable for user
it will also move us closer to removing ci_build.sh at all and just execute
avocado with some minimal parameters.
2) Reusing of caches. This will be helpful later on after moving to parallel
test cases execution.
3) Improve testsuite documentation
4) Moving avocado binary up to date
All this improvements can be found in the following patchset.
Anton Mikanovich (9):
CI: Update Avocado version
CI: Add avocado version checking
CI: Set default QEMU timeout to 10 minutes
CI: Make quite option to be set by default
CI: Remove norun option
CI: Enable sstate dir sharing
CI: Enable ccache dir sharing
CI: Update avocado install documentation
CI: Add documentation for test case creation
Uladzimir Bely (2):
scripts: Add option to directly specify avocado tags
gitlab-ci.yml: Run dev tests set on push
.gitlab-ci.yml | 13 ++++++++---
scripts/ci_build.sh | 49 ++++++++++++++++++++++++++++--------------
testsuite/README.md | 48 +++++++++++++++++++++++++++++++++++------
testsuite/cibase.py | 8 +++----
testsuite/cibuilder.py | 18 +++++++++++++---
5 files changed, 103 insertions(+), 33 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 01/11] CI: Update Avocado version
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 02/11] CI: Add avocado version checking Anton Mikanovich
` (9 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Dump Avocado framework version to 100.1
This requires removing deprecated nrunner suffix.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 4 ++--
testsuite/README.md | 8 ++++----
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 5350ad8..37d51f7 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -22,7 +22,7 @@ if ! command -v avocado > /dev/null; then
virtualenv --python python3 /tmp/avocado_venv
# shellcheck source=/dev/null
source /tmp/avocado_venv/bin/activate
- pip install avocado-framework==99.0
+ pip install avocado-framework==100.1
fi
# Get Avocado build tests path
@@ -133,5 +133,5 @@ export VIRTUAL_ENV="./"
set -x
avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
- -t "${TAGS}" --nrunner-max-parallel-tasks=1 --disable-sysinfo \
+ -t "${TAGS}" --max-parallel-tasks=1 --disable-sysinfo \
-p quiet="${QUIET}" -p time_to_wait="${TIMEOUT}"
diff --git a/testsuite/README.md b/testsuite/README.md
index 50698da..d8adc7f 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -15,25 +15,25 @@ $ pip install avocado-framework==99.0
## Quick developers test
```
-$ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1
+$ avocado run ../testsuite/citest.py -t dev --max-parallel-tasks=1
```
## Single target test
```
-$ avocado run ../testsuite/citest.py -t single --nrunner-max-parallel-tasks=1 -p machine=qemuamd64 -p distro=bullseye
+$ avocado run ../testsuite/citest.py -t single --max-parallel-tasks=1 -p machine=qemuamd64 -p distro=bullseye
```
## Fast build test
```
-$ avocado run ../testsuite/citest.py -t fast --nrunner-max-parallel-tasks=1 -p quiet=1
+$ avocado run ../testsuite/citest.py -t fast --max-parallel-tasks=1 -p quiet=1
```
## Full build test
```
-$ avocado run ../testsuite/citest.py -t full --nrunner-max-parallel-tasks=1 -p quiet=1
+$ avocado run ../testsuite/citest.py -t full --max-parallel-tasks=1 -p quiet=1
```
## Fast boot test
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 02/11] CI: Add avocado version checking
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
2023-02-23 15:41 ` [PATCH 01/11] CI: Update Avocado version Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:45 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 03/11] scripts: Add option to directly specify avocado tags Anton Mikanovich
` (8 subsequent siblings)
10 siblings, 1 reply; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
We need to keep backward compatibility with older Avocado versions
until everyone will move to 100+. Then this can be reverted.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 37d51f7..e37114b 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -129,9 +129,17 @@ logs_dir = $(realpath "${BASE_DIR}")/job-results
EOF
export VIRTUAL_ENV="./"
+# use correct variable
+case "$(avocado --version)" in
+ *100*) PARALLEL="--max-parallel-tasks=1"
+ ;;
+ *) PARALLEL="--nrunner-max-parallel-tasks=1"
+ ;;
+esac
+
# the real stuff starts here, trace commands from now on
set -x
avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
- -t "${TAGS}" --max-parallel-tasks=1 --disable-sysinfo \
+ -t "${TAGS}" ${PARALLEL} --disable-sysinfo \
-p quiet="${QUIET}" -p time_to_wait="${TIMEOUT}"
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 03/11] scripts: Add option to directly specify avocado tags
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
2023-02-23 15:41 ` [PATCH 01/11] CI: Update Avocado version Anton Mikanovich
2023-02-23 15:41 ` [PATCH 02/11] CI: Add avocado version checking Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 04/11] gitlab-ci.yml: Run dev tests set on push Anton Mikanovich
` (7 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely
From: Uladzimir Bely <ubely@ilbers.de>
Add to "ci_build.sh" option to specify avocado tags directly by passing
"--tags=<tags>" option.
For compatibility reasons, tread "-f" as "--tags=fast", if user
did not specify "--tags" option directly.
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
scripts/ci_build.sh | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index e37114b..7382862 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -49,7 +49,7 @@ show_help() {
echo " -b, --base BASE_DIR set path to base directory. If not set,"
echo " the tests will be started in current path."
echo " -d, --debug enable debug bitbake output."
- echo " -f, --fast cross build reduced set of configurations."
+ echo " -T, --tags specify basic avocado tags."
echo " -q, --quiet suppress verbose bitbake output."
echo " -n, --norun do not execute QEMU run tests."
echo " -t, --timeout SEC specify time in seconds to wait before stop QEMU."
@@ -61,7 +61,6 @@ show_help() {
echo " 3 if invalid parameters are passed."
}
-TAGS="full"
QUIET="0"
TIMEOUT=300
@@ -82,9 +81,14 @@ do
-d|--debug)
VERBOSE="--show=app,test"
;;
+ -T|--tags)
+ TAGS="$2"
+ shift
+ ;;
-f|--fast)
# Start build for the reduced set of configurations
- TAGS="fast"
+ FAST="1"
+ echo "warning: deprecated parameter '$key', consider using '-T fast' instead"
;;
-q|--quiet)
QUIET="1"
@@ -109,6 +113,14 @@ do
shift
done
+if [ -z "$TAGS" ]; then
+ if [ -n "$FAST" ]; then
+ TAGS="fast"
+ else
+ TAGS="full"
+ fi
+fi
+
if [ -n "$NORUN" ]; then
TAGS="$TAGS,-startvm"
else
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 04/11] gitlab-ci.yml: Run dev tests set on push
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (2 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 03/11] scripts: Add option to directly specify avocado tags Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 05/11] CI: Set default QEMU timeout to 10 minutes Anton Mikanovich
` (6 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely
From: Uladzimir Bely <ubely@ilbers.de>
This makes only shorter "dev" set of tests run in Gitlab on push.
Other tags, "fast" and "full" can be run by schedule.
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
.gitlab-ci.yml | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d1e8249..f9235fc 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -22,12 +22,19 @@ variables:
when: always
expire_in: 1 week
-fast-ci:
+dev-ci:
<<: *common-build
except:
- schedules
script:
- - scripts/ci_build.sh -q -f
+ - scripts/ci_build.sh -q -T dev
+
+fast-ci:
+ <<: *common-build
+ only:
+ - schedules
+ script:
+ - scripts/ci_build.sh -q -T fast
full-ci:
<<: *common-build
@@ -37,7 +44,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q;
+ scripts/ci_build.sh -q -T full;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 05/11] CI: Set default QEMU timeout to 10 minutes
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (3 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 04/11] gitlab-ci.yml: Run dev tests set on push Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 06/11] CI: Make quite option to be set by default Anton Mikanovich
` (5 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Also remove default value inside bash script.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 7 ++-----
testsuite/README.md | 4 ++--
testsuite/cibuilder.py | 4 +++-
3 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 7382862..9f62db5 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -52,8 +52,6 @@ show_help() {
echo " -T, --tags specify basic avocado tags."
echo " -q, --quiet suppress verbose bitbake output."
echo " -n, --norun do not execute QEMU run tests."
- echo " -t, --timeout SEC specify time in seconds to wait before stop QEMU."
- echo " The default is: 300"
echo " --help display this message and exit."
echo
echo "Exit status:"
@@ -62,7 +60,6 @@ show_help() {
}
QUIET="0"
-TIMEOUT=300
# Parse command line to get user configuration
while [ $# -gt 0 ]
@@ -97,7 +94,7 @@ do
NORUN="1"
;;
-t|--timeout)
- TIMEOUT=$2
+ TIMEOUT="-p time_to_wait=$2"
shift
;;
-c|--cross|-r|--repro|-s|--sign)
@@ -154,4 +151,4 @@ set -x
avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
-t "${TAGS}" ${PARALLEL} --disable-sysinfo \
- -p quiet="${QUIET}" -p time_to_wait="${TIMEOUT}"
+ -p quiet="${QUIET}" ${TIMEOUT}
diff --git a/testsuite/README.md b/testsuite/README.md
index d8adc7f..10fbd79 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -39,13 +39,13 @@ $ avocado run ../testsuite/citest.py -t full --max-parallel-tasks=1 -p quiet=1
## Fast boot test
```
-$ avocado run ../testsuite/citest.py -t startvm,fast -p time_to_wait=300
+$ avocado run ../testsuite/citest.py -t startvm,fast
```
## Full boot test
```
-$ avocado run ../testsuite/citest.py -t startvm,full -p time_to_wait=300
+$ avocado run ../testsuite/citest.py -t startvm,full
```
# Running qemu images
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 5eebd5a..65eeb4b 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -15,6 +15,8 @@ from avocado import Test
from avocado.utils import path
from avocado.utils import process
+DEF_VM_TO_SEC = 600
+
isar_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
backup_prefix = '.ci-backup'
@@ -262,7 +264,7 @@ class CIBuilder(Test):
def vm_start(self, arch='amd64', distro='buster',
enforce_pcbios=False, skip_modulecheck=False,
image='isar-image-base', cmd=None, script=None):
- time_to_wait = self.params.get('time_to_wait', default=60)
+ time_to_wait = self.params.get('time_to_wait', default=DEF_VM_TO_SEC)
self.log.info('===================================================')
self.log.info('Running Isar VM boot test for (' + distro + '-' + arch + ')')
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 06/11] CI: Make quite option to be set by default
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (4 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 05/11] CI: Set default QEMU timeout to 10 minutes Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-28 6:40 ` Cedric Hombourger
2023-02-23 15:41 ` [PATCH 07/11] CI: Remove norun option Anton Mikanovich
` (4 subsequent siblings)
10 siblings, 1 reply; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
It looks like nobody run bitbake with additional verbose during
normal testsuite operation, so set non verbose output ("quite") to
be default one. Verbose output still can be enabled by adding
"-p quite=0" directly to avocado command.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
.gitlab-ci.yml | 6 +++---
scripts/ci_build.sh | 7 ++-----
testsuite/README.md | 4 ++--
testsuite/cibuilder.py | 2 +-
4 files changed, 8 insertions(+), 11 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f9235fc..eb38be5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,14 +27,14 @@ dev-ci:
except:
- schedules
script:
- - scripts/ci_build.sh -q -T dev
+ - scripts/ci_build.sh -T dev
fast-ci:
<<: *common-build
only:
- schedules
script:
- - scripts/ci_build.sh -q -T fast
+ - scripts/ci_build.sh -T fast
full-ci:
<<: *common-build
@@ -44,7 +44,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -T full;
+ scripts/ci_build.sh -T full;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 9f62db5..fcac180 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -50,7 +50,6 @@ show_help() {
echo " the tests will be started in current path."
echo " -d, --debug enable debug bitbake output."
echo " -T, --tags specify basic avocado tags."
- echo " -q, --quiet suppress verbose bitbake output."
echo " -n, --norun do not execute QEMU run tests."
echo " --help display this message and exit."
echo
@@ -59,8 +58,6 @@ show_help() {
echo " 3 if invalid parameters are passed."
}
-QUIET="0"
-
# Parse command line to get user configuration
while [ $# -gt 0 ]
do
@@ -88,7 +85,7 @@ do
echo "warning: deprecated parameter '$key', consider using '-T fast' instead"
;;
-q|--quiet)
- QUIET="1"
+ echo "warning: deprecated parameter '$key', it is applied by default"
;;
-n|--norun)
NORUN="1"
@@ -151,4 +148,4 @@ set -x
avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
-t "${TAGS}" ${PARALLEL} --disable-sysinfo \
- -p quiet="${QUIET}" ${TIMEOUT}
+ ${TIMEOUT}
diff --git a/testsuite/README.md b/testsuite/README.md
index 10fbd79..c9d5862 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -27,13 +27,13 @@ $ avocado run ../testsuite/citest.py -t single --max-parallel-tasks=1 -p machine
## Fast build test
```
-$ avocado run ../testsuite/citest.py -t fast --max-parallel-tasks=1 -p quiet=1
+$ avocado run ../testsuite/citest.py -t fast --max-parallel-tasks=1
```
## Full build test
```
-$ avocado run ../testsuite/citest.py -t full --max-parallel-tasks=1 -p quiet=1
+$ avocado run ../testsuite/citest.py -t full --max-parallel-tasks=1
```
## Fast boot test
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 65eeb4b..0a07bf0 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -64,7 +64,7 @@ class CIBuilder(Test):
self.check_init()
# get parameters from avocado cmdline
- quiet = bool(int(self.params.get('quiet', default=0)))
+ quiet = bool(int(self.params.get('quiet', default=1)))
if dl_dir is None:
dl_dir = os.path.join(isar_root, 'downloads')
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 07/11] CI: Remove norun option
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (5 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 06/11] CI: Make quite option to be set by default Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 08/11] CI: Enable sstate dir sharing Anton Mikanovich
` (3 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
This actually removes option added in 155c139 because '-n' flag no more
used in .gitlab-ci.yml. If someone still need to skip startvm test
cases tag '-startvm' can be used.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index fcac180..cd4249f 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -50,7 +50,6 @@ show_help() {
echo " the tests will be started in current path."
echo " -d, --debug enable debug bitbake output."
echo " -T, --tags specify basic avocado tags."
- echo " -n, --norun do not execute QEMU run tests."
echo " --help display this message and exit."
echo
echo "Exit status:"
@@ -89,6 +88,7 @@ do
;;
-n|--norun)
NORUN="1"
+ echo "warning: deprecated parameter '$key', consider using '-T <TAG>,-startvm' instead"
;;
-t|--timeout)
TIMEOUT="-p time_to_wait=$2"
@@ -115,9 +115,12 @@ if [ -z "$TAGS" ]; then
fi
fi
+# Deprecated
if [ -n "$NORUN" ]; then
TAGS="$TAGS,-startvm"
-else
+fi
+
+if echo "$TAGS" | grep -Fqive "-startvm"; then
if [ ! -f /usr/share/doc/qemu-system/copyright ]; then
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends qemu-system ovmf
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 08/11] CI: Enable sstate dir sharing
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (6 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 07/11] CI: Remove norun option Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 09/11] CI: Enable ccache " Anton Mikanovich
` (2 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Test cases performed in different build dirs now can share SSTATE_DIR.
Enable it by default and disable sharing for cache-related test cases.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibase.py | 8 ++++----
testsuite/cibuilder.py | 7 +++++++
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index 6239b4d..3267c17 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -31,7 +31,7 @@ class CIBaseTest(CIBuilder):
gpg_pub_key = os.path.dirname(__file__) + '/keys/base-apt/test_pub.key'
gpg_priv_key = os.path.dirname(__file__) + '/keys/base-apt/test_priv.key'
- self.configure(gpg_pub_key=gpg_pub_key if signed else None, **kwargs)
+ self.configure(gpg_pub_key=gpg_pub_key if signed else None, sstate_dir="", **kwargs)
os.chdir(self.build_dir)
@@ -44,7 +44,7 @@ class CIBaseTest(CIBuilder):
self.bitbake(targets, **kwargs)
self.delete_from_build_dir('tmp')
- self.configure(gpg_pub_key=gpg_pub_key if signed else None, offline=True, **kwargs)
+ self.configure(gpg_pub_key=gpg_pub_key if signed else None, offline=True, sstate_dir="", **kwargs)
self.bitbake(targets, **kwargs)
@@ -67,7 +67,7 @@ class CIBaseTest(CIBuilder):
count += int(content[field])
return count
- self.configure(ccache=True, **kwargs)
+ self.configure(ccache=True, sstate_dir="", **kwargs)
# Field that stores direct ccache hits
direct_cache_hit = 22
@@ -119,7 +119,7 @@ class CIBaseTest(CIBuilder):
return False
return True
- self.configure(sstate=True, **kwargs)
+ self.configure(sstate=True, sstate_dir="", **kwargs)
# Cleanup sstate and tmp before test
self.delete_from_build_dir('sstate-cache')
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 0a07bf0..ef46352 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -58,6 +58,7 @@ class CIBuilder(Test):
def configure(self, compat_arch=True, cross=True, debsrc_cache=False,
container=False, ccache=False, sstate=False, offline=False,
gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
+ sstate_dir=None,
source_date_epoch=None, image_install=None, **kwargs):
# write configuration file and set bitbake_args
# can run multiple times per test case
@@ -66,8 +67,11 @@ class CIBuilder(Test):
# get parameters from avocado cmdline
quiet = bool(int(self.params.get('quiet', default=1)))
+ # set those to "" to not set dir value but use system default
if dl_dir is None:
dl_dir = os.path.join(isar_root, 'downloads')
+ if sstate_dir is None:
+ sstate_dir = os.path.join(isar_root, 'sstate-cache')
# get parameters from environment
distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS')
@@ -85,6 +89,7 @@ class CIBuilder(Test):
f' wic_deploy_parts = {wic_deploy_parts}\n'
f' source_date_epoch = {source_date_epoch} \n'
f' dl_dir = {dl_dir}\n'
+ f' sstate_dir = {sstate_dir}\n'
f' image_install = {image_install}\n'
f'===================================================')
@@ -124,6 +129,8 @@ class CIBuilder(Test):
f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch)
if dl_dir:
f.write('DL_DIR = "%s"\n' % dl_dir)
+ if sstate_dir:
+ f.write('SSTATE_DIR = "%s"\n' % sstate_dir)
if image_install is not None:
f.write('IMAGE_INSTALL = "%s"' % image_install)
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 09/11] CI: Enable ccache dir sharing
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (7 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 08/11] CI: Enable sstate dir sharing Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 10/11] CI: Update avocado install documentation Anton Mikanovich
2023-02-23 15:41 ` [PATCH 11/11] CI: Add documentation for test case creation Anton Mikanovich
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Test cases performed in different build dirs now can share
CCACHE_TOP_DIR. Enable ccache sharing if ccache got enabled.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index ef46352..0f84cca 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -58,7 +58,7 @@ class CIBuilder(Test):
def configure(self, compat_arch=True, cross=True, debsrc_cache=False,
container=False, ccache=False, sstate=False, offline=False,
gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
- sstate_dir=None,
+ sstate_dir=None, ccache_dir=None,
source_date_epoch=None, image_install=None, **kwargs):
# write configuration file and set bitbake_args
# can run multiple times per test case
@@ -72,6 +72,8 @@ class CIBuilder(Test):
dl_dir = os.path.join(isar_root, 'downloads')
if sstate_dir is None:
sstate_dir = os.path.join(isar_root, 'sstate-cache')
+ if ccache_dir is None:
+ ccache_dir = '${TOPDIR}/ccache'
# get parameters from environment
distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS')
@@ -90,6 +92,7 @@ class CIBuilder(Test):
f' source_date_epoch = {source_date_epoch} \n'
f' dl_dir = {dl_dir}\n'
f' sstate_dir = {sstate_dir}\n'
+ f' ccache_dir = {ccache_dir}\n'
f' image_install = {image_install}\n'
f'===================================================')
@@ -124,7 +127,7 @@ class CIBuilder(Test):
f.write('DISTRO_APT_PREMIRRORS = "%s"\n' % distro_apt_premir)
if ccache:
f.write('USE_CCACHE = "1"\n')
- f.write('CCACHE_TOP_DIR = "${TOPDIR}/ccache"\n')
+ f.write('CCACHE_TOP_DIR = "%s"\n' % ccache_dir)
if source_date_epoch:
f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch)
if dl_dir:
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 10/11] CI: Update avocado install documentation
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (8 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 09/11] CI: Enable ccache " Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 11/11] CI: Add documentation for test case creation Anton Mikanovich
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Align preferred install steps with current ci_build.sh state.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/README.md | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/testsuite/README.md b/testsuite/README.md
index c9d5862..570ead9 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -7,7 +7,12 @@ The framework could be installed by using standard HOWTO:
## For Debian (tested on Debian 11.x)
```
-$ pip install avocado-framework==99.0
+$ sudo apt-get update -qq
+$ sudo apt-get install -y virtualenv
+$ rm -rf /tmp/avocado_venv
+$ virtualenv --python python3 /tmp/avocado_venv
+$ source /tmp/avocado_venv/bin/activate
+$ pip install avocado-framework==100.1
```
# Run test
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 11/11] CI: Add documentation for test case creation
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
` (9 preceding siblings ...)
2023-02-23 15:41 ` [PATCH 10/11] CI: Update avocado install documentation Anton Mikanovich
@ 2023-02-23 15:41 ` Anton Mikanovich
10 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:41 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/README.md | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/testsuite/README.md b/testsuite/README.md
index 570ead9..476b860 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -93,3 +93,32 @@ can be executed instead of command:
The default location of custom scripts is `isar/testsuite/`. It can be changed
by passing `-p test_script_dir="custom_path"` to `avocado run`
arguments.
+
+# Custom test case creation
+
+The minimal build test can be look like:
+
+```
+#!/usr/bin/env python3
+
+from cibase import CIBaseTest
+
+class SampleTest(CIBaseTest):
+ def test_sample(self):
+ self.init()
+ self.perform_build_test("mc:qemuamd64-bullseye:isar-image-base")
+```
+
+To show the list of available tests you can run:
+
+```
+$ avocado list sample.py
+avocado-instrumented sample.py:SampleTest.test_sample
+```
+
+And to execute this example:
+
+```
+$ avocado run sample.py:SampleTest.test_sample
+```
+
--
2.34.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 02/11] CI: Add avocado version checking
2023-02-23 15:41 ` [PATCH 02/11] CI: Add avocado version checking Anton Mikanovich
@ 2023-02-23 15:45 ` Anton Mikanovich
0 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-23 15:45 UTC (permalink / raw)
To: isar-users
23/02/2023 17:41, Anton Mikanovich wrote:
> We need to keep backward compatibility with older Avocado versions
> until everyone will move to 100+. Then this can be reverted.
>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
This commit is just temporary for the period all the CI will be moved to at
least Avocado 99.0 version. If no one actually need this it can be skipped.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 06/11] CI: Make quite option to be set by default
2023-02-23 15:41 ` [PATCH 06/11] CI: Make quite option to be set by default Anton Mikanovich
@ 2023-02-28 6:40 ` Cedric Hombourger
2023-02-28 7:18 ` Anton Mikanovich
0 siblings, 1 reply; 15+ messages in thread
From: Cedric Hombourger @ 2023-02-28 6:40 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 3534 bytes --]
On Thursday, February 23, 2023 at 4:41:44 PM UTC+1 Anton Mikanovich wrote:
It looks like nobody run bitbake with additional verbose during
normal testsuite operation, so set non verbose output ("quite") to
be default one. Verbose output still can be enabled by adding
"-p quite=0" directly to avocado command.
typo: quite => quiet
Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---
.gitlab-ci.yml | 6 +++---
scripts/ci_build.sh | 7 ++-----
testsuite/README.md | 4 ++--
testsuite/cibuilder.py | 2 +-
4 files changed, 8 insertions(+), 11 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f9235fc..eb38be5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,14 +27,14 @@ dev-ci:
except:
- schedules
script:
- - scripts/ci_build.sh -q -T dev
+ - scripts/ci_build.sh -T dev
fast-ci:
<<: *common-build
only:
- schedules
script:
- - scripts/ci_build.sh -q -T fast
+ - scripts/ci_build.sh -T fast
full-ci:
<<: *common-build
@@ -44,7 +44,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -T full;
+ scripts/ci_build.sh -T full;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 9f62db5..fcac180 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -50,7 +50,6 @@ show_help() {
echo " the tests will be started in current path."
echo " -d, --debug enable debug bitbake output."
echo " -T, --tags specify basic avocado tags."
- echo " -q, --quiet suppress verbose bitbake output."
echo " -n, --norun do not execute QEMU run tests."
echo " --help display this message and exit."
echo
@@ -59,8 +58,6 @@ show_help() {
echo " 3 if invalid parameters are passed."
}
-QUIET="0"
-
# Parse command line to get user configuration
while [ $# -gt 0 ]
do
@@ -88,7 +85,7 @@ do
echo "warning: deprecated parameter '$key', consider using '-T fast'
instead"
;;
-q|--quiet)
- QUIET="1"
+ echo "warning: deprecated parameter '$key', it is applied by default"
;;
-n|--norun)
NORUN="1"
@@ -151,4 +148,4 @@ set -x
avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
-t "${TAGS}" ${PARALLEL} --disable-sysinfo \
- -p quiet="${QUIET}" ${TIMEOUT}
+ ${TIMEOUT}
diff --git a/testsuite/README.md b/testsuite/README.md
index 10fbd79..c9d5862 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -27,13 +27,13 @@ $ avocado run ../testsuite/citest.py -t single
--max-parallel-tasks=1 -p machine
## Fast build test
```
-$ avocado run ../testsuite/citest.py -t fast --max-parallel-tasks=1 -p
quiet=1
+$ avocado run ../testsuite/citest.py -t fast --max-parallel-tasks=1
```
## Full build test
```
-$ avocado run ../testsuite/citest.py -t full --max-parallel-tasks=1 -p
quiet=1
+$ avocado run ../testsuite/citest.py -t full --max-parallel-tasks=1
```
## Fast boot test
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 65eeb4b..0a07bf0 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -64,7 +64,7 @@ class CIBuilder(Test):
self.check_init()
# get parameters from avocado cmdline
- quiet = bool(int(self.params.get('quiet', default=0)))
+ quiet = bool(int(self.params.get('quiet', default=1)))
if dl_dir is None:
dl_dir = os.path.join(isar_root, 'downloads')
--
2.34.1
[-- Attachment #1.2: Type: text/html, Size: 4900 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 06/11] CI: Make quite option to be set by default
2023-02-28 6:40 ` Cedric Hombourger
@ 2023-02-28 7:18 ` Anton Mikanovich
0 siblings, 0 replies; 15+ messages in thread
From: Anton Mikanovich @ 2023-02-28 7:18 UTC (permalink / raw)
To: isar-users
28/02/2023 08:40, Cedric Hombourger wrote:
>
>
> On Thursday, February 23, 2023 at 4:41:44 PM UTC+1 Anton Mikanovich wrote:
>
> It looks like nobody run bitbake with additional verbose during
> normal testsuite operation, so set non verbose output ("quite") to
> be default one. Verbose output still can be enabled by adding
> "-p quite=0" directly to avocado command.
>
>
> typo: quite => quiet
Thanks for notify. There are also other typos in this patchset,
will fix them in next version or during the merge.
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2023-02-28 7:18 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-23 15:41 [PATCH 00/11] Next avocado improvements Anton Mikanovich
2023-02-23 15:41 ` [PATCH 01/11] CI: Update Avocado version Anton Mikanovich
2023-02-23 15:41 ` [PATCH 02/11] CI: Add avocado version checking Anton Mikanovich
2023-02-23 15:45 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 03/11] scripts: Add option to directly specify avocado tags Anton Mikanovich
2023-02-23 15:41 ` [PATCH 04/11] gitlab-ci.yml: Run dev tests set on push Anton Mikanovich
2023-02-23 15:41 ` [PATCH 05/11] CI: Set default QEMU timeout to 10 minutes Anton Mikanovich
2023-02-23 15:41 ` [PATCH 06/11] CI: Make quite option to be set by default Anton Mikanovich
2023-02-28 6:40 ` Cedric Hombourger
2023-02-28 7:18 ` Anton Mikanovich
2023-02-23 15:41 ` [PATCH 07/11] CI: Remove norun option Anton Mikanovich
2023-02-23 15:41 ` [PATCH 08/11] CI: Enable sstate dir sharing Anton Mikanovich
2023-02-23 15:41 ` [PATCH 09/11] CI: Enable ccache " Anton Mikanovich
2023-02-23 15:41 ` [PATCH 10/11] CI: Update avocado install documentation Anton Mikanovich
2023-02-23 15:41 ` [PATCH 11/11] CI: Add documentation for test case creation Anton Mikanovich
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox