* [PATCH v3 1/7] CI: Reorganize testsuite folders
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 2/7] CI: Migrate all tests to one location Anton Mikanovich
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Move all tests into one directory to be able to unify them.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 4 +--
scripts/vm_smoke_test | 4 +--
testsuite/README.md | 26 +++++++++++++++++---
testsuite/{build_test => }/build_test.py | 0
testsuite/build_test/run_fast.sh | 3 ---
testsuite/build_test/run_full.sh | 3 ---
testsuite/{build_test => }/cibase.py | 4 +--
testsuite/{build_test => }/cibuilder.py | 2 +-
testsuite/{ => keys}/base-apt/test_priv.key | 0
testsuite/{ => keys}/base-apt/test_pub.key | 0
testsuite/{vm_boot_test => }/vm_boot_test.py | 1 -
testsuite/vm_boot_test/run_fast.sh | 3 ---
testsuite/vm_boot_test/run_full.sh | 3 ---
13 files changed, 30 insertions(+), 23 deletions(-)
rename testsuite/{build_test => }/build_test.py (100%)
mode change 100644 => 100755
delete mode 100755 testsuite/build_test/run_fast.sh
delete mode 100755 testsuite/build_test/run_full.sh
rename testsuite/{build_test => }/cibase.py (97%)
mode change 100644 => 100755
rename testsuite/{build_test => }/cibuilder.py (99%)
mode change 100644 => 100755
rename testsuite/{ => keys}/base-apt/test_priv.key (100%)
rename testsuite/{ => keys}/base-apt/test_pub.key (100%)
rename testsuite/{vm_boot_test => }/vm_boot_test.py (98%)
mode change 100644 => 100755
delete mode 100755 testsuite/vm_boot_test/run_fast.sh
delete mode 100755 testsuite/vm_boot_test/run_full.sh
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 339ebca..fe139d1 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -25,7 +25,7 @@ if ! command -v avocado > /dev/null; then
fi
# Get Avocado build tests path
-BUILD_TEST_DIR="$(pwd)/testsuite/build_test"
+TESTSUITE_DIR="$(pwd)/testsuite"
# Start tests in current path by default
BASE_DIR=./build
@@ -127,6 +127,6 @@ export VIRTUAL_ENV="./"
# the real stuff starts here, trace commands from now on
set -x
-avocado $VERBOSE run "$BUILD_TEST_DIR/build_test.py" \
+avocado $VERBOSE run "$TESTSUITE_DIR/build_test.py" \
-t $TAGS --test-runner=runner --disable-sysinfo \
-p quiet=$QUIET -p cross=$CROSS_BUILD
diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
index 6416250..50051a7 100755
--- a/scripts/vm_smoke_test
+++ b/scripts/vm_smoke_test
@@ -14,7 +14,7 @@ ES_BUG=3
RET=$ES_FAIL
# Get Avocado QEMU tests path
-VM_TEST_DIR="$(dirname "$0")/../testsuite/vm_boot_test"
+TESTSUITE_DIR="$(dirname "$0")/../testsuite"
# Go to Isar root
cd "$(dirname "$0")/.."
@@ -95,7 +95,7 @@ logs_dir = $BUILD_DIR/job-results
EOF
export VIRTUAL_ENV="./"
-if avocado $VERBOSE run "$VM_TEST_DIR/vm_boot_test.py" -t $TAGS \
+if avocado $VERBOSE run "$TESTSUITE_DIR/vm_boot_test.py" -t $TAGS \
--test-runner=runner --disable-sysinfo \
-p build_dir="$BUILD_DIR" -p time_to_wait=$TIMEOUT; then
RET=$ES_OK
diff --git a/testsuite/README.md b/testsuite/README.md
index fcbea1e..b83a8f9 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -10,9 +10,29 @@ The framework could be installed by using standard HOWTO:
# Run test
-Each testsuite directory contains:
- - run_*.sh - script to start tests
- - *.py - test case
+## Fast build test
+
+```
+$ avocado run build_test.py -t fast -p quiet=1 -p cross=1
+```
+
+## Full build test
+
+```
+$ avocado run build_test.py -t full -p quiet=1
+```
+
+## Fast boot test
+
+```
+$ avocado run vm_boot_test.py -t fast -p build_dir="$BUILDDIR" -p time_to_wait=300
+```
+
+## Full boot test
+
+```
+$ avocado run vm_boot_test.py -t full -p build_dir="$BUILDDIR" -p time_to_wait=300
+```
# Other
diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test.py
old mode 100644
new mode 100755
similarity index 100%
rename from testsuite/build_test/build_test.py
rename to testsuite/build_test.py
diff --git a/testsuite/build_test/run_fast.sh b/testsuite/build_test/run_fast.sh
deleted file mode 100755
index 4d1bf44..0000000
--- a/testsuite/build_test/run_fast.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-avocado run build_test.py -t fast -p quiet=1 -p cross=1
diff --git a/testsuite/build_test/run_full.sh b/testsuite/build_test/run_full.sh
deleted file mode 100755
index af5ec59..0000000
--- a/testsuite/build_test/run_full.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-avocado run build_test.py -t full -p quiet=1
diff --git a/testsuite/build_test/cibase.py b/testsuite/cibase.py
old mode 100644
new mode 100755
similarity index 97%
rename from testsuite/build_test/cibase.py
rename to testsuite/cibase.py
index dbf4d3a..0281be4
--- a/testsuite/build_test/cibase.py
+++ b/testsuite/cibase.py
@@ -18,8 +18,8 @@ class CIBaseTest(CIBuilder):
self.bitbake(targets, **kwargs)
def perform_repro_test(self, targets, signed=False, **kwargs):
- gpg_pub_key = os.path.dirname(__file__) + '/../base-apt/test_pub.key'
- gpg_priv_key = os.path.dirname(__file__) + '/../base-apt/test_priv.key'
+ gpg_pub_key = os.path.dirname(__file__) + '/keys/base-apt/test_pub.key'
+ gpg_priv_key = os.path.dirname(__file__) + '/keys/base-apt/test_priv.key'
self.configure(gpg_pub_key=gpg_pub_key if signed else None, **kwargs)
diff --git a/testsuite/build_test/cibuilder.py b/testsuite/cibuilder.py
old mode 100644
new mode 100755
similarity index 99%
rename from testsuite/build_test/cibuilder.py
rename to testsuite/cibuilder.py
index 2cfb6ec..f0c4bc3
--- a/testsuite/build_test/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -10,7 +10,7 @@ from avocado import Test
from avocado.utils import path
from avocado.utils import process
-isar_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..'))
+isar_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
backup_prefix = '.ci-backup'
app_log = logging.getLogger("avocado.app")
diff --git a/testsuite/base-apt/test_priv.key b/testsuite/keys/base-apt/test_priv.key
similarity index 100%
rename from testsuite/base-apt/test_priv.key
rename to testsuite/keys/base-apt/test_priv.key
diff --git a/testsuite/base-apt/test_pub.key b/testsuite/keys/base-apt/test_pub.key
similarity index 100%
rename from testsuite/base-apt/test_pub.key
rename to testsuite/keys/base-apt/test_pub.key
diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test.py
old mode 100644
new mode 100755
similarity index 98%
rename from testsuite/vm_boot_test/vm_boot_test.py
rename to testsuite/vm_boot_test.py
index f8c655c..6c4e979
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test.py
@@ -8,7 +8,6 @@ import time
import tempfile
from os.path import dirname
-sys.path.append(dirname(__file__) + '/..')
import start_vm
diff --git a/testsuite/vm_boot_test/run_fast.sh b/testsuite/vm_boot_test/run_fast.sh
deleted file mode 100755
index 0fc77b3..0000000
--- a/testsuite/vm_boot_test/run_fast.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-avocado run vm_boot_test.py -t fast -p build_dir="$BUILDDIR" -p time_to_wait=300
diff --git a/testsuite/vm_boot_test/run_full.sh b/testsuite/vm_boot_test/run_full.sh
deleted file mode 100755
index a561a36..0000000
--- a/testsuite/vm_boot_test/run_full.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-avocado run vm_boot_test.py -t full -p build_dir="$BUILDDIR" -p time_to_wait=300
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v3 2/7] CI: Migrate all tests to one location
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 1/7] CI: Reorganize testsuite folders Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 3/7] CI: Adopt vm_start test logging Anton Mikanovich
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Move QEMU start tests to the same class as build tests to be able to
run everything from the single avocado command.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/vm_smoke_test | 2 +-
testsuite/build_test.py | 46 +++++++++++++
testsuite/cibuilder.py | 68 ++++++++++++++++++++
testsuite/vm_boot_test.py | 132 --------------------------------------
4 files changed, 115 insertions(+), 133 deletions(-)
delete mode 100755 testsuite/vm_boot_test.py
diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
index 50051a7..28c959a 100755
--- a/scripts/vm_smoke_test
+++ b/scripts/vm_smoke_test
@@ -95,7 +95,7 @@ logs_dir = $BUILD_DIR/job-results
EOF
export VIRTUAL_ENV="./"
-if avocado $VERBOSE run "$TESTSUITE_DIR/vm_boot_test.py" -t $TAGS \
+if avocado $VERBOSE run "$TESTSUITE_DIR/build_test.py" -t $TAGS,startvm \
--test-runner=runner --disable-sysinfo \
-p build_dir="$BUILD_DIR" -p time_to_wait=$TIMEOUT; then
RET=$ES_OK
diff --git a/testsuite/build_test.py b/testsuite/build_test.py
index e0eea1b..d1e66d7 100755
--- a/testsuite/build_test.py
+++ b/testsuite/build_test.py
@@ -244,3 +244,49 @@ class SstateTest(CIBaseTest):
self.init('build-sstate')
self.perform_sstate_test(image_target, package_target)
+
+class VmBootTestFast(CIBaseTest):
+
+ """
+ Test QEMU image start (fast)
+
+ :avocado: tags=startvm,fast,full
+ """
+ def test_arm_bullseye(self):
+ self.init()
+ self.vm_start('arm','bullseye')
+
+ def test_arm_buster(self):
+ self.init()
+ self.vm_start('arm','buster')
+
+ def test_arm64_bullseye(self):
+ self.init()
+ self.vm_start('arm64','bullseye')
+
+ def test_amd64_bullseye(self):
+ self.init()
+ self.vm_start('amd64','bullseye')
+
+class VmBootTestFull(CIBaseTest):
+
+ """
+ Test QEMU image start (full)
+
+ :avocado: tags=startvm,full
+ """
+ def test_i386_stretch(self):
+ self.init()
+ self.vm_start('i386','stretch')
+
+ def test_i386_buster(self):
+ self.init()
+ self.vm_start('i386','buster')
+
+ def test_amd64_buster(self):
+ self.init()
+ self.vm_start('amd64','buster')
+
+ def test_amd64_focal(self):
+ self.init()
+ self.vm_start('amd64','focal')
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index f0c4bc3..2170ea6 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -5,6 +5,10 @@ import os
import select
import shutil
import subprocess
+import time
+import tempfile
+
+import start_vm
from avocado import Test
from avocado.utils import path
@@ -15,6 +19,9 @@ backup_prefix = '.ci-backup'
app_log = logging.getLogger("avocado.app")
+class CanBeFinished(Exception):
+ pass
+
class CIBuilder(Test):
def setUp(self):
super(CIBuilder, self).setUp()
@@ -181,3 +188,64 @@ class CIBuilder(Test):
for x in output.splitlines() if x != ''))
return env['LAYERDIR_' + layer].strip('"')
+
+ def vm_start(self, arch='amd64', distro='buster'):
+ time_to_wait = self.params.get('time_to_wait', default=60)
+
+ self.log.info('===================================================')
+ self.log.info('Running Isar VM boot test for (' + distro + '-' + arch + ')')
+ self.log.info('Isar build folder is: ' + self.build_dir)
+ self.log.info('===================================================')
+
+ self.check_init()
+
+ fd, output_file = tempfile.mkstemp(suffix='_log.txt',
+ prefix='vm_start_' + distro + '_' +
+ arch + '_', dir=self.build_dir, text=True)
+ os.chmod(output_file, 0o644)
+
+ cmdline = start_vm.format_qemu_cmdline(arch, self.build_dir, distro,
+ output_file, None)
+ cmdline.insert(1, '-nographic')
+
+ self.log.info('QEMU boot line: ' + str(cmdline))
+
+ login_prompt = b'isar login:'
+ service_prompt = b'Just an example'
+
+ timeout = time.time() + int(time_to_wait)
+
+ p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ try:
+ poller = select.poll()
+ poller.register(p1.stdout, select.POLLIN)
+ poller.register(p1.stderr, select.POLLIN)
+ while time.time() < timeout and p1.poll() is None:
+ events = poller.poll(1000 * (timeout - time.time()))
+ for fd, event in events:
+ if fd == p1.stdout.fileno():
+ # Wait for the complete string if it is read in chunks
+ # like "i", "sar", " login:"
+ time.sleep(0.01)
+ data = os.read(fd, 1024)
+ if login_prompt in data:
+ raise CanBeFinished
+ if fd == p1.stderr.fileno():
+ self.log.error(p1.stderr.readline())
+ except CanBeFinished:
+ self.log.debug('Got login prompt')
+ finally:
+ if p1.poll() is None:
+ p1.kill()
+ p1.wait()
+
+ if os.path.exists(output_file):
+ with open(output_file, "rb") as f1:
+ data = f1.read()
+ if service_prompt in data and login_prompt in data:
+ return
+ else:
+ self.log.error(data)
+
+ self.fail('Log ' + output_file)
diff --git a/testsuite/vm_boot_test.py b/testsuite/vm_boot_test.py
deleted file mode 100755
index 6c4e979..0000000
--- a/testsuite/vm_boot_test.py
+++ /dev/null
@@ -1,132 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import select
-import subprocess
-import sys
-import time
-import tempfile
-
-from os.path import dirname
-
-import start_vm
-
-from avocado import Test
-from avocado.utils import process
-from avocado.utils import path
-
-class CanBeFinished(Exception):
- pass
-
-class VmBase(Test):
-
- def check_bitbake(self, build_dir):
- try:
- path.find_command('bitbake')
- except path.CmdNotFoundError:
- out = process.getoutput('/bin/bash -c "cd ../.. && \
- source isar-init-build-env \
- %s 2>&1 >/dev/null; env"' % build_dir)
- env = dict(((x.split('=', 1) + [''])[:2] \
- for x in output.splitlines() if x != ''))
- os.environ.update(env)
-
- def vm_start(self, arch='amd64', distro='buster'):
- build_dir = self.params.get('build_dir', default='.')
- time_to_wait = self.params.get('time_to_wait', default=60)
-
- self.log.info('===================================================')
- self.log.info('Running Isar VM boot test for (' + distro + '-' + arch + ')')
- self.log.info('Isar build folder is: ' + build_dir)
- self.log.info('===================================================')
-
- self.check_bitbake(build_dir)
-
- fd, output_file = tempfile.mkstemp(suffix='_log.txt',
- prefix='vm_start_' + distro + '_' +
- arch + '_', dir=build_dir, text=True)
- os.chmod(output_file, 0o644)
-
- cmdline = start_vm.format_qemu_cmdline(arch, build_dir, distro,
- output_file, None)
- cmdline.insert(1, '-nographic')
-
- self.log.info('QEMU boot line: ' + str(cmdline))
-
- login_prompt = b'isar login:'
- service_prompt = b'Just an example'
-
- timeout = time.time() + int(time_to_wait)
-
- p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE,
- stderr=subprocess.PIPE)
- try:
- poller = select.poll()
- poller.register(p1.stdout, select.POLLIN)
- poller.register(p1.stderr, select.POLLIN)
- while time.time() < timeout and p1.poll() is None:
- events = poller.poll(1000 * (timeout - time.time()))
- for fd, event in events:
- if fd == p1.stdout.fileno():
- # Wait for the complete string if it is read in chunks
- # like "i", "sar", " login:"
- time.sleep(0.01)
- data = os.read(fd, 1024)
- if login_prompt in data:
- raise CanBeFinished
- if fd == p1.stderr.fileno():
- self.log.error(p1.stderr.readline())
- except CanBeFinished:
- self.log.debug('Got login prompt')
- finally:
- if p1.poll() is None:
- p1.kill()
- p1.wait()
-
- if os.path.exists(output_file):
- with open(output_file, "rb") as f1:
- data = f1.read()
- if service_prompt in data and login_prompt in data:
- return
- else:
- self.log.error(data)
-
- self.fail('Log ' + output_file)
-
-class VmBootTestFast(VmBase):
-
- """
- Test QEMU image start (fast)
-
- :avocado: tags=fast,full
- """
- def test_arm_bullseye(self):
- self.vm_start('arm','bullseye')
-
- def test_arm_buster(self):
- self.vm_start('arm','buster')
-
- def test_arm64_bullseye(self):
- self.vm_start('arm64','bullseye')
-
- def test_amd64_bullseye(self):
- self.vm_start('amd64','bullseye')
-
-class VmBootTestFull(VmBase):
-
- """
- Test QEMU image start (full)
-
- :avocado: tags=full
- """
- def test_i386_stretch(self):
- self.vm_start('i386','stretch')
-
- def test_i386_buster(self):
- self.vm_start('i386','buster')
-
- def test_amd64_buster(self):
- self.vm_start('amd64','buster')
-
- def test_amd64_focal(self):
- self.vm_start('amd64','focal')
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v3 3/7] CI: Adopt vm_start test logging
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 1/7] CI: Reorganize testsuite folders Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 2/7] CI: Migrate all tests to one location Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 4/7] CI: Do not run QEMU start tests from Gitlab Anton Mikanovich
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Adopt stderr output format to made QEMU start tests output logs in the
same way like build tests are.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 2170ea6..52bb2a5 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -216,7 +216,7 @@ class CIBuilder(Test):
timeout = time.time() + int(time_to_wait)
p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE,
- stderr=subprocess.PIPE)
+ stderr=subprocess.PIPE, universal_newlines=True)
try:
poller = select.poll()
poller.register(p1.stdout, select.POLLIN)
@@ -224,6 +224,8 @@ class CIBuilder(Test):
while time.time() < timeout and p1.poll() is None:
events = poller.poll(1000 * (timeout - time.time()))
for fd, event in events:
+ if event != select.POLLIN:
+ continue
if fd == p1.stdout.fileno():
# Wait for the complete string if it is read in chunks
# like "i", "sar", " login:"
@@ -232,7 +234,7 @@ class CIBuilder(Test):
if login_prompt in data:
raise CanBeFinished
if fd == p1.stderr.fileno():
- self.log.error(p1.stderr.readline())
+ app_log.error(p1.stderr.readline().rstrip())
except CanBeFinished:
self.log.debug('Got login prompt')
finally:
@@ -240,12 +242,12 @@ class CIBuilder(Test):
p1.kill()
p1.wait()
- if os.path.exists(output_file):
+ if os.path.exists(output_file) and os.path.getsize(output_file) > 0:
with open(output_file, "rb") as f1:
data = f1.read()
if service_prompt in data and login_prompt in data:
return
else:
- self.log.error(data)
+ app_log.error(data.decode(errors='replace'))
self.fail('Log ' + output_file)
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v3 4/7] CI: Do not run QEMU start tests from Gitlab
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
` (2 preceding siblings ...)
2022-03-16 7:16 ` [PATCH v3 3/7] CI: Adopt vm_start test logging Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 5/7] CI: Rename build_test.py to citest.py Anton Mikanovich
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To leave existing behaviour of vm_start tests execution introduce
'--norun' flag to use it in .gitlab-ci.yml scripts.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
.gitlab-ci.yml | 4 ++--
scripts/ci_build.sh | 8 ++++++++
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 37882ef..d0488a0 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,7 +27,7 @@ fast-ci:
except:
- schedules
script:
- - scripts/ci_build.sh -q -f -d
+ - scripts/ci_build.sh -q -f -d -n
full-ci:
<<: *common-build
@@ -37,7 +37,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -c -r -d;
+ scripts/ci_build.sh -q -c -r -d -n;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index fe139d1..ea69553 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -52,6 +52,7 @@ show_help() {
echo " -f, --fast cross build reduced set of configurations."
echo " -q, --quiet suppress verbose bitbake output."
echo " -r, --repro enable use of cached base repository."
+ echo " -n, --norun do not execute QEMU run tests."
echo " --help display this message and exit."
echo
echo "Exit status:"
@@ -100,6 +101,9 @@ do
-s|--sign) shift ;;
esac
;;
+ -n|--norun)
+ NORUN="1"
+ ;;
*)
echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
exit $ES_BUG
@@ -113,6 +117,10 @@ if [ -z "$REPRO_BUILD" ]; then
TAGS="$TAGS,-repro"
fi
+if [ -n "$NORUN" ]; then
+ TAGS="$TAGS,-startvm"
+fi
+
# Provide working path
mkdir -p .config/avocado
cat <<EOF > .config/avocado/avocado.conf
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v3 5/7] CI: Rename build_test.py to citest.py
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
` (3 preceding siblings ...)
2022-03-16 7:16 ` [PATCH v3 4/7] CI: Do not run QEMU start tests from Gitlab Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 6/7] CI: Change QEMU boot logs format Anton Mikanovich
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
It is no more just build test after adding QEMU start testing.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 2 +-
scripts/vm_smoke_test | 2 +-
testsuite/{build_test.py => citest.py} | 0
3 files changed, 2 insertions(+), 2 deletions(-)
rename testsuite/{build_test.py => citest.py} (100%)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ea69553..41257cb 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -135,6 +135,6 @@ export VIRTUAL_ENV="./"
# the real stuff starts here, trace commands from now on
set -x
-avocado $VERBOSE run "$TESTSUITE_DIR/build_test.py" \
+avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
-t $TAGS --test-runner=runner --disable-sysinfo \
-p quiet=$QUIET -p cross=$CROSS_BUILD
diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
index 28c959a..5c16d86 100755
--- a/scripts/vm_smoke_test
+++ b/scripts/vm_smoke_test
@@ -95,7 +95,7 @@ logs_dir = $BUILD_DIR/job-results
EOF
export VIRTUAL_ENV="./"
-if avocado $VERBOSE run "$TESTSUITE_DIR/build_test.py" -t $TAGS,startvm \
+if avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" -t $TAGS,startvm \
--test-runner=runner --disable-sysinfo \
-p build_dir="$BUILD_DIR" -p time_to_wait=$TIMEOUT; then
RET=$ES_OK
diff --git a/testsuite/build_test.py b/testsuite/citest.py
similarity index 100%
rename from testsuite/build_test.py
rename to testsuite/citest.py
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v3 6/7] CI: Change QEMU boot logs format
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
` (4 preceding siblings ...)
2022-03-16 7:16 ` [PATCH v3 5/7] CI: Rename build_test.py to citest.py Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-16 7:16 ` [PATCH v3 7/7] CI: Add timeout setting to ci_build Anton Mikanovich
2022-03-23 7:00 ` [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Move startvm boot logs into separate location with log datetime and
'latest' log symlinks.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 52bb2a5..069a1d9 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -199,10 +199,18 @@ class CIBuilder(Test):
self.check_init()
- fd, output_file = tempfile.mkstemp(suffix='_log.txt',
- prefix='vm_start_' + distro + '_' +
- arch + '_', dir=self.build_dir, text=True)
+ logdir = '%s/vm_start' % self.build_dir
+ if not os.path.exists(logdir):
+ os.mkdir(logdir)
+ prefix = '%s-vm_start_%s_%s_' % (time.strftime('%Y%m%d-%H%M%S'),
+ distro, arch)
+ fd, output_file = tempfile.mkstemp(suffix='_log.txt', prefix=prefix,
+ dir=logdir, text=True)
os.chmod(output_file, 0o644)
+ latest_link = '%s/vm_start_%s_%s_latest.txt' % (logdir, distro, arch)
+ if os.path.exists(latest_link):
+ os.unlink(latest_link)
+ os.symlink(os.path.basename(output_file), latest_link)
cmdline = start_vm.format_qemu_cmdline(arch, self.build_dir, distro,
output_file, None)
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v3 7/7] CI: Add timeout setting to ci_build
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
` (5 preceding siblings ...)
2022-03-16 7:16 ` [PATCH v3 6/7] CI: Change QEMU boot logs format Anton Mikanovich
@ 2022-03-16 7:16 ` Anton Mikanovich
2022-03-23 7:00 ` [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-16 7:16 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Migrate this option from vm_moke_test.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 41257cb..2513f1a 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -53,6 +53,8 @@ show_help() {
echo " -q, --quiet suppress verbose bitbake output."
echo " -r, --repro enable use of cached base repository."
echo " -n, --norun do not execute QEMU run tests."
+ echo " -t, --timeout SEC specify time in seconds to wait before stop QEMU."
+ echo " The default is: 300"
echo " --help display this message and exit."
echo
echo "Exit status:"
@@ -63,6 +65,7 @@ show_help() {
TAGS="full"
CROSS_BUILD="0"
QUIET="0"
+TIMEOUT=300
# Parse command line to get user configuration
while [ $# -gt 0 ]
@@ -104,6 +107,10 @@ do
-n|--norun)
NORUN="1"
;;
+ -t|--timeout)
+ TIMEOUT=$2
+ shift
+ ;;
*)
echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
exit $ES_BUG
@@ -137,4 +144,4 @@ set -x
avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
-t $TAGS --test-runner=runner --disable-sysinfo \
- -p quiet=$QUIET -p cross=$CROSS_BUILD
+ -p quiet=$QUIET -p cross=$CROSS_BUILD -p time_to_wait=$TIMEOUT
--
2.17.1
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v3 0/7] Unify build and run test cases
2022-03-16 7:16 [PATCH v3 0/7] Unify build and run test cases Anton Mikanovich
` (6 preceding siblings ...)
2022-03-16 7:16 ` [PATCH v3 7/7] CI: Add timeout setting to ci_build Anton Mikanovich
@ 2022-03-23 7:00 ` Anton Mikanovich
7 siblings, 0 replies; 9+ messages in thread
From: Anton Mikanovich @ 2022-03-23 7:00 UTC (permalink / raw)
To: isar-users
16.03.2022 10:16, Anton Mikanovich wrote:
> Make both build and start_vm CI tests run from one entry point to be
> able to manage CI in more flexible way.
> Also move QEMU boot logs into subdir to not flood ./build.
>
> Changes since v2:
> - Move timeout setting to unified test.
> Changes since v1:
> - Fix key paths for repro test.
>
> Anton Mikanovich (7):
> CI: Reorganize testsuite folders
> CI: Migrate all tests to one location
> CI: Adopt vm_start test logging
> CI: Do not run QEMU start tests from Gitlab
> CI: Rename build_test.py to citest.py
> CI: Change QEMU boot logs format
> CI: Add timeout setting to ci_build
>
> .gitlab-ci.yml | 4 +-
> scripts/ci_build.sh | 21 ++-
> scripts/vm_smoke_test | 4 +-
> testsuite/README.md | 26 +++-
> testsuite/build_test/run_fast.sh | 3 -
> testsuite/build_test/run_full.sh | 3 -
> testsuite/{build_test => }/cibase.py | 4 +-
> testsuite/{build_test => }/cibuilder.py | 80 ++++++++++-
> .../{build_test/build_test.py => citest.py} | 46 ++++++
> testsuite/{ => keys}/base-apt/test_priv.key | 0
> testsuite/{ => keys}/base-apt/test_pub.key | 0
> testsuite/vm_boot_test/run_fast.sh | 3 -
> testsuite/vm_boot_test/run_full.sh | 3 -
> testsuite/vm_boot_test/vm_boot_test.py | 133 ------------------
> 14 files changed, 172 insertions(+), 158 deletions(-)
> delete mode 100755 testsuite/build_test/run_fast.sh
> delete mode 100755 testsuite/build_test/run_full.sh
> rename testsuite/{build_test => }/cibase.py (97%)
> mode change 100644 => 100755
> rename testsuite/{build_test => }/cibuilder.py (69%)
> mode change 100644 => 100755
> rename testsuite/{build_test/build_test.py => citest.py} (87%)
> mode change 100644 => 100755
> rename testsuite/{ => keys}/base-apt/test_priv.key (100%)
> rename testsuite/{ => keys}/base-apt/test_pub.key (100%)
> delete mode 100755 testsuite/vm_boot_test/run_fast.sh
> delete mode 100755 testsuite/vm_boot_test/run_full.sh
> delete mode 100644 testsuite/vm_boot_test/vm_boot_test.py
Applied to next.
^ permalink raw reply [flat|nested] 9+ messages in thread