* [PATCH v10 01/17] start_vm.py: Fix target name handling
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 02/17] start_vm.py: Add output and PID file vm_start.py options Anton Mikanovich
` (17 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Roman Pletnev, Anton Mikanovich
From: Roman Pletnev <rpletnev@ilbers.de>
This patch fix bb_output search (issue with wrong arch name being
selected for qemu) which caused qemu-system-riscv64 to be used for all
the targets:
>ERROR: [Errno 2] No such file or directory: u'qemu-system-riscv64' (1.05 s)
Signed-off-by: Roman Pletnev <rpletnev@ilbers.de>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/start_vm.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index a3e32ac..c10db67 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -17,7 +17,7 @@ def get_bitbake_env(arch, distro):
def get_bitbake_var(output, var):
ret = ''
for line in output.splitlines():
- if line.startswith(var):
+ if line.startswith(var + '='):
ret = line.split('"')[1]
return ret
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 02/17] start_vm.py: Add output and PID file vm_start.py options
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 01/17] start_vm.py: Fix target name handling Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 03/17] start_vm.py: Add MIPS support Anton Mikanovich
` (16 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Roman Pletnev, Anton Mikanovich
From: Roman Pletnev <rpletnev@ilbers.de>
This patch adds options -o (output file) and -p (pid file).
Implement virtual machine log output both to the file and stdout.
Signed-off-by: Roman Pletnev <rpletnev@ilbers.de>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/start_vm.py | 18 ++++++++++++++----
testsuite/vm_boot_test/vm_boot_test.py | 5 ++---
2 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index c10db67..3736e51 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -21,7 +21,7 @@ def get_bitbake_var(output, var):
ret = line.split('"')[1]
return ret
-def format_qemu_cmdline(arch, build, distro):
+def format_qemu_cmdline(arch, build, distro, out, pid):
bb_output = get_bitbake_env(arch, distro).decode()
rootfs_image = ''
@@ -57,6 +57,13 @@ def format_qemu_cmdline(arch, build, distro):
qemu_cpu = get_bitbake_var(bb_output, 'QEMU_CPU')
qemu_disk_args = get_bitbake_var(bb_output, 'QEMU_DISK_ARGS')
+ if out:
+ extra_args.extend(['-chardev','stdio,id=ch0,logfile=' + out])
+ extra_args.extend(['-serial','chardev:ch0'])
+ extra_args.extend(['-monitor','none'])
+ if pid:
+ extra_args.extend(['-pidfile', pid])
+
qemu_disk_args = qemu_disk_args.replace('##ROOTFS_IMAGE##', deploy_dir_image + '/' + rootfs_image).split()
cmd = ['qemu-system-' + qemu_arch, '-m', '1024M']
@@ -72,10 +79,11 @@ def format_qemu_cmdline(arch, build, distro):
return cmd
-def start_qemu(arch, build, distro):
- cmdline = format_qemu_cmdline(arch, build, distro)
+def start_qemu(arch, build, distro, out, pid):
+ cmdline = format_qemu_cmdline(arch, build, distro, out, pid)
cmdline.insert(1, '-nographic')
+ print(cmdline)
p1 = subprocess.call(cmdline)
if __name__ == "__main__":
@@ -83,6 +91,8 @@ if __name__ == "__main__":
parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386'], help='set isar machine architecture.', default='arm')
parser.add_argument('-b', '--build', help='set path to build directory.', default=os.getcwd())
parser.add_argument('-d', '--distro', choices=['jessie', 'stretch'], help='set isar Debian distribution.', default='stretch')
+ parser.add_argument('-o', '--out', help='Route QEMU console output to specified file.')
+ parser.add_argument('-p', '--pid', help='Store QEMU pid to specified file.')
args = parser.parse_args()
- start_qemu(args.arch, args.build, args.distro)
+ start_qemu(args.arch, args.build, args.distro, args.out, args.pid)
diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py
index d4849c7..96e02f7 100644
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test/vm_boot_test.py
@@ -30,10 +30,9 @@ class VmBootTest(Test):
if os.path.exists(output_file):
os.remove(output_file)
- cmdline = start_vm.format_qemu_cmdline(arch, build_dir, distro)
+ cmdline = start_vm.format_qemu_cmdline(arch, build_dir, distro,
+ output_file, None)
cmdline.insert(1, '-nographic')
- cmdline.append('-serial')
- cmdline.append('file:' + output_file)
self.log.info('QEMU boot line: ' + str(cmdline))
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 03/17] start_vm.py: Add MIPS support
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 01/17] start_vm.py: Fix target name handling Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 02/17] start_vm.py: Add output and PID file vm_start.py options Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 04/17] start_vm.py: Fix ubuntu image name Anton Mikanovich
` (15 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Roman Pletnev, Anton Mikanovich
From: Roman Pletnev <rpletnev@ilbers.de>
This patch adds mipsel to the list of supported architectures.
Signed-off-by: Roman Pletnev <rpletnev@ilbers.de>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/start_vm.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index 3736e51..face747 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -88,7 +88,7 @@ def start_qemu(arch, build, distro, out, pid):
if __name__ == "__main__":
parser = argparse.ArgumentParser()
- parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386'], help='set isar machine architecture.', default='arm')
+ parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386', 'mipsel'], help='set isar machine architecture.', default='arm')
parser.add_argument('-b', '--build', help='set path to build directory.', default=os.getcwd())
parser.add_argument('-d', '--distro', choices=['jessie', 'stretch'], help='set isar Debian distribution.', default='stretch')
parser.add_argument('-o', '--out', help='Route QEMU console output to specified file.')
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 04/17] start_vm.py: Fix ubuntu image name
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (2 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 03/17] start_vm.py: Add MIPS support Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 05/17] vm_boot_test: Fix log file path in vm_boot_test Anton Mikanovich
` (14 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Choose 'ubuntu' base name for 'focal' target, not 'debian'.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/start_vm.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index face747..8f0ccd6 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -30,8 +30,9 @@ def format_qemu_cmdline(arch, build, distro, out, pid):
image_type = get_bitbake_var(bb_output, 'IMAGE_TYPE')
deploy_dir_image = get_bitbake_var(bb_output, 'DEPLOY_DIR_IMAGE')
+ base = 'ubuntu' if distro in ['focal', 'bionic'] else 'debian'
if image_type == 'ext4-img':
- rootfs_image = 'isar-image-base-debian-' + distro + '-qemu' + arch + '.ext4.img'
+ rootfs_image = 'isar-image-base-' + base + '-' + distro + '-qemu' + arch + '.ext4.img'
kernel_image = deploy_dir_image + '/' + get_bitbake_var(bb_output, 'KERNEL_IMAGE')
initrd_image = get_bitbake_var(bb_output, 'INITRD_IMAGE')
@@ -47,7 +48,7 @@ def format_qemu_cmdline(arch, build, distro, out, pid):
extra_args = ['-kernel', kernel_image, '-initrd', initrd_image]
extra_args.extend(kargs)
elif image_type == 'wic-img':
- rootfs_image = 'isar-image-base-debian-' + distro + '-qemu' + arch + '.wic.img'
+ rootfs_image = 'isar-image-base-' + base + '-' + distro + '-qemu' + arch + '.wic.img'
extra_args = ['-snapshot']
else:
raise ValueError('Invalid image type: ' + str(image_type))
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 05/17] vm_boot_test: Fix log file path in vm_boot_test
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (3 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 04/17] start_vm.py: Fix ubuntu image name Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 06/17] vm_boot_test: Remove external varianter Anton Mikanovich
` (13 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Make log filename random and store it in build folder.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/vm_boot_test/vm_boot_test.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py
index 96e02f7..64dfda6 100644
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test/vm_boot_test.py
@@ -4,6 +4,7 @@ import os
import subprocess32
import sys
import time
+import tempfile
from os.path import dirname
sys.path.append(dirname(__file__) + '/..')
@@ -26,9 +27,9 @@ class VmBootTest(Test):
self.log.info('Isar build folder is: ' + build_dir)
self.log.info('===================================================')
- output_file = '/tmp/vm_boot_test.log'
- if os.path.exists(output_file):
- os.remove(output_file)
+ fd, output_file = tempfile.mkstemp(suffix='_log.txt',
+ prefix='vm_start_' + distro + '_' +
+ arch + '_', dir=build_dir, text=True)
cmdline = start_vm.format_qemu_cmdline(arch, build_dir, distro,
output_file, None)
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 06/17] vm_boot_test: Remove external varianter
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (4 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 05/17] vm_boot_test: Fix log file path in vm_boot_test Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 07/17] vm_boot_test: Improve QEMU images checking Anton Mikanovich
` (12 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To get rid of avocado-framework-plugin-varianter-yaml-to-mux pip
dependency we should not use yaml-to-mux parameters.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/vm_boot_test/run.sh | 3 --
testsuite/vm_boot_test/run_fast.sh | 3 ++
testsuite/vm_boot_test/run_full.sh | 3 ++
testsuite/vm_boot_test/variant.yaml | 22 -------------
testsuite/vm_boot_test/vm_boot_test.py | 45 +++++++++++++++++++++++---
5 files changed, 46 insertions(+), 30 deletions(-)
delete mode 100755 testsuite/vm_boot_test/run.sh
create mode 100755 testsuite/vm_boot_test/run_fast.sh
create mode 100755 testsuite/vm_boot_test/run_full.sh
delete mode 100644 testsuite/vm_boot_test/variant.yaml
diff --git a/testsuite/vm_boot_test/run.sh b/testsuite/vm_boot_test/run.sh
deleted file mode 100755
index 9fdda95..0000000
--- a/testsuite/vm_boot_test/run.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-avocado run vm_boot_test.py --mux-yaml test:variant.yaml --mux-inject build_dir:$BUILDDIR time_to_wait:300
diff --git a/testsuite/vm_boot_test/run_fast.sh b/testsuite/vm_boot_test/run_fast.sh
new file mode 100755
index 0000000..0fc77b3
--- /dev/null
+++ b/testsuite/vm_boot_test/run_fast.sh
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+avocado run vm_boot_test.py -t fast -p build_dir="$BUILDDIR" -p time_to_wait=300
diff --git a/testsuite/vm_boot_test/run_full.sh b/testsuite/vm_boot_test/run_full.sh
new file mode 100755
index 0000000..a561a36
--- /dev/null
+++ b/testsuite/vm_boot_test/run_full.sh
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+avocado run vm_boot_test.py -t full -p build_dir="$BUILDDIR" -p time_to_wait=300
diff --git a/testsuite/vm_boot_test/variant.yaml b/testsuite/vm_boot_test/variant.yaml
deleted file mode 100644
index 9ddc634..0000000
--- a/testsuite/vm_boot_test/variant.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-variants: !mux
- stretch-amd64:
- distro: "stretch"
- arch: "amd64"
- stretch-i386:
- distro: "stretch"
- arch: "i386"
- stretch-arm:
- distro: "stretch"
- arch: "arm"
- stretch-arm64:
- distro: "stretch"
- arch: "arm64"
- buster-amd64:
- distro: "buster"
- arch: "amd64"
- buster-i386:
- distro: "buster"
- arch: "i386"
- buster-arm:
- distro: "buster"
- arch: "arm"
diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py
index 64dfda6..eee14b2 100644
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test/vm_boot_test.py
@@ -13,13 +13,10 @@ import start_vm
from avocado import Test
-class VmBootTest(Test):
+class VmBase(Test):
- def test(self):
- # TODO: add default values
+ def vm_start(self, arch='amd64', distro='buster'):
build_dir = self.params.get('build_dir', default='.')
- arch = self.params.get('arch', default='arm')
- distro = self.params.get('distro', default='stretch')
time_to_wait = self.params.get('time_to_wait', default=60)
self.log.info('===================================================')
@@ -49,3 +46,41 @@ class VmBootTest(Test):
return
self.fail('Test failed')
+
+class VmBootTestFast(VmBase):
+
+ """
+ Test QEMU image start (fast)
+
+ :avocado: tags=fast,full
+ """
+ def test_arm_stretch(self):
+ self.vm_start('arm','stretch')
+
+ def test_arm_buster(self):
+ self.vm_start('arm','buster')
+
+ def test_arm64_stretch(self):
+ self.vm_start('arm64','stretch')
+
+ def test_amd64_stretch(self):
+ self.vm_start('amd64','stretch')
+
+class VmBootTestFull(VmBase):
+
+ """
+ Test QEMU image start (full)
+
+ :avocado: tags=full
+ """
+ def test_i386_stretch(self):
+ self.vm_start('i386','stretch')
+
+ def test_i386_buster(self):
+ self.vm_start('i386','buster')
+
+ def test_amd64_buster(self):
+ self.vm_start('amd64','buster')
+
+ def test_amd64_focal(self):
+ self.vm_start('amd64','focal')
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 07/17] vm_boot_test: Improve QEMU images checking
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (5 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 06/17] vm_boot_test: Remove external varianter Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 08/17] build_test: Refactoring build tests cases Anton Mikanovich
` (11 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Makes QEMU start test to analyze boot log in real-time. It helps test
cases to finish as soon as booting succeeds and do not wasting time on
waiting.
Get rid of python-subprocess32 backport package.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/vm_boot_test/vm_boot_test.py | 53 ++++++++++++++++++++------
1 file changed, 42 insertions(+), 11 deletions(-)
diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py
index eee14b2..6e580d0 100644
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test/vm_boot_test.py
@@ -1,7 +1,8 @@
#!/usr/bin/env python3
import os
-import subprocess32
+import select
+import subprocess
import sys
import time
import tempfile
@@ -13,6 +14,9 @@ import start_vm
from avocado import Test
+class CanBeFinished(Exception):
+ pass
+
class VmBase(Test):
def vm_start(self, arch='amd64', distro='buster'):
@@ -34,18 +38,45 @@ class VmBase(Test):
self.log.info('QEMU boot line: ' + str(cmdline))
- devnull = open(os.devnull, 'w')
-
- p1 = subprocess32.Popen(cmdline, stdout=devnull, stderr=devnull)
- time.sleep(int(time_to_wait))
- p1.kill()
- p1.wait()
+ login_prompt = b'isar login:'
+ service_prompt = b'Just an example'
+
+ timeout = time.time() + int(time_to_wait)
+
+ p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ try:
+ poller = select.poll()
+ poller.register(p1.stdout, select.POLLIN)
+ poller.register(p1.stderr, select.POLLIN)
+ while time.time() < timeout and p1.poll() is None:
+ events = poller.poll(1000 * (timeout - time.time()))
+ for fd, event in events:
+ if fd == p1.stdout.fileno():
+ # Wait for the complete string if it is read in chunks
+ # like "i", "sar", " login:"
+ time.sleep(0.01)
+ data = os.read(fd, 1024)
+ if login_prompt in data:
+ raise CanBeFinished
+ if fd == p1.stderr.fileno():
+ self.log.error(p1.stderr.readline())
+ except CanBeFinished:
+ self.log.debug('Got login prompt')
+ finally:
+ if p1.poll() is None:
+ p1.kill()
+ p1.wait()
if os.path.exists(output_file):
- if 'isar login:' in open(output_file).read():
- return
-
- self.fail('Test failed')
+ with open(output_file, "rb") as f1:
+ data = f1.read()
+ if service_prompt in data and login_prompt in data:
+ return
+ else:
+ self.log.error(data)
+
+ self.fail('Log ' + output_file)
class VmBootTestFast(VmBase):
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 08/17] build_test: Refactoring build tests cases
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (6 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 07/17] vm_boot_test: Improve QEMU images checking Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 09/17] testsuite: Add Python generations for testsuite in gitignore Anton Mikanovich
` (10 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Split build test into classes and separate cases. Implement test cases
tagging for external configuring. Also remove the usage of external
varianter plugin.
In this step, all ci_build test cases were copied as-is without
functional changes.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/build_test/build_test.py | 208 ++++++++++++++++++++++++++---
testsuite/build_test/cibase.py | 110 +++++++++++++++
testsuite/build_test/cibuilder.py | 130 ++++++++++++++++++
testsuite/build_test/run.sh | 3 -
testsuite/build_test/run_fast.sh | 3 +
testsuite/build_test/run_full.sh | 3 +
testsuite/build_test/variant.yaml | 22 ---
7 files changed, 434 insertions(+), 45 deletions(-)
create mode 100644 testsuite/build_test/cibase.py
create mode 100644 testsuite/build_test/cibuilder.py
delete mode 100755 testsuite/build_test/run.sh
create mode 100755 testsuite/build_test/run_fast.sh
create mode 100755 testsuite/build_test/run_full.sh
delete mode 100644 testsuite/build_test/variant.yaml
diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py
index 7a55c2f..9038ec5 100644
--- a/testsuite/build_test/build_test.py
+++ b/testsuite/build_test/build_test.py
@@ -1,29 +1,197 @@
#!/usr/bin/env python3
import os
-import subprocess32
-import sys
-from os.path import dirname
-from avocado import Test
+from avocado import skipUnless
+from avocado.utils import path
+from cibase import CIBaseTest
-class BuildTest(Test):
+UMOCI_AVAILABLE = True
+SKOPEO_AVAILABLE = True
+try:
+ path.find_command('umoci')
+except path.CmdNotFoundError:
+ UMOCI_AVAILABLE = False
+try:
+ path.find_command('skopeo')
+except path.CmdNotFoundError:
+ SKOPEO_AVAILABLE = False
- def test(self):
- # TODO: add default values
- build_dir = self.params.get('build_dir', default='.')
- arch = self.params.get('arch', default='arm')
- distro = self.params.get('distro', default='stretch')
+class ReproTest(CIBaseTest):
- self.log.info('===================================================')
- self.log.info('Running Isar build test for (' + distro + '-' + arch + ')')
- self.log.info('Isar build folder is: ' + build_dir)
- self.log.info('===================================================')
+ """
+ Test cached base repository
- #isar_root = dirname(__file__) + '/..'
- os.chdir(build_dir)
- cmdline = ['bitbake', 'mc:qemu' + arch + '-' + distro + ':isar-image-base']
- p1 = subprocess32.run(cmdline)
+ :avocado: tags=repro,fast,full
+ """
+ def test_repro_signed(self):
+ targets = [
+ 'mc:de0-nano-soc-buster:isar-image-base',
+ 'mc:qemuarm64-stretch:isar-image-base'
+ ]
- if p1.returncode:
- self.fail('Test failed')
+ self.perform_repro_test(targets, 1)
+
+ def test_repro_unsigned(self):
+ targets = [
+ 'mc:qemuamd64-stretch:isar-image-base',
+ 'mc:qemuarm-stretch:isar-image-base'
+ ]
+
+ self.perform_repro_test(targets, 0)
+
+class CrossTest(CIBaseTest):
+
+ """
+ Start cross build for the defined set of configurations
+
+ :avocado: tags=cross,fast,full
+ """
+ def test_cross(self):
+ targets = [
+ 'mc:qemuarm-stretch:isar-image-base',
+ 'mc:qemuarm-buster:isar-image-base',
+ 'mc:qemuarm64-stretch:isar-image-base',
+ 'mc:qemuamd64-stretch:isar-image-base',
+ 'mc:de0-nano-soc-buster:isar-image-base',
+ 'mc:stm32mp15x-buster:isar-image-base',
+ 'mc:rpi-stretch:isar-image-base',
+ 'mc:qemuarm64-focal:isar-image-base'
+ ]
+
+ self.perform_build_test(targets, 1, None)
+
+ def test_cross_bullseye(self):
+ targets = [
+ 'mc:qemuarm-bullseye:isar-image-base'
+ ]
+
+ try:
+ self.perform_build_test(targets, 1, None)
+ except:
+ self.cancel('KFAIL')
+
+class SdkTest(CIBaseTest):
+
+ """
+ In addition test SDK creation
+
+ :avocado: tags=sdk,fast,full
+ """
+ def test_sdk(self):
+ targets = ['mc:qemuarm-stretch:isar-image-base']
+
+ self.perform_build_test(targets, 1, 'do_populate_sdk')
+
+class NoCrossTest(CIBaseTest):
+
+ """
+ Start non-cross build for the defined set of configurations
+
+ :avocado: tags=nocross,full
+ """
+ def test_nocross(self):
+ targets = [
+ 'mc:qemuarm-stretch:isar-image-base',
+ 'mc:qemuarm-buster:isar-image-base',
+ 'mc:qemuarm64-stretch:isar-image-base',
+ 'mc:qemui386-stretch:isar-image-base',
+ 'mc:qemui386-buster:isar-image-base',
+ 'mc:qemuamd64-stretch:isar-image-base',
+ 'mc:qemuamd64-buster:isar-image-base',
+ 'mc:qemuamd64-buster-tgz:isar-image-base',
+ 'mc:qemuamd64-buster:isar-initramfs',
+ 'mc:qemumipsel-stretch:isar-image-base',
+ 'mc:qemumipsel-buster:isar-image-base',
+ 'mc:nand-ubi-demo-buster:isar-image-ubi',
+ 'mc:rpi-stretch:isar-image-base',
+ 'mc:qemuamd64-focal:isar-image-base'
+ ]
+
+ # Cleanup after cross build
+ self.deletetmp(self.params.get('build_dir',
+ default=os.path.dirname(__file__) + '/../../build'))
+
+ self.perform_build_test(targets, 0, None)
+
+ def test_nocross_bullseye(self):
+ targets = [
+ 'mc:qemuamd64-bullseye:isar-image-base',
+ 'mc:qemuarm-bullseye:isar-image-base',
+ 'mc:qemui386-bullseye:isar-image-base',
+ 'mc:qemumipsel-bullseye:isar-image-base'
+ ]
+
+ try:
+ self.perform_build_test(targets, 0, None)
+ except:
+ self.cancel('KFAIL')
+
+class RebuildTest(CIBaseTest):
+
+ """
+ Test image rebuild
+
+ :avocado: tags=rebuild,fast,full
+ """
+ def test_rebuild(self):
+ is_cross_build = int(self.params.get('cross', default=0))
+
+ layerdir_core = self.getlayerdir('core')
+
+ dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass'
+
+ self.backupfile(dpkgbase_file)
+ with open(dpkgbase_file, 'a') as file:
+ file.write('do_fetch_append() {\n\n}')
+
+ try:
+ self.perform_build_test('mc:qemuamd64-stretch:isar-image-base',
+ is_cross_build, None)
+ finally:
+ self.restorefile(dpkgbase_file)
+
+class WicTest(CIBaseTest):
+
+ """
+ Test wic --exclude-path
+
+ :avocado: tags=wic,fast,full
+ """
+ def test_wic_exclude(self):
+ # TODO: remove hardcoded filenames
+ wks_path = '/scripts/lib/wic/canned-wks/sdimage-efi.wks'
+ wic_path = '/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img'
+
+ self.perform_wic_test('mc:qemuamd64-stretch:isar-image-base',
+ wks_path, wic_path)
+
+class ContainerImageTest(CIBaseTest):
+
+ """
+ Test containerized images creation
+
+ :avocado: tags=containerbuild,fast,full,container
+ """
+ @skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
+ def test_nocross(self):
+ targets = [
+ 'mc:container-amd64-stretch:isar-image-base',
+ 'mc:container-amd64-buster:isar-image-base',
+ 'mc:container-amd64-bullseye:isar-image-base'
+ ]
+
+ self.perform_container_test(targets, None)
+
+class ContainerSdkTest(CIBaseTest):
+
+ """
+ Test SDK container image creation
+
+ :avocado: tags=containersdk,fast,full,container
+ """
+ @skipUnless(UMOCI_AVAILABLE and SKOPEO_AVAILABLE, 'umoci/skopeo not found')
+ def test_container_sdk(self):
+ targets = ['mc:container-amd64-stretch:isar-image-base']
+
+ self.perform_container_test(targets, 'do_populate_sdk')
diff --git a/testsuite/build_test/cibase.py b/testsuite/build_test/cibase.py
new file mode 100644
index 0000000..1a89f43
--- /dev/null
+++ b/testsuite/build_test/cibase.py
@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+
+import os
+import re
+import tempfile
+
+from cibuilder import CIBuilder
+from avocado.utils import process
+
+isar_root = os.path.dirname(__file__) + '/../..'
+
+class CIBaseTest(CIBuilder):
+
+ def prep(self, testname, targets, cross, debsrc_cache):
+ build_dir = self.params.get('build_dir', default=isar_root + '/build')
+ build_dir = os.path.realpath(build_dir)
+ quiet = int(self.params.get('quiet', default=0))
+ bitbake_args = '-v'
+
+ if quiet:
+ bitbake_args = ''
+
+ self.log.info('===================================================')
+ self.log.info('Running ' + testname + ' test for:')
+ self.log.info(targets)
+ self.log.info('Isar build folder is: ' + build_dir)
+ self.log.info('===================================================')
+
+ self.init(build_dir)
+ self.confprepare(build_dir, 1, cross, debsrc_cache)
+
+ return build_dir, bitbake_args;
+
+ def perform_build_test(self, targets, cross, bitbake_cmd):
+ build_dir, bb_args = self.prep('Isar build', targets, cross, 1)
+
+ self.log.info('Starting build...')
+
+ self.bitbake(build_dir, targets, bitbake_cmd, bb_args)
+
+ def perform_repro_test(self, targets, signed):
+ cross = int(self.params.get('cross', default=0))
+ build_dir, bb_args = self.prep('repro Isar build', targets, cross, 0)
+
+ gpg_pub_key = os.path.dirname(__file__) + '/../base-apt/test_pub.key'
+ gpg_priv_key = os.path.dirname(__file__) + '/../base-apt/test_priv.key'
+
+ if signed:
+ with open(build_dir + '/conf/ci_build.conf', 'a') as file:
+ # Enable use of signed cached base repository
+ file.write('BASE_REPO_KEY="file://' + gpg_pub_key + '"\n')
+
+ os.chdir(build_dir)
+
+ os.environ['GNUPGHOME'] = tempfile.mkdtemp()
+ result = process.run('gpg --import %s %s' % (gpg_pub_key, gpg_priv_key))
+
+ if result.exit_status:
+ self.fail('GPG import failed')
+
+ self.bitbake(build_dir, targets, None, bb_args)
+
+ self.deletetmp(build_dir)
+ with open(build_dir + '/conf/ci_build.conf', 'a') as file:
+ file.write('ISAR_USE_CACHED_BASE_REPO = "1"\n')
+ file.write('BB_NO_NETWORK = "1"\n')
+
+ self.bitbake(build_dir, targets, None, bb_args)
+
+ # Cleanup and disable use of signed cached base repository
+ self.deletetmp(build_dir)
+ self.confcleanup(build_dir)
+
+ def perform_wic_test(self, targets, wks_path, wic_path):
+ cross = int(self.params.get('cross', default=0))
+ build_dir, bb_args = self.prep('WIC exclude build', targets, cross, 1)
+
+ layerdir_isar = self.getlayerdir('isar')
+
+ wks_file = layerdir_isar + wks_path
+ wic_img = build_dir + wic_path
+
+ if not os.path.isfile(wic_img):
+ self.fail('No build started before: ' + wic_img + ' not exist')
+
+ self.backupfile(wks_file)
+ self.backupmove(wic_img)
+
+ with open(wks_file, 'r') as file:
+ lines = file.readlines()
+ with open(wks_file, 'w') as file:
+ for line in lines:
+ file.write(re.sub(r'part \/ ', 'part \/ --exclude-path usr ',
+ line))
+
+ try:
+ self.bitbake(build_dir, targets, None, bb_args)
+ finally:
+ self.restorefile(wks_file)
+
+ self.restorefile(wic_img)
+
+ def perform_container_test(self, targets, bitbake_cmd):
+ cross = int(self.params.get('cross', default=0))
+ build_dir, bb_args = self.prep('Isar Container', targets, cross, 1)
+
+ self.containerprep(build_dir)
+
+ self.bitbake(build_dir, targets, bitbake_cmd, bb_args)
+
diff --git a/testsuite/build_test/cibuilder.py b/testsuite/build_test/cibuilder.py
new file mode 100644
index 0000000..187363f
--- /dev/null
+++ b/testsuite/build_test/cibuilder.py
@@ -0,0 +1,130 @@
+#!/usr/bin/env python3
+
+import logging
+import os
+import re
+import select
+import shutil
+import subprocess
+
+from avocado import Test
+from avocado.utils import path
+from avocado.utils import process
+
+isar_root = os.path.dirname(__file__) + '/../..'
+backup_prefix = '.ci-backup'
+
+app_log = logging.getLogger("avocado.app")
+
+class CIBuilder(Test):
+ def setUp(self):
+ super(CIBuilder, self).setUp()
+ job_log = os.path.join(os.path.dirname(self.logdir), '..', 'job.log')
+ self._file_handler = logging.FileHandler(filename=job_log)
+ self._file_handler.setLevel(logging.ERROR)
+ fmt = ('%(asctime)s %(module)-16.16s L%(lineno)-.4d %('
+ 'levelname)-5.5s| %(message)s')
+ formatter = logging.Formatter(fmt=fmt)
+ self._file_handler.setFormatter(formatter)
+ app_log.addHandler(self._file_handler)
+
+ def init(self, build_dir):
+ os.chdir(isar_root)
+ path.usable_rw_dir(build_dir)
+ output = process.getoutput('/bin/bash -c "source isar-init-build-env \
+ %s 2>&1 >/dev/null; env"' % build_dir)
+ env = dict(((x.split('=', 1) + [''])[:2] \
+ for x in output.splitlines() if x != ''))
+ os.environ.update(env)
+
+ def confprepare(self, build_dir, compat_arch, cross, debsrc_cache):
+ with open(build_dir + '/conf/ci_build.conf', 'w') as f:
+ if compat_arch:
+ f.write('ISAR_ENABLE_COMPAT_ARCH_amd64 = "1"\n')
+ f.write('ISAR_ENABLE_COMPAT_ARCH_arm64 = "1"\n')
+ f.write('ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0"\n')
+ f.write('IMAGE_INSTALL += "kselftest"\n')
+ if cross:
+ f.write('ISAR_CROSS_COMPILE = "1"\n')
+ if debsrc_cache:
+ f.write('BASE_REPO_FEATURES = "cache-deb-src"\n')
+
+ with open(build_dir + '/conf/local.conf', 'r+') as f:
+ for line in f:
+ if 'include ci_build.conf' in line:
+ break
+ else:
+ f.write('\ninclude ci_build.conf')
+
+ def containerprep(self, build_dir):
+ with open(build_dir + '/conf/ci_build.conf', 'a') as f:
+ f.write('SDK_FORMATS = "docker-archive"\n')
+ f.write('IMAGE_INSTALL_remove = "example-module-${KERNEL_NAME} enable-fsck"\n')
+
+ def confcleanup(self, build_dir):
+ open(build_dir + '/conf/ci_build.conf', 'w').close()
+
+ def deletetmp(self, build_dir):
+ process.run('rm -rf ' + build_dir + '/tmp', sudo=True)
+
+ def bitbake(self, build_dir, target, cmd, args):
+ os.chdir(build_dir)
+ cmdline = ['bitbake']
+ if args:
+ cmdline.append(args)
+ if cmd:
+ cmdline.append('-c')
+ cmdline.append(cmd)
+ if isinstance(target, list):
+ cmdline.extend(target)
+ else:
+ cmdline.append(target)
+
+ with subprocess.Popen(" ".join(cmdline), stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE, universal_newlines=True,
+ shell=True) as p1:
+ poller = select.poll()
+ poller.register(p1.stdout, select.POLLIN)
+ poller.register(p1.stderr, select.POLLIN)
+ while p1.poll() is None:
+ events = poller.poll(1000)
+ for fd, event in events:
+ if fd == p1.stdout.fileno():
+ self.log.info(p1.stdout.readline().rstrip())
+ if fd == p1.stderr.fileno():
+ app_log.error(p1.stderr.readline().rstrip())
+ p1.wait()
+ if p1.returncode:
+ self.fail('Bitbake failed')
+
+ def backupfile(self, path):
+ try:
+ shutil.copy2(path, path + backup_prefix)
+ except FileNotFoundError:
+ self.log.warn(path + ' not exist')
+
+ def backupmove(self, path):
+ try:
+ shutil.move(path, path + backup_prefix)
+ except FileNotFoundError:
+ self.log.warn(path + ' not exist')
+
+ def restorefile(self, path):
+ try:
+ shutil.move(path + backup_prefix, path)
+ except FileNotFoundError:
+ self.log.warn(path + backup_prefix + ' not exist')
+
+ def getlayerdir(self, layer):
+ try:
+ path.find_command('bitbake')
+ except path.CmdNotFoundError:
+ build_dir = self.params.get('build_dir',
+ default=isar_root + '/build')
+ self.init(build_dir)
+ output = process.getoutput('bitbake -e | grep "^LAYERDIR_.*="')
+ env = dict(((x.split('=', 1) + [''])[:2] \
+ for x in output.splitlines() if x != ''))
+
+ return env['LAYERDIR_' + layer].strip('"')
+
diff --git a/testsuite/build_test/run.sh b/testsuite/build_test/run.sh
deleted file mode 100755
index a8ea9cc..0000000
--- a/testsuite/build_test/run.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-
-avocado run build_test.py --mux-yaml test:variant.yaml --mux-inject build_dir:$BUILDDIR
diff --git a/testsuite/build_test/run_fast.sh b/testsuite/build_test/run_fast.sh
new file mode 100755
index 0000000..4d1bf44
--- /dev/null
+++ b/testsuite/build_test/run_fast.sh
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+avocado run build_test.py -t fast -p quiet=1 -p cross=1
diff --git a/testsuite/build_test/run_full.sh b/testsuite/build_test/run_full.sh
new file mode 100755
index 0000000..af5ec59
--- /dev/null
+++ b/testsuite/build_test/run_full.sh
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+avocado run build_test.py -t full -p quiet=1
diff --git a/testsuite/build_test/variant.yaml b/testsuite/build_test/variant.yaml
deleted file mode 100644
index 9ddc634..0000000
--- a/testsuite/build_test/variant.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-variants: !mux
- stretch-amd64:
- distro: "stretch"
- arch: "amd64"
- stretch-i386:
- distro: "stretch"
- arch: "i386"
- stretch-arm:
- distro: "stretch"
- arch: "arm"
- stretch-arm64:
- distro: "stretch"
- arch: "arm64"
- buster-amd64:
- distro: "buster"
- arch: "amd64"
- buster-i386:
- distro: "buster"
- arch: "i386"
- buster-arm:
- distro: "buster"
- arch: "arm"
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 09/17] testsuite: Add Python generations for testsuite in gitignore
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (7 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 08/17] build_test: Refactoring build tests cases Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 10/17] testsuite: Fix test suite prepare guide Anton Mikanovich
` (9 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Ignore Python byte code auto generated files inside testsuite.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/.gitignore | 1 +
1 file changed, 1 insertion(+)
create mode 100644 testsuite/.gitignore
diff --git a/testsuite/.gitignore b/testsuite/.gitignore
new file mode 100644
index 0000000..0d20b64
--- /dev/null
+++ b/testsuite/.gitignore
@@ -0,0 +1 @@
+*.pyc
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 10/17] testsuite: Fix test suite prepare guide
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (8 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 09/17] testsuite: Add Python generations for testsuite in gitignore Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 11/17] gitlab-ci: Add Avocado build artifacts Anton Mikanovich
` (8 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Remove deprecated yaml-to-mux plugin and yml variants from the doc.
Also remove subprocess32 backport installation which is not used.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/README.md | 21 +++------------------
1 file changed, 3 insertions(+), 18 deletions(-)
diff --git a/testsuite/README.md b/testsuite/README.md
index 5e64223..f91ea29 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -4,29 +4,14 @@ The framework could be installed by using standard HOWTO:
https://github.com/avocado-framework/avocado#installing-with-standard-python-tools
-Then you need to install varianter yaml-to-mux plugin by following these instructions:
-
- https://github.com/avocado-framework/avocado/tree/master/optional_plugins
-
-## For Debian 9.x
-
- $ sudo apt-get install python-pip
- $ pip install --user subprocess32
- $ pip install --user avocado-framework
- $ pip install --user avocado-framework-plugin-varianter-yaml-to-mux
-
-# Pre
-
- $ export PATH=$PATH:~/.local/bin
- $ cd isar
- $ source isar-init-build-env
+## For Debian (tested on Debian 10.x)
+ $ sudo dpkg -i avocado_85.0_all.deb
# Run test
Each testsuite directory contains:
- - run.sh - script to start tests
- - variants.yaml - set of input data
+ - run_*.sh - script to start tests
- *.py - test case
# Other
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 11/17] gitlab-ci: Add Avocado build artifacts
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (9 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 10/17] testsuite: Fix test suite prepare guide Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 12/17] gitlab-ci: Add debug flag Anton Mikanovich
` (7 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Save build logs in case of Avocado CI build.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
.gitlab-ci.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 7fc1612..56e98dc 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -14,6 +14,7 @@ variables:
name: "logs-$CI_JOB_ID"
paths:
- build/tmp/work/*/*/*/temp
+ - build/job-results
when: on_failure
expire_in: 1 week
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 12/17] gitlab-ci: Add debug flag
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (10 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 11/17] gitlab-ci: Add Avocado build artifacts Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 13/17] vm_boot_test: Add automatic bitbake init Anton Mikanovich
` (6 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Gitlab users expect full build log output witch was enabled by default.
Debug flag was ignored and never used, now it becomes functional.
To keed compatibility add '-d' flag to ci_build.sh call.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
.gitlab-ci.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 56e98dc..ac5e0c4 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -23,7 +23,7 @@ fast-ci:
except:
- schedules
script:
- - scripts/ci_build.sh -q -f
+ - scripts/ci_build.sh -q -f -d
full-ci:
<<: *common-build
@@ -33,7 +33,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -c -r;
+ scripts/ci_build.sh -q -c -r -d;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 13/17] vm_boot_test: Add automatic bitbake init
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (11 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 12/17] gitlab-ci: Add debug flag Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 14/17] build_test: Protect ubuntu target with KFAIL Anton Mikanovich
` (5 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Implement source of isar-init-build-env in case there is no bitbake
command found.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/vm_boot_test/vm_boot_test.py | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py
index 6e580d0..01623c5 100644
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test/vm_boot_test.py
@@ -13,12 +13,25 @@ sys.path.append(dirname(__file__) + '/..')
import start_vm
from avocado import Test
+from avocado.utils import process
+from avocado.utils import path
class CanBeFinished(Exception):
pass
class VmBase(Test):
+ def check_bitbake(self, build_dir):
+ try:
+ path.find_command('bitbake')
+ except path.CmdNotFoundError:
+ out = process.getoutput('/bin/bash -c "cd ../.. && \
+ source isar-init-build-env \
+ %s 2>&1 >/dev/null; env"' % build_dir)
+ env = dict(((x.split('=', 1) + [''])[:2] \
+ for x in output.splitlines() if x != ''))
+ os.environ.update(env)
+
def vm_start(self, arch='amd64', distro='buster'):
build_dir = self.params.get('build_dir', default='.')
time_to_wait = self.params.get('time_to_wait', default=60)
@@ -28,6 +41,8 @@ class VmBase(Test):
self.log.info('Isar build folder is: ' + build_dir)
self.log.info('===================================================')
+ self.check_bitbake(build_dir)
+
fd, output_file = tempfile.mkstemp(suffix='_log.txt',
prefix='vm_start_' + distro + '_' +
arch + '_', dir=build_dir, text=True)
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 14/17] build_test: Protect ubuntu target with KFAIL
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (12 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 13/17] vm_boot_test: Add automatic bitbake init Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 15/17] ci_build: Migrate to Avocado Anton Mikanovich
` (4 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
We experience the same QEMU-related semop issue on qemuarm64-focal:
| ERROR: mc:qemuarm64-focal:kselftest-5.6.13-r0 do_install_builddeps: Execution of '/build/tmp/work/ubuntu-focal-arm64/kselftest/5.6.13-r0/temp/run.do_install_builddeps.33809' failed with exit code 1:
| /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
| hostname: Temporary failure in name resolution
| semop(1): encountered an error: Function not implemented
| Error in the build process: exit status 1
| dpkg: error: cannot access archive 'kselftest-build-deps_5.6.13_arm64.deb': No such file or directory
So we need to protect with KFAIL not only bullseye targets, but Ubuntu
Focal targets also.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/build_test/build_test.py | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py
index 9038ec5..63ea301 100644
--- a/testsuite/build_test/build_test.py
+++ b/testsuite/build_test/build_test.py
@@ -55,12 +55,21 @@ class CrossTest(CIBaseTest):
'mc:qemuamd64-stretch:isar-image-base',
'mc:de0-nano-soc-buster:isar-image-base',
'mc:stm32mp15x-buster:isar-image-base',
- 'mc:rpi-stretch:isar-image-base',
- 'mc:qemuarm64-focal:isar-image-base'
+ 'mc:rpi-stretch:isar-image-base'
]
self.perform_build_test(targets, 1, None)
+ def test_cross_ubuntu(self):
+ targets = [
+ 'mc:qemuarm64-focal:isar-image-base'
+ ]
+
+ try:
+ self.perform_build_test(targets, 1, None)
+ except:
+ self.cancel('KFAIL')
+
def test_cross_bullseye(self):
targets = [
'mc:qemuarm-bullseye:isar-image-base'
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 15/17] ci_build: Migrate to Avocado
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (13 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 14/17] build_test: Protect ubuntu target with KFAIL Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 16/17] vm_smoke_test: " Anton Mikanovich
` (3 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Call Avocado test cases instead of shell based.
Build artifacts are placed in build directory.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 198 ++++-----------------------
scripts/ci_build_legacy.sh | 267 +++++++++++++++++++++++++++++++++++++
2 files changed, 293 insertions(+), 172 deletions(-)
create mode 100755 scripts/ci_build_legacy.sh
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 49d6aa8..ffe50c6 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -14,6 +14,9 @@ export PATH=$PATH:/sbin
# Go to Isar root
cd "$(dirname "$0")/.."
+# Get Avocado build tests path
+BUILD_TEST_DIR="$(pwd)/testsuite/build_test"
+
# Start build in Isar tree by default
BUILD_DIR=./build
@@ -21,68 +24,10 @@ BUILD_DIR=./build
DEPENDENCIES="umoci skopeo"
for prog in ${DEPENDENCIES} ; do
if [ ! -x "$(which $prog)" ] ; then
- echo "missing $prog in PATH, exiting" >&2
- exit 1
+ echo "missing $prog in PATH" >&2
fi
done
-BB_ARGS="-v"
-
-TARGETS_SET="\
- mc:qemuarm-stretch:isar-image-base \
- mc:qemuarm-buster:isar-image-base \
- mc:qemuarm64-stretch:isar-image-base \
- mc:qemui386-stretch:isar-image-base \
- mc:qemui386-buster:isar-image-base \
- mc:qemuamd64-stretch:isar-image-base \
- mc:qemuamd64-buster:isar-image-base \
- mc:qemuamd64-buster-tgz:isar-image-base \
- mc:qemuamd64-buster:isar-initramfs \
- mc:qemumipsel-stretch:isar-image-base \
- mc:qemumipsel-buster:isar-image-base \
- mc:nand-ubi-demo-buster:isar-image-ubi \
- mc:rpi-stretch:isar-image-base \
- mc:qemuamd64-focal:isar-image-base \
- "
- # qemu-user-static of <= buster too old to build that
- # mc:qemuarm64-buster:isar-image-base
- # mc:qemuarm64-bullseye:isar-image-base
-
-TARGETS_SET_BULLSEYE="\
- mc:qemuamd64-bullseye:isar-image-base \
- mc:qemuarm-bullseye:isar-image-base \
- mc:qemui386-bullseye:isar-image-base \
- mc:qemumipsel-bullseye:isar-image-base \
-"
-
-TARGETS_CONTAINERS="\
- mc:container-amd64-stretch:isar-image-base \
- mc:container-amd64-buster:isar-image-base \
- mc:container-amd64-bullseye:isar-image-base \
-"
-
-CROSS_TARGETS_SET="\
- mc:qemuarm-stretch:isar-image-base \
- mc:qemuarm-buster:isar-image-base \
- mc:qemuarm64-stretch:isar-image-base \
- mc:qemuamd64-stretch:isar-image-base \
- mc:de0-nano-soc-buster:isar-image-base \
- mc:stm32mp15x-buster:isar-image-base \
- mc:rpi-stretch:isar-image-base \
- mc:qemuarm64-focal:isar-image-base"
-
-CROSS_TARGETS_SET_BULLSEYE="\
- mc:qemuarm-bullseye:isar-image-base \
-"
-
-REPRO_TARGETS_SET_SIGNED="\
- mc:de0-nano-soc-buster:isar-image-base \
- mc:qemuarm64-stretch:isar-image-base"
-
-REPRO_TARGETS_SET="\
- mc:qemuamd64-stretch:isar-image-base \
- mc:qemuarm-buster:isar-image-base"
-
show_help() {
echo "This script builds the default Isar images."
echo
@@ -104,6 +49,10 @@ show_help() {
echo " 3 if invalid parameters are passed."
}
+TAGS="full"
+CROSS_BUILD="0"
+QUIET="0"
+
# Parse command line to get user configuration
while [ $# -gt 0 ]
do
@@ -122,16 +71,16 @@ do
CROSS_BUILD="1"
;;
-d|--debug)
- BB_ARGS="$BB_ARGS -D"
+ VERBOSE="--show=app,test"
;;
-f|--fast)
# Start build for the reduced set of configurations
# Enforce cross-compilation to speed up the build
- FAST_BUILD="1"
+ TAGS="fast"
CROSS_BUILD="1"
;;
-q|--quiet)
- BB_ARGS=""
+ QUIET="1"
;;
-r|--repro)
REPRO_BUILD="1"
@@ -150,118 +99,23 @@ do
shift
done
+if [ -z "$REPRO_BUILD" ]; then
+ TAGS="$TAGS,-repro"
+fi
+
# the real stuff starts here, trace commands from now on
set -x
-# Setup build folder for the current build
-if [ ! -d "$BUILD_DIR" ]; then
- mkdir -p "$BUILD_DIR"
-fi
-source isar-init-build-env "$BUILD_DIR"
-
-cat >>conf/local.conf <<EOF
-ISAR_ENABLE_COMPAT_ARCH_amd64 = "1"
-ISAR_ENABLE_COMPAT_ARCH_arm64 = "1"
-ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0"
-IMAGE_INSTALL += "kselftest"
+# Provide working path
+mkdir -p .config/avocado
+cat <<EOF > .config/avocado/avocado.conf
+[datadir.paths]
+base_dir = $BUILD_DIR/
+test_dir = $BUILD_DIR/tests
+data_dir = $BUILD_DIR/data
+logs_dir = $BUILD_DIR/job-results
EOF
+export VIRTUAL_ENV="./"
-if [ -n "$CROSS_BUILD" ]; then
- sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf
-fi
-
-if [ -n "$REPRO_BUILD" ]; then
- ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key"
- ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key"
- export GNUPGHOME=$(mktemp -d)
- gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
-
- # Enable use of signed cached base repository
- echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf
- bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
- while [ -e bitbake.sock ]; do sleep 1; done
- sudo rm -rf tmp
- sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
- sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf
- bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
- while [ -e bitbake.sock ]; do sleep 1; done
- # Cleanup and disable use of signed cached base repository
- sudo rm -rf tmp
- sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
- sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf
- sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
-
- # Enable use of unsigned cached base repository
- bitbake $BB_ARGS $REPRO_TARGETS_SET
- while [ -e bitbake.sock ]; do sleep 1; done
- sudo rm -rf tmp
- sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
- sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf
- bitbake $BB_ARGS $REPRO_TARGETS_SET
- while [ -e bitbake.sock ]; do sleep 1; done
- # Cleanup and disable use of unsigned cached base repository
- sudo rm -rf tmp
- sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
- sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf
-fi
-
-sed -i -e 's/^#BASE_REPO_FEATURES ?= "cache-deb-src"/BASE_REPO_FEATURES ?= "cache-deb-src"/g' conf/local.conf
-# Start cross build for the defined set of configurations
-sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf
-bitbake $BB_ARGS $CROSS_TARGETS_SET
-while [ -e bitbake.sock ]; do sleep 1; done
-if bitbake $BB_ARGS $CROSS_TARGETS_SET_BULLSEYE; then
- echo "bullseye cross: PASSED"
-else
- echo "bullseye cross: KFAIL"
-fi
-# In addition test SDK creation
-bitbake $BB_ARGS -c do_populate_sdk mc:qemuarm-stretch:isar-image-base
-while [ -e bitbake.sock ]; do sleep 1; done
-
-if [ -z "$FAST_BUILD" ]; then
- # Cleanup and disable cross build
- sudo rm -rf tmp
- sed -i -e 's/ISAR_CROSS_COMPILE ?= "1"/ISAR_CROSS_COMPILE ?= "0"/g' conf/local.conf
- bitbake $BB_ARGS $TARGETS_SET
- while [ -e bitbake.sock ]; do sleep 1; done
-
- if bitbake $BB_ARGS $TARGETS_SET_BULLSEYE; then
- echo "bullseye: PASSED"
- else
- echo "bullseye: KFAIL"
- fi
- while [ -e bitbake.sock ]; do sleep 1; done
-fi
-
-eval $(bitbake -e | grep "^LAYERDIR_core=")
-eval $(bitbake -e | grep "^LAYERDIR_isar=")
-cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup"
-echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/classes/dpkg-base.bbclass"
-
-bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
-
-mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/classes/dpkg-base.bbclass"
-
-# Test wic --exclude-path
-cp -a "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"
-mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \
- ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup
-sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
-
-bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
-
-mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
-mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \
- ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img
-
-# Finalize with containerized images, since they remove some not-needed packages from the local.conf
-sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf
-sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf
-bitbake $BB_ARGS $TARGETS_CONTAINERS
-while [ -e bitbake.sock ]; do sleep 1; done
-# and SDK container image creation
-echo 'SDK_FORMATS = "docker-archive"' >> conf/local.conf
-bitbake $BB_ARGS -c do_populate_sdk mc:container-amd64-stretch:isar-image-base
-while [ -e bitbake.sock ]; do sleep 1; done
-
+avocado $VERBOSE run "$BUILD_TEST_DIR/build_test.py" -t $TAGS --disable-sysinfo \
+ -p build_dir="$BUILD_DIR" -p quiet=$QUIET -p cross=$CROSS_BUILD
diff --git a/scripts/ci_build_legacy.sh b/scripts/ci_build_legacy.sh
new file mode 100755
index 0000000..49d6aa8
--- /dev/null
+++ b/scripts/ci_build_legacy.sh
@@ -0,0 +1,267 @@
+#!/usr/bin/env bash
+# Script for CI system build
+#
+# Alexander Smirnov <asmirnov@ilbers.de>
+# Copyright (c) 2016-2018 ilbers GmbH
+
+set -e
+
+ES_BUG=3
+
+# Export $PATH to use 'parted' tool
+export PATH=$PATH:/sbin
+
+# Go to Isar root
+cd "$(dirname "$0")/.."
+
+# Start build in Isar tree by default
+BUILD_DIR=./build
+
+# Check dependencies
+DEPENDENCIES="umoci skopeo"
+for prog in ${DEPENDENCIES} ; do
+ if [ ! -x "$(which $prog)" ] ; then
+ echo "missing $prog in PATH, exiting" >&2
+ exit 1
+ fi
+done
+
+BB_ARGS="-v"
+
+TARGETS_SET="\
+ mc:qemuarm-stretch:isar-image-base \
+ mc:qemuarm-buster:isar-image-base \
+ mc:qemuarm64-stretch:isar-image-base \
+ mc:qemui386-stretch:isar-image-base \
+ mc:qemui386-buster:isar-image-base \
+ mc:qemuamd64-stretch:isar-image-base \
+ mc:qemuamd64-buster:isar-image-base \
+ mc:qemuamd64-buster-tgz:isar-image-base \
+ mc:qemuamd64-buster:isar-initramfs \
+ mc:qemumipsel-stretch:isar-image-base \
+ mc:qemumipsel-buster:isar-image-base \
+ mc:nand-ubi-demo-buster:isar-image-ubi \
+ mc:rpi-stretch:isar-image-base \
+ mc:qemuamd64-focal:isar-image-base \
+ "
+ # qemu-user-static of <= buster too old to build that
+ # mc:qemuarm64-buster:isar-image-base
+ # mc:qemuarm64-bullseye:isar-image-base
+
+TARGETS_SET_BULLSEYE="\
+ mc:qemuamd64-bullseye:isar-image-base \
+ mc:qemuarm-bullseye:isar-image-base \
+ mc:qemui386-bullseye:isar-image-base \
+ mc:qemumipsel-bullseye:isar-image-base \
+"
+
+TARGETS_CONTAINERS="\
+ mc:container-amd64-stretch:isar-image-base \
+ mc:container-amd64-buster:isar-image-base \
+ mc:container-amd64-bullseye:isar-image-base \
+"
+
+CROSS_TARGETS_SET="\
+ mc:qemuarm-stretch:isar-image-base \
+ mc:qemuarm-buster:isar-image-base \
+ mc:qemuarm64-stretch:isar-image-base \
+ mc:qemuamd64-stretch:isar-image-base \
+ mc:de0-nano-soc-buster:isar-image-base \
+ mc:stm32mp15x-buster:isar-image-base \
+ mc:rpi-stretch:isar-image-base \
+ mc:qemuarm64-focal:isar-image-base"
+
+CROSS_TARGETS_SET_BULLSEYE="\
+ mc:qemuarm-bullseye:isar-image-base \
+"
+
+REPRO_TARGETS_SET_SIGNED="\
+ mc:de0-nano-soc-buster:isar-image-base \
+ mc:qemuarm64-stretch:isar-image-base"
+
+REPRO_TARGETS_SET="\
+ mc:qemuamd64-stretch:isar-image-base \
+ mc:qemuarm-buster:isar-image-base"
+
+show_help() {
+ echo "This script builds the default Isar images."
+ echo
+ echo "Usage:"
+ echo " $0 [params]"
+ echo
+ echo "Parameters:"
+ echo " -b, --build BUILD_DIR set path to build directory. If not set,"
+ echo " the build will be started in current path."
+ echo " -c, --cross enable cross-compilation."
+ echo " -d, --debug enable debug bitbake output."
+ echo " -f, --fast cross build reduced set of configurations."
+ echo " -q, --quiet suppress verbose bitbake output."
+ echo " -r, --repro enable use of cached base repository."
+ echo " --help display this message and exit."
+ echo
+ echo "Exit status:"
+ echo " 0 if OK,"
+ echo " 3 if invalid parameters are passed."
+}
+
+# Parse command line to get user configuration
+while [ $# -gt 0 ]
+do
+ key="$1"
+
+ case $key in
+ -h|--help)
+ show_help
+ exit 0
+ ;;
+ -b|--build)
+ BUILD_DIR="$2"
+ shift
+ ;;
+ -c|--cross)
+ CROSS_BUILD="1"
+ ;;
+ -d|--debug)
+ BB_ARGS="$BB_ARGS -D"
+ ;;
+ -f|--fast)
+ # Start build for the reduced set of configurations
+ # Enforce cross-compilation to speed up the build
+ FAST_BUILD="1"
+ CROSS_BUILD="1"
+ ;;
+ -q|--quiet)
+ BB_ARGS=""
+ ;;
+ -r|--repro)
+ REPRO_BUILD="1"
+ # This switch is deprecated, just here to not cause failing CI on
+ # legacy configs
+ case "$2" in
+ -s|--sign) shift ;;
+ esac
+ ;;
+ *)
+ echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
+ exit $ES_BUG
+ ;;
+ esac
+
+ shift
+done
+
+# the real stuff starts here, trace commands from now on
+set -x
+
+# Setup build folder for the current build
+if [ ! -d "$BUILD_DIR" ]; then
+ mkdir -p "$BUILD_DIR"
+fi
+source isar-init-build-env "$BUILD_DIR"
+
+cat >>conf/local.conf <<EOF
+ISAR_ENABLE_COMPAT_ARCH_amd64 = "1"
+ISAR_ENABLE_COMPAT_ARCH_arm64 = "1"
+ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0"
+IMAGE_INSTALL += "kselftest"
+EOF
+
+if [ -n "$CROSS_BUILD" ]; then
+ sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf
+fi
+
+if [ -n "$REPRO_BUILD" ]; then
+ ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key"
+ ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key"
+ export GNUPGHOME=$(mktemp -d)
+ gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
+
+ # Enable use of signed cached base repository
+ echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf
+ bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
+ while [ -e bitbake.sock ]; do sleep 1; done
+ sudo rm -rf tmp
+ sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+ sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf
+ bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
+ while [ -e bitbake.sock ]; do sleep 1; done
+ # Cleanup and disable use of signed cached base repository
+ sudo rm -rf tmp
+ sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+ sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf
+ sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
+
+ # Enable use of unsigned cached base repository
+ bitbake $BB_ARGS $REPRO_TARGETS_SET
+ while [ -e bitbake.sock ]; do sleep 1; done
+ sudo rm -rf tmp
+ sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+ sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf
+ bitbake $BB_ARGS $REPRO_TARGETS_SET
+ while [ -e bitbake.sock ]; do sleep 1; done
+ # Cleanup and disable use of unsigned cached base repository
+ sudo rm -rf tmp
+ sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+ sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf
+fi
+
+sed -i -e 's/^#BASE_REPO_FEATURES ?= "cache-deb-src"/BASE_REPO_FEATURES ?= "cache-deb-src"/g' conf/local.conf
+# Start cross build for the defined set of configurations
+sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf
+bitbake $BB_ARGS $CROSS_TARGETS_SET
+while [ -e bitbake.sock ]; do sleep 1; done
+if bitbake $BB_ARGS $CROSS_TARGETS_SET_BULLSEYE; then
+ echo "bullseye cross: PASSED"
+else
+ echo "bullseye cross: KFAIL"
+fi
+# In addition test SDK creation
+bitbake $BB_ARGS -c do_populate_sdk mc:qemuarm-stretch:isar-image-base
+while [ -e bitbake.sock ]; do sleep 1; done
+
+if [ -z "$FAST_BUILD" ]; then
+ # Cleanup and disable cross build
+ sudo rm -rf tmp
+ sed -i -e 's/ISAR_CROSS_COMPILE ?= "1"/ISAR_CROSS_COMPILE ?= "0"/g' conf/local.conf
+ bitbake $BB_ARGS $TARGETS_SET
+ while [ -e bitbake.sock ]; do sleep 1; done
+
+ if bitbake $BB_ARGS $TARGETS_SET_BULLSEYE; then
+ echo "bullseye: PASSED"
+ else
+ echo "bullseye: KFAIL"
+ fi
+ while [ -e bitbake.sock ]; do sleep 1; done
+fi
+
+eval $(bitbake -e | grep "^LAYERDIR_core=")
+eval $(bitbake -e | grep "^LAYERDIR_isar=")
+cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup"
+echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/classes/dpkg-base.bbclass"
+
+bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
+
+mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/classes/dpkg-base.bbclass"
+
+# Test wic --exclude-path
+cp -a "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup"
+mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \
+ ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup
+sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
+
+bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
+
+mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
+mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \
+ ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img
+
+# Finalize with containerized images, since they remove some not-needed packages from the local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf
+bitbake $BB_ARGS $TARGETS_CONTAINERS
+while [ -e bitbake.sock ]; do sleep 1; done
+# and SDK container image creation
+echo 'SDK_FORMATS = "docker-archive"' >> conf/local.conf
+bitbake $BB_ARGS -c do_populate_sdk mc:container-amd64-stretch:isar-image-base
+while [ -e bitbake.sock ]; do sleep 1; done
+
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 16/17] vm_smoke_test: Migrate to Avocado
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (14 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 15/17] ci_build: Migrate to Avocado Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-07-30 8:27 ` [PATCH v10 17/17] ci_build: Install Avocado if needed Anton Mikanovich
` (2 subsequent siblings)
18 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Call Avocado test cases instead of shell based.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/vm_smoke_test | 110 ++++-------------------
scripts/vm_smoke_test_legacy | 168 +++++++++++++++++++++++++++++++++++
2 files changed, 185 insertions(+), 93 deletions(-)
create mode 100755 scripts/vm_smoke_test_legacy
diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
index ce08213..647b417 100755
--- a/scripts/vm_smoke_test
+++ b/scripts/vm_smoke_test
@@ -3,9 +3,7 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2018 ilbers GmbH
-CONSOLE_OUTPUT=/tmp/isar_console
-PID_FILE=/tmp/qemu.pid
-VERBOSE=1
+VERBOSE="--show=test"
TIMEOUT=300
# Error codes:
@@ -13,75 +11,12 @@ ES_OK=0
ES_FAIL=1
ES_BUG=3
-RET=$ES_OK
+RET=$ES_FAIL
-dump_boot_log() {
- echo "Boot log:\n8<--"
- cat $CONSOLE_OUTPUT
- echo "\n8<--"
-}
-
-check_login_prompt() {
- echo -n "Check login prompt: "
-
- str=$(grep "isar login: " $CONSOLE_OUTPUT)
-
- if [ -n "$str" ]; then
- echo "PASSED"
- else
- echo "FAIL"
- RET=$ES_FAIL
- FAIL=1
- fi
-}
-
-check_example_module() {
- echo -n "Check example module: "
-
- str=$(grep "Just an example" $CONSOLE_OUTPUT)
-
- if [ -n "$str" ]; then
- echo "PASSED"
- else
- echo "FAIL"
- RET=$ES_FAIL
- FAIL=1
- fi
-}
-
-run_test () {
- ARCH=$1
- DISTRO=$2
-
- echo "-------------------------------------------------"
- echo "Testing Isar [$DISTRO] image for [$ARCH] machine:"
+# Get Avocado QEMU tests path
+VM_TEST_DIR="$(dirname "$0")/../testsuite/vm_boot_test"
- start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE > /dev/null 2>&1 &
- sleep 5
-
- if [ -z `ps -p $! -o pid=` ]; then
- echo "QEMU start: FAILED"
- RET=$ES_FAIL
- echo "Command output:\n8<--"
- start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE
- echo "\n8<--"
- else
- sleep $TIMEOUT
- kill `cat $PID_FILE`
-
- FAIL=0
-
- check_login_prompt
-
- check_example_module
-
- [ $VERBOSE -eq 1 -o $FAIL -eq 1 ] && dump_boot_log
-
- rm $CONSOLE_OUTPUT
- fi
-
- rm $PID_FILE
-}
+BUILD_DIR=$PWD
show_help() {
echo "This script tests the Isar images for default targets in QEMU."
@@ -91,10 +26,6 @@ show_help() {
echo
echo "Parameters:"
echo " -f,--fast test reduced set of supported targets."
- echo " -o,--output FILE specify file to store console output."
- echo " The default is: /tmp/isar_console"
- echo " -p,--pid-file FILE specify file to store QEMU process PID."
- echo " The default is: /tmp/qemu.pid"
echo " -q, --quiet do not display boot logs for all the targets."
echo " If test failed for the specific configuration,"
echo " the respective boot log will be printed anyway."
@@ -119,18 +50,18 @@ do
exit 0
;;
-o|--output)
- CONSOLE_OUTPUT=$2
+ # Deprecated option
shift
;;
-p|--pid-file)
- PID_FILE=$2
+ # Deprecated option
shift
;;
-f|--fast)
FAST_BUILD="1"
;;
-q|--quiet)
- VERBOSE=0
+ VERBOSE=""
;;
-t|--timeout)
TIMEOUT=$2
@@ -145,24 +76,17 @@ do
shift
done
-# ARM machine
-run_test arm stretch
-run_test arm buster
-
-# AMD64 machine
-if [ -z "$FAST_BUILD" ]; then
- run_test amd64 buster
- run_test amd64 focal
+TAGS="full"
+if [ -n "$FAST_BUILD" ]; then
+ TAGS="fast"
fi
-run_test amd64 stretch
-# i386 machine
-if [ -z "$FAST_BUILD" ]; then
- run_test i386 stretch
- run_test i386 buster
-fi
+# Provide working path
+export VIRTUAL_ENV="./"
-# ARM64 machine
-run_test arm64 stretch
+if avocado $VERBOSE run "$VM_TEST_DIR/vm_boot_test.py" -t $TAGS \
+ -p build_dir="$BUILD_DIR" -p time_to_wait=$TIMEOUT; then
+ RET=$ES_OK
+fi
exit $RET
diff --git a/scripts/vm_smoke_test_legacy b/scripts/vm_smoke_test_legacy
new file mode 100755
index 0000000..ce08213
--- /dev/null
+++ b/scripts/vm_smoke_test_legacy
@@ -0,0 +1,168 @@
+#!/bin/sh
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2018 ilbers GmbH
+
+CONSOLE_OUTPUT=/tmp/isar_console
+PID_FILE=/tmp/qemu.pid
+VERBOSE=1
+TIMEOUT=300
+
+# Error codes:
+ES_OK=0
+ES_FAIL=1
+ES_BUG=3
+
+RET=$ES_OK
+
+dump_boot_log() {
+ echo "Boot log:\n8<--"
+ cat $CONSOLE_OUTPUT
+ echo "\n8<--"
+}
+
+check_login_prompt() {
+ echo -n "Check login prompt: "
+
+ str=$(grep "isar login: " $CONSOLE_OUTPUT)
+
+ if [ -n "$str" ]; then
+ echo "PASSED"
+ else
+ echo "FAIL"
+ RET=$ES_FAIL
+ FAIL=1
+ fi
+}
+
+check_example_module() {
+ echo -n "Check example module: "
+
+ str=$(grep "Just an example" $CONSOLE_OUTPUT)
+
+ if [ -n "$str" ]; then
+ echo "PASSED"
+ else
+ echo "FAIL"
+ RET=$ES_FAIL
+ FAIL=1
+ fi
+}
+
+run_test () {
+ ARCH=$1
+ DISTRO=$2
+
+ echo "-------------------------------------------------"
+ echo "Testing Isar [$DISTRO] image for [$ARCH] machine:"
+
+ start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE > /dev/null 2>&1 &
+ sleep 5
+
+ if [ -z `ps -p $! -o pid=` ]; then
+ echo "QEMU start: FAILED"
+ RET=$ES_FAIL
+ echo "Command output:\n8<--"
+ start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE
+ echo "\n8<--"
+ else
+ sleep $TIMEOUT
+ kill `cat $PID_FILE`
+
+ FAIL=0
+
+ check_login_prompt
+
+ check_example_module
+
+ [ $VERBOSE -eq 1 -o $FAIL -eq 1 ] && dump_boot_log
+
+ rm $CONSOLE_OUTPUT
+ fi
+
+ rm $PID_FILE
+}
+
+show_help() {
+ echo "This script tests the Isar images for default targets in QEMU."
+ echo
+ echo "Usage:"
+ echo " $0 [params]"
+ echo
+ echo "Parameters:"
+ echo " -f,--fast test reduced set of supported targets."
+ echo " -o,--output FILE specify file to store console output."
+ echo " The default is: /tmp/isar_console"
+ echo " -p,--pid-file FILE specify file to store QEMU process PID."
+ echo " The default is: /tmp/qemu.pid"
+ echo " -q, --quiet do not display boot logs for all the targets."
+ echo " If test failed for the specific configuration,"
+ echo " the respective boot log will be printed anyway."
+ echo " -t,--timeout SEC specify time in seconds to wait before stop QEMU."
+ echo " The default is: 300"
+ echo " -h, --help display this message and exit."
+ echo
+ echo "Exit status:"
+ echo " 0 if OK,"
+ echo " 1 if test failed,"
+ echo " 3 if invalid parameters are passed."
+}
+
+# Parse command line to get user configuration
+while [ $# -gt 0 ]
+do
+ key="$1"
+
+ case $key in
+ -h|--help)
+ show_help
+ exit 0
+ ;;
+ -o|--output)
+ CONSOLE_OUTPUT=$2
+ shift
+ ;;
+ -p|--pid-file)
+ PID_FILE=$2
+ shift
+ ;;
+ -f|--fast)
+ FAST_BUILD="1"
+ ;;
+ -q|--quiet)
+ VERBOSE=0
+ ;;
+ -t|--timeout)
+ TIMEOUT=$2
+ shift
+ ;;
+ *)
+ echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
+ exit $ES_BUG
+ ;;
+ esac
+
+ shift
+done
+
+# ARM machine
+run_test arm stretch
+run_test arm buster
+
+# AMD64 machine
+if [ -z "$FAST_BUILD" ]; then
+ run_test amd64 buster
+ run_test amd64 focal
+fi
+run_test amd64 stretch
+
+# i386 machine
+if [ -z "$FAST_BUILD" ]; then
+ run_test i386 stretch
+ run_test i386 buster
+fi
+
+# ARM64 machine
+run_test arm64 stretch
+
+exit $RET
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v10 17/17] ci_build: Install Avocado if needed
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (15 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 16/17] vm_smoke_test: " Anton Mikanovich
@ 2021-07-30 8:27 ` Anton Mikanovich
2021-08-12 15:58 ` Henning Schild
2021-08-12 16:13 ` [PATCH v10 00/17] Update Avocado testsuite Henning Schild
2021-08-12 16:19 ` Henning Schild
18 siblings, 1 reply; 22+ messages in thread
From: Anton Mikanovich @ 2021-07-30 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To keep compatibility will the build systems without avocado-framework
preinstalled add the code to install it from debian-isar repo.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
scripts/ci_build.sh | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ffe50c6..a6a4b95 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -28,6 +28,23 @@ for prog in ${DEPENDENCIES} ; do
fi
done
+# Install Avocado if needed
+if [ ! -x "$(which avocado)" ] ; then
+ set +e
+ APT_LIST="/etc/apt/sources.list.d"
+ ISAR_REPO="deb http://deb.isar-build.org/debian-isar buster-isar main"
+ ISAR_KEY="http://deb.isar-build.org/debian-isar.key"
+ sudo -s <<EOSUDO
+ mkdir -p $APT_LIST
+ echo "$ISAR_REPO" > $APT_LIST/isar.list
+ apt-key adv --fetch-keys $ISAR_KEY
+ apt update
+ yes | apt install avocado
+EOSUDO
+ echo "Installed Avocado-framework"
+ set -e
+fi
+
show_help() {
echo "This script builds the default Isar images."
echo
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v10 17/17] ci_build: Install Avocado if needed
2021-07-30 8:27 ` [PATCH v10 17/17] ci_build: Install Avocado if needed Anton Mikanovich
@ 2021-08-12 15:58 ` Henning Schild
2021-09-16 14:11 ` Baurzhan Ismagulov
0 siblings, 1 reply; 22+ messages in thread
From: Henning Schild @ 2021-08-12 15:58 UTC (permalink / raw)
To: Anton Mikanovich; +Cc: isar-users
Am Fri, 30 Jul 2021 11:27:39 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:
> To keep compatibility will the build systems without avocado-framework
> preinstalled add the code to install it from debian-isar repo.
>
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
> scripts/ci_build.sh | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
>
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index ffe50c6..a6a4b95 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -28,6 +28,23 @@ for prog in ${DEPENDENCIES} ; do
> fi
> done
>
> +# Install Avocado if needed
> +if [ ! -x "$(which avocado)" ] ; then
> + set +e
> + APT_LIST="/etc/apt/sources.list.d"
> + ISAR_REPO="deb
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdeb.isar-build.org%2Fdebian-isar&data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Caf72bf4e5ccc4f56a51408d95333f5de%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637632305320761070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4CxE70HE969aU649GEQhzxXm9YT4qQm%2FBcOo5r0uXqQ%3D&reserved=0
> buster-isar main"
> +
> ISAR_KEY="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdeb.isar-build.org%2Fdebian-isar.key&data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7Caf72bf4e5ccc4f56a51408d95333f5de%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637632305320761070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DDNiBFCFIkCmXAD%2BAhNe%2F2PIj9VeVrFOkEGvdZJq8GE%3D&reserved=0"
> + sudo -s <<EOSUDO
I do not like this. The package should be mainlined into debian and
until that day maybe be installed with pip inside a venv. And i want
more than "buster". This is what i came up with
+sudo apt-get update -qq
+sudo apt-get install -y virtualenv
+
+rm -rf /tmp/venv
+virtualenv --python python3 /tmp/venv
+source /tmp/venv/bin/activate
+pip install subprocess32 avocado-framework-plugin-varianter-yaml-to-mux
> + mkdir -p $APT_LIST
> + echo "$ISAR_REPO" > $APT_LIST/isar.list
> + apt-key adv --fetch-keys $ISAR_KEY
> + apt update
> + yes | apt install avocado
apt must not be used in scripts ... and it tells you about it.
Henning
> +EOSUDO
> + echo "Installed Avocado-framework"
> + set -e
> +fi
> +
> show_help() {
> echo "This script builds the default Isar images."
> echo
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v10 17/17] ci_build: Install Avocado if needed
2021-08-12 15:58 ` Henning Schild
@ 2021-09-16 14:11 ` Baurzhan Ismagulov
0 siblings, 0 replies; 22+ messages in thread
From: Baurzhan Ismagulov @ 2021-09-16 14:11 UTC (permalink / raw)
To: isar-users
On Thu, Aug 12, 2021 at 05:58:58PM +0200, Henning Schild wrote:
> > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
...
> > +# Install Avocado if needed
...
> I do not like this.
...
> apt must not be used in scripts ... and it tells you about it.
That was only intended to help testing the series, dropped in v11. The
execution environment should provide the tools, with Debian packages or pip.
> The package should be mainlined into debian
We've filed an ITP: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993862
> And i want more than "buster".
Provided stretch, buster, bullseye, bookworm at
http://deb.isar-build.org/debian-isar. The bookworm one should work for sid,
but we haven't tested it.
> I started looking into the avocado tests we currently have. Seems that they
> are still pretty different from what we used to have. Will this new version
> implement all the tests we used to have?
...
> I was only playing with what we currently have in next. But that seems to
> serialize the builds listed in that yaml. While before bitbake did run them
> all at once.
The initial Avocado implementation (in next today) provided only a matrix for
building all of individual multiconfigs, one per testcase. Yes, v11 intends to
provide exactly the same testcases that we have in ci_build.sh today. We could
add the individual single multiconfigs later if needed.
> My solution was to install that very unlucky choice of software with
> virtualenv
I assume this is about Avocado -- perhaps you could share some specific issues
you'd like to have addressed?
We did spend some time to evaluate our alternatives before choosing a specific
tool. Avocado is suitable for running Python and shell testcases, is not
limited to module testing, and is written in Python. It is a part of an
ecosystem supporting use cases like virtual machines, parallel execution,
easy testsuite composing via testcase tags, etc. It is the successor of
Autotest and has wide and very active community. We've used it in other
projects before Isar and got good impression. Having a test framework is waaay
better than maintaining ci_build.sh -- in the same way Isar is better than
build.sh ;) .
Of course, we are open to suggestions and improvements. One thing we have on
our radar is artifact reusing and testcase dependencies. E.g., with parallel
testcase execution, a start_vm testcase could depend on a bitbake one.
> I think the tests from that yaml should run in parallel to cause the same
> kind of stress ... just parallel in the testsuite instead of bitbake. And to
> hopefully speed things up a bit.
For the migration step, we aim at providing exactly the same testsuite as in
ci_build.sh and rework in later steps -- we'll share an RFC for that. Regarding
parallel execution, I think we need both parallel and bitbake with several
multiconfigs. This tradeoff isn't only about performance and stress; the latter
use case is necessary for per-build stuff (build numbers, release management,
etc.) and could potentially catch issues like missing dependencies, locking,
etc. that would be missed with the parallel execution of single multiconfig
builds.
A note on yaml -- that approach was very rigid, it supported only combinations
of single multiconfigs per testcase (we need more as described above) and
executed all of them (we need less to minimize the execution time).
With kind regards,
Baurzhan.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v10 00/17] Update Avocado testsuite
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (16 preceding siblings ...)
2021-07-30 8:27 ` [PATCH v10 17/17] ci_build: Install Avocado if needed Anton Mikanovich
@ 2021-08-12 16:13 ` Henning Schild
2021-08-12 16:19 ` Henning Schild
18 siblings, 0 replies; 22+ messages in thread
From: Henning Schild @ 2021-08-12 16:13 UTC (permalink / raw)
To: Anton Mikanovich; +Cc: isar-users
I started looking into the avocado tests we currently have. Seems that
they are still pretty different from what we used to have. Will this
new version implement all the tests we used to have?
My solution was to install that very unlucky choice of software with
virtualenv (what i always do when forced to pip).
And i worked on the gitlab-ci stuff. Came up with something similar
like you. Just a few more bits on nice integration i will propose later
when this series got merged.
Henning
Am Fri, 30 Jul 2021 11:27:22 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:
> This patchset provides implementation of build and vm_start tests
> functionality in Python with help of Avocado framework.
> Scripts ci_build and vm_smoke_test are rewritten to call Avocado.
> ci_build and vm_smoke_test are left for compatibility and
> should be replaced by direct avocado calls later.
> Original ci_build and vm_smoke_test are renamed to *_legacy.
> Build and run logs are hidden now until an error occurs.
>
> ---
> Changes since v9:
> - Rebase on next.
> - Fix avocado artifacts path.
> Changes since v8:
> - Remove duplication in logs.
> - Disable sysinfo plugin.
> - Show full errors in default mode.
> Changes since v7:
> - Remove code duplication.
> - Fix build_dir parsing.
> - Add missed protect files backup logic.
> Changes since v6:
> - Improve environment parsing.
> - Protect files backup logic.
> - Rebuild full log output option.
> - Protect ubuntu target with KFAIL.
> Changes since v5:
> - Rebase on current next.
> Changes since v4:
> - Add container tests.
> - Implement testcase skipping logic.
> - Separate and protect Bullseye tests.
> - Add checks for container dependencies.
> - Add Ilbers repo key.
> - Improve error handling.
> - Fix Ubuntu qemu image name.
> Changes since v3:
> - Add automatic bitbake init.
> - Fix line lengths.
> - Fix build without repro.
> - Fix build artifact paths.
> - Prepare for Gitlab build.
> - Change debug settings.
> - Move log splitting to start_vm.
> - Add deb package install.
> Changes since v2:
> - Make CI call Avocado tests.
> - Wait less on vm_start test.
> - Split Roman's vm_start commit.
> - Put logs to build folder.
> - Rearrange patches.
> - Split build test to classes.
> - Add test case tagging.
> - Get rid of avocado-framework-plugin-varianter-yaml-to-mux.
> - Get rid of python-subprocess32.
> - Improve logging.
> Changes since v1:
> - Merge start_vm rebuild patches.
> - Fix patch comments.
>
> Anton Mikanovich (14):
> start_vm.py: Fix ubuntu image name
> vm_boot_test: Fix log file path in vm_boot_test
> vm_boot_test: Remove external varianter
> vm_boot_test: Improve QEMU images checking
> build_test: Refactoring build tests cases
> testsuite: Add Python generations for testsuite in gitignore
> testsuite: Fix test suite prepare guide
> gitlab-ci: Add Avocado build artifacts
> gitlab-ci: Add debug flag
> vm_boot_test: Add automatic bitbake init
> build_test: Protect ubuntu target with KFAIL
> ci_build: Migrate to Avocado
> vm_smoke_test: Migrate to Avocado
> ci_build: Install Avocado if needed
>
> Roman Pletnev (3):
> start_vm.py: Fix target name handling
> start_vm.py: Add output and PID file vm_start.py options
> start_vm.py: Add MIPS support
>
> .gitlab-ci.yml | 5 +-
> scripts/ci_build.sh | 213 ++++----------------
> scripts/ci_build_legacy.sh | 267
> +++++++++++++++++++++++++ scripts/vm_smoke_test |
> 110 ++-------- scripts/vm_smoke_test_legacy | 168
> ++++++++++++++++ testsuite/.gitignore | 1 +
> testsuite/README.md | 21 +-
> testsuite/build_test/build_test.py | 217 ++++++++++++++++++--
> testsuite/build_test/cibase.py | 110 ++++++++++
> testsuite/build_test/cibuilder.py | 130 ++++++++++++
> testsuite/build_test/run.sh | 3 -
> testsuite/build_test/run_fast.sh | 3 +
> testsuite/build_test/run_full.sh | 3 +
> testsuite/build_test/variant.yaml | 22 --
> testsuite/start_vm.py | 27 ++-
> testsuite/vm_boot_test/run.sh | 3 -
> testsuite/vm_boot_test/run_fast.sh | 3 +
> testsuite/vm_boot_test/run_full.sh | 3 +
> testsuite/vm_boot_test/variant.yaml | 22 --
> testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++--
> 20 files changed, 1070 insertions(+), 382 deletions(-)
> create mode 100755 scripts/ci_build_legacy.sh
> create mode 100755 scripts/vm_smoke_test_legacy
> create mode 100644 testsuite/.gitignore
> create mode 100644 testsuite/build_test/cibase.py
> create mode 100644 testsuite/build_test/cibuilder.py
> delete mode 100755 testsuite/build_test/run.sh
> create mode 100755 testsuite/build_test/run_fast.sh
> create mode 100755 testsuite/build_test/run_full.sh
> delete mode 100644 testsuite/build_test/variant.yaml
> delete mode 100755 testsuite/vm_boot_test/run.sh
> create mode 100755 testsuite/vm_boot_test/run_fast.sh
> create mode 100755 testsuite/vm_boot_test/run_full.sh
> delete mode 100644 testsuite/vm_boot_test/variant.yaml
>
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v10 00/17] Update Avocado testsuite
2021-07-30 8:27 [PATCH v10 00/17] Update Avocado testsuite Anton Mikanovich
` (17 preceding siblings ...)
2021-08-12 16:13 ` [PATCH v10 00/17] Update Avocado testsuite Henning Schild
@ 2021-08-12 16:19 ` Henning Schild
18 siblings, 0 replies; 22+ messages in thread
From: Henning Schild @ 2021-08-12 16:19 UTC (permalink / raw)
To: Anton Mikanovich; +Cc: isar-users
I was only playing with what we currently have in next. But that seems
to serialize the builds listed in that yaml. While before bitbake did
run them all at once.
I think the tests from that yaml should run in parallel to cause the
same kind of stress ... just parallel in the testsuite instead of
bitbake. And to hopefully speed things up a bit.
Henning
Am Fri, 30 Jul 2021 11:27:22 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:
> This patchset provides implementation of build and vm_start tests
> functionality in Python with help of Avocado framework.
> Scripts ci_build and vm_smoke_test are rewritten to call Avocado.
> ci_build and vm_smoke_test are left for compatibility and
> should be replaced by direct avocado calls later.
> Original ci_build and vm_smoke_test are renamed to *_legacy.
> Build and run logs are hidden now until an error occurs.
>
> ---
> Changes since v9:
> - Rebase on next.
> - Fix avocado artifacts path.
> Changes since v8:
> - Remove duplication in logs.
> - Disable sysinfo plugin.
> - Show full errors in default mode.
> Changes since v7:
> - Remove code duplication.
> - Fix build_dir parsing.
> - Add missed protect files backup logic.
> Changes since v6:
> - Improve environment parsing.
> - Protect files backup logic.
> - Rebuild full log output option.
> - Protect ubuntu target with KFAIL.
> Changes since v5:
> - Rebase on current next.
> Changes since v4:
> - Add container tests.
> - Implement testcase skipping logic.
> - Separate and protect Bullseye tests.
> - Add checks for container dependencies.
> - Add Ilbers repo key.
> - Improve error handling.
> - Fix Ubuntu qemu image name.
> Changes since v3:
> - Add automatic bitbake init.
> - Fix line lengths.
> - Fix build without repro.
> - Fix build artifact paths.
> - Prepare for Gitlab build.
> - Change debug settings.
> - Move log splitting to start_vm.
> - Add deb package install.
> Changes since v2:
> - Make CI call Avocado tests.
> - Wait less on vm_start test.
> - Split Roman's vm_start commit.
> - Put logs to build folder.
> - Rearrange patches.
> - Split build test to classes.
> - Add test case tagging.
> - Get rid of avocado-framework-plugin-varianter-yaml-to-mux.
> - Get rid of python-subprocess32.
> - Improve logging.
> Changes since v1:
> - Merge start_vm rebuild patches.
> - Fix patch comments.
>
> Anton Mikanovich (14):
> start_vm.py: Fix ubuntu image name
> vm_boot_test: Fix log file path in vm_boot_test
> vm_boot_test: Remove external varianter
> vm_boot_test: Improve QEMU images checking
> build_test: Refactoring build tests cases
> testsuite: Add Python generations for testsuite in gitignore
> testsuite: Fix test suite prepare guide
> gitlab-ci: Add Avocado build artifacts
> gitlab-ci: Add debug flag
> vm_boot_test: Add automatic bitbake init
> build_test: Protect ubuntu target with KFAIL
> ci_build: Migrate to Avocado
> vm_smoke_test: Migrate to Avocado
> ci_build: Install Avocado if needed
>
> Roman Pletnev (3):
> start_vm.py: Fix target name handling
> start_vm.py: Add output and PID file vm_start.py options
> start_vm.py: Add MIPS support
>
> .gitlab-ci.yml | 5 +-
> scripts/ci_build.sh | 213 ++++----------------
> scripts/ci_build_legacy.sh | 267
> +++++++++++++++++++++++++ scripts/vm_smoke_test |
> 110 ++-------- scripts/vm_smoke_test_legacy | 168
> ++++++++++++++++ testsuite/.gitignore | 1 +
> testsuite/README.md | 21 +-
> testsuite/build_test/build_test.py | 217 ++++++++++++++++++--
> testsuite/build_test/cibase.py | 110 ++++++++++
> testsuite/build_test/cibuilder.py | 130 ++++++++++++
> testsuite/build_test/run.sh | 3 -
> testsuite/build_test/run_fast.sh | 3 +
> testsuite/build_test/run_full.sh | 3 +
> testsuite/build_test/variant.yaml | 22 --
> testsuite/start_vm.py | 27 ++-
> testsuite/vm_boot_test/run.sh | 3 -
> testsuite/vm_boot_test/run_fast.sh | 3 +
> testsuite/vm_boot_test/run_full.sh | 3 +
> testsuite/vm_boot_test/variant.yaml | 22 --
> testsuite/vm_boot_test/vm_boot_test.py | 121 +++++++++--
> 20 files changed, 1070 insertions(+), 382 deletions(-)
> create mode 100755 scripts/ci_build_legacy.sh
> create mode 100755 scripts/vm_smoke_test_legacy
> create mode 100644 testsuite/.gitignore
> create mode 100644 testsuite/build_test/cibase.py
> create mode 100644 testsuite/build_test/cibuilder.py
> delete mode 100755 testsuite/build_test/run.sh
> create mode 100755 testsuite/build_test/run_fast.sh
> create mode 100755 testsuite/build_test/run_full.sh
> delete mode 100644 testsuite/build_test/variant.yaml
> delete mode 100755 testsuite/vm_boot_test/run.sh
> create mode 100755 testsuite/vm_boot_test/run_fast.sh
> create mode 100755 testsuite/vm_boot_test/run_full.sh
> delete mode 100644 testsuite/vm_boot_test/variant.yaml
>
^ permalink raw reply [flat|nested] 22+ messages in thread