From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6674828288478674944 X-Received: by 2002:ac2:51a1:: with SMTP id f1mr1680651lfk.13.1554104566034; Mon, 01 Apr 2019 00:42:46 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:43af:: with SMTP id t15ls717102lfl.8.gmail; Mon, 01 Apr 2019 00:42:45 -0700 (PDT) X-Google-Smtp-Source: APXvYqyM1WI/HDZYbX7EN2TY5mH+w5eNOaagnpdKI59BUwvIlSCCauoY3m8RxruYxXM3n7I3s1m9 X-Received: by 2002:ac2:5463:: with SMTP id e3mr1682893lfn.6.1554104565483; Mon, 01 Apr 2019 00:42:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554104565; cv=none; d=google.com; s=arc-20160816; b=nBW4ED0rCJ6h8QkP2ihF5YqKoXxhvQrZJE+DmNYxs9+H5Cbfs2D4fIraFSo7j7Veb7 X57MS4oA/Al4fjvbmXJrnK87fLtjYVVSA3dgvyjjRH48A2iDzR2vzntu1SWfjqZbbOfj W2olK2MuLXdtN4KkiOACDaMS8JrxYIvjTl0mWpX8Or2q1z99ukgKFxMiN6L3XaKPvrEM 8nbABhnL0u8wRVChLtVnGH0P25akX8QSBARQns/tfc8HRkD/nz6g0J4nJMoXYwy9D+PW EBAA+TlHETx+VuVA6OW1+Umnw7Jmewa1+NhgDSoQcLppyuvJBoIoNRTykYSf39EQZDp1 Cy4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:to:from; bh=oZFPxMbfwEcOwPtGKT/0k/1TU0tmitvBbTxhK3Ce2Yw=; b=FLZ/5AvsAhjz/U9mr+//Q9aOcWoiXEObshYIVUh2L+dkoCru2QkEDZpNWKgRuGDqgM /Q7v+KKthuv1osrEEuH9pT3VGAHPlh71nvhLEzghMZnjmMHfiHYO0Ome2e86MTD5p2P0 AmHBktbQYmfE4QNfWAPokNXzFP5ege9ARGfpB3egV5oyvNrqbaLXy/9PKhP45iGU6EZU Zdr/6FnnlUNGbceiSx7em3xL1w+ppZpzl8yLpdlTZWnAXkAYOeGC5O7HxPeP8HJSFtSk 4RGBWzf0n0en/1VEBnxrsqIkj8+vcrk4ut6g9wy6E1Mo2L8wEm0pOlJVFqBNKmb/nBjA cG0A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id a22si327347lfl.3.2019.04.01.00.42.45 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 01 Apr 2019 00:42:45 -0700 (PDT) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from azat.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x317gPYF027367 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 1 Apr 2019 09:42:43 +0200 From: "Maxim Yu. Osipov" To: isar-users@googlegroups.com Subject: [PATCH] testsuite: Initial avocado tests implementation Date: Mon, 1 Apr 2019 09:42:24 +0200 Message-Id: <20190401074224.10500-1-mosipov@ilbers.de> X-Mailer: git-send-email 2.11.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: flF7BgzIJN5E From: Alexander Smirnov Signed-off-by: Alexander Smirnov Tested-by: Maxim Yu. Osipov --- testsuite/README.md | 34 +++++++++++++ testsuite/build_test/build_test.py | 29 +++++++++++ testsuite/build_test/run.sh | 3 ++ testsuite/build_test/variant.yaml | 22 +++++++++ testsuite/start_vm.py | 88 ++++++++++++++++++++++++++++++++++ testsuite/vm_boot_test/run.sh | 3 ++ testsuite/vm_boot_test/variant.yaml | 22 +++++++++ testsuite/vm_boot_test/vm_boot_test.py | 51 ++++++++++++++++++++ 8 files changed, 252 insertions(+) create mode 100644 testsuite/README.md create mode 100644 testsuite/build_test/build_test.py create mode 100755 testsuite/build_test/run.sh create mode 100644 testsuite/build_test/variant.yaml create mode 100755 testsuite/start_vm.py create mode 100755 testsuite/vm_boot_test/run.sh create mode 100644 testsuite/vm_boot_test/variant.yaml create mode 100644 testsuite/vm_boot_test/vm_boot_test.py diff --git a/testsuite/README.md b/testsuite/README.md new file mode 100644 index 0000000..5e64223 --- /dev/null +++ b/testsuite/README.md @@ -0,0 +1,34 @@ +# Install Avocado + +The framework could be installed by using standard HOWTO: + + https://github.com/avocado-framework/avocado#installing-with-standard-python-tools + +Then you need to install varianter yaml-to-mux plugin by following these instructions: + + https://github.com/avocado-framework/avocado/tree/master/optional_plugins + +## For Debian 9.x + + $ sudo apt-get install python-pip + $ pip install --user subprocess32 + $ pip install --user avocado-framework + $ pip install --user avocado-framework-plugin-varianter-yaml-to-mux + +# Pre + + $ export PATH=$PATH:~/.local/bin + $ cd isar + $ source isar-init-build-env + + +# Run test + +Each testsuite directory contains: + - run.sh - script to start tests + - variants.yaml - set of input data + - *.py - test case + +# Other + +There is a tool start_vm.py which is the replacement for the bash script in isar/scripts directory. diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py new file mode 100644 index 0000000..4220d3a --- /dev/null +++ b/testsuite/build_test/build_test.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python3 + +import os +import subprocess32 +import sys +from os.path import dirname + +from avocado import Test + +class BuildTest(Test): + + def test(self): + # TODO: add default values + build_dir = self.params.get('build_dir', default='.') + arch = self.params.get('arch', default='arm') + distro = self.params.get('distro', default='stretch') + + self.log.info('===================================================') + self.log.info('Running Isar build test for (' + distro + '-' + arch + ')') + self.log.info('Isar build folder is: ' + build_dir) + self.log.info('===================================================') + + #isar_root = dirname(__file__) + '/..' + os.chdir(build_dir) + cmdline = ['bitbake', 'multiconfig:qemu' + arch + '-' + distro + ':isar-image-base'] + p1 = subprocess32.run(cmdline) + + if p1.returncode: + self.fail('Test failed') diff --git a/testsuite/build_test/run.sh b/testsuite/build_test/run.sh new file mode 100755 index 0000000..a8ea9cc --- /dev/null +++ b/testsuite/build_test/run.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +avocado run build_test.py --mux-yaml test:variant.yaml --mux-inject build_dir:$BUILDDIR diff --git a/testsuite/build_test/variant.yaml b/testsuite/build_test/variant.yaml new file mode 100644 index 0000000..9ddc634 --- /dev/null +++ b/testsuite/build_test/variant.yaml @@ -0,0 +1,22 @@ +variants: !mux + stretch-amd64: + distro: "stretch" + arch: "amd64" + stretch-i386: + distro: "stretch" + arch: "i386" + stretch-arm: + distro: "stretch" + arch: "arm" + stretch-arm64: + distro: "stretch" + arch: "arm64" + buster-amd64: + distro: "buster" + arch: "amd64" + buster-i386: + distro: "buster" + arch: "i386" + buster-arm: + distro: "buster" + arch: "arm" diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py new file mode 100755 index 0000000..02a4b51 --- /dev/null +++ b/testsuite/start_vm.py @@ -0,0 +1,88 @@ +#!/usr/bin/env python3 +# +# Helper script to start QEMU with Isar image +# Copyright (c) 2019, ilbers GmbH + +import argparse +import os +import subprocess +import sys +import time + +def get_bitbake_env(arch, distro): + multiconfig = 'multiconfig:qemu' + arch + '-' + distro + ':isar-image-base' + output = subprocess.check_output(['bitbake', '-e', str(multiconfig)]) + return output + +def get_bitbake_var(output, var): + ret = '' + for line in output.splitlines(): + if line.startswith(var): + ret = line.split('"')[1] + return ret + +def format_qemu_cmdline(arch, build, distro): + bb_output = get_bitbake_env(arch, distro).decode() + + rootfs_image = '' + extra_args = '' + cpu = [''] + + image_type = get_bitbake_var(bb_output, 'IMAGE_TYPE') + deploy_dir_image = get_bitbake_var(bb_output, 'DEPLOY_DIR_IMAGE') + if image_type == 'ext4-img': + rootfs_image = 'isar-image-base-debian-' + distro + '-qemu' + arch + '.ext4.img' + kernel_image = deploy_dir_image + '/' + get_bitbake_var(bb_output, 'KERNEL_IMAGE') + initrd_image = get_bitbake_var(bb_output, 'INITRD_IMAGE') + + if not initrd_image: + initrd_image = '/dev/null' + else: + initrd_image = deploy_dir_image + '/' + initrd_image + + serial = get_bitbake_var(bb_output, 'MACHINE_SERIAL') + root_dev = get_bitbake_var(bb_output, 'QEMU_ROOTFS_DEV') + kargs = ['-append', '"console=' + serial + ' root=/dev/' + root_dev + ' rw"'] + + extra_args = ['-kernel', kernel_image, '-initrd', initrd_image] + extra_args.extend(kargs) + elif image_type == 'wic-img': + rootfs_image = 'isar-image-base-debian-' + distro + '-qemu' + arch + '.wic.img' + extra_args = ['-snapshot'] + else: + raise ValueError('Invalid image type: ' + str(image_type)) + + qemu_arch = get_bitbake_var(bb_output, 'QEMU_ARCH') + qemu_machine = get_bitbake_var(bb_output, 'QEMU_MACHINE') + qemu_cpu = get_bitbake_var(bb_output, 'QEMU_CPU') + qemu_disk_args = get_bitbake_var(bb_output, 'QEMU_DISK_ARGS') + + qemu_disk_args = qemu_disk_args.replace('##ROOTFS_IMAGE##', deploy_dir_image + '/' + rootfs_image).split() + + cmd = ['qemu-system-' + qemu_arch, '-m', '1024M'] + + if qemu_machine: + cmd.extend(['-M', qemu_machine]) + + if qemu_cpu: + cmd.extend(['-cpu', qemu_cpu]) + + cmd.extend(extra_args) + cmd.extend(qemu_disk_args) + + return cmd + +def start_qemu(arch, build, distro): + cmdline = format_qemu_cmdline(arch, build, distro) + cmdline.insert(1, '-nographic') + + p1 = subprocess.call(cmdline) + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386'], help='set isar machine architecture.', default='arm') + parser.add_argument('-b', '--build', help='set path to build directory.', default=os.getcwd()) + parser.add_argument('-d', '--distro', choices=['jessie', 'stretch'], help='set isar Debian distribution.', default='stretch') + args = parser.parse_args() + + start_qemu(args.arch, args.build, args.distro) diff --git a/testsuite/vm_boot_test/run.sh b/testsuite/vm_boot_test/run.sh new file mode 100755 index 0000000..9fdda95 --- /dev/null +++ b/testsuite/vm_boot_test/run.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +avocado run vm_boot_test.py --mux-yaml test:variant.yaml --mux-inject build_dir:$BUILDDIR time_to_wait:300 diff --git a/testsuite/vm_boot_test/variant.yaml b/testsuite/vm_boot_test/variant.yaml new file mode 100644 index 0000000..9ddc634 --- /dev/null +++ b/testsuite/vm_boot_test/variant.yaml @@ -0,0 +1,22 @@ +variants: !mux + stretch-amd64: + distro: "stretch" + arch: "amd64" + stretch-i386: + distro: "stretch" + arch: "i386" + stretch-arm: + distro: "stretch" + arch: "arm" + stretch-arm64: + distro: "stretch" + arch: "arm64" + buster-amd64: + distro: "buster" + arch: "amd64" + buster-i386: + distro: "buster" + arch: "i386" + buster-arm: + distro: "buster" + arch: "arm" diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py new file mode 100644 index 0000000..d4849c7 --- /dev/null +++ b/testsuite/vm_boot_test/vm_boot_test.py @@ -0,0 +1,51 @@ +#!/usr/bin/env python3 + +import os +import subprocess32 +import sys +import time + +from os.path import dirname +sys.path.append(dirname(__file__) + '/..') + +import start_vm + +from avocado import Test + +class VmBootTest(Test): + + def test(self): + # TODO: add default values + build_dir = self.params.get('build_dir', default='.') + arch = self.params.get('arch', default='arm') + distro = self.params.get('distro', default='stretch') + time_to_wait = self.params.get('time_to_wait', default=60) + + self.log.info('===================================================') + self.log.info('Running Isar VM boot test for (' + distro + '-' + arch + ')') + self.log.info('Isar build folder is: ' + build_dir) + self.log.info('===================================================') + + output_file = '/tmp/vm_boot_test.log' + if os.path.exists(output_file): + os.remove(output_file) + + cmdline = start_vm.format_qemu_cmdline(arch, build_dir, distro) + cmdline.insert(1, '-nographic') + cmdline.append('-serial') + cmdline.append('file:' + output_file) + + self.log.info('QEMU boot line: ' + str(cmdline)) + + devnull = open(os.devnull, 'w') + + p1 = subprocess32.Popen(cmdline, stdout=devnull, stderr=devnull) + time.sleep(int(time_to_wait)) + p1.kill() + p1.wait() + + if os.path.exists(output_file): + if 'isar login:' in open(output_file).read(): + return + + self.fail('Test failed') -- 2.11.0