From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6928418269464363008 X-Received: by 2002:a05:6000:1547:: with SMTP id 7mr17951376wry.301.1613381233446; Mon, 15 Feb 2021 01:27:13 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:770f:: with SMTP id t15ls132193wmi.0.gmail; Mon, 15 Feb 2021 01:27:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJwonROSGc4QYuUWF3LQevfy+Bic3SSW5G2USe72fnClfmQwU5UmGBUU9gf08FMVaKaRNWMp X-Received: by 2002:a1c:c289:: with SMTP id s131mr7407207wmf.85.1613381232567; Mon, 15 Feb 2021 01:27:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1613381232; cv=none; d=google.com; s=arc-20160816; b=R81zKDAXhcADf165ugCk0haSnh8mLR71QI08mohFSy58Pe8TLvCoqTyBBwRWIQSOOi 96VZdiafw08OP28zrHa/ymJcD4N7YrMXhG7AvpGUg4RpUb6vKU0DevqzjHxvSl5v2ZN3 Bo1lIHoceQd0ICvk+UrTnd9XsdF4Y5pQTHl8UxI/7yLt5klIU920U7s3rDeRhxKmW9Yk qzt7fDZxgiJDrBiwwGElj7XZTReiAMfHFIFF2kNzD1y3UmpXTFfireUeo4+JtswtoBEz Sn4jjA3Iu/9FCpGeR79WM41C+5j2tGYz4Cwv03wKG+zfhnzbSvookU/5mfZZfIJfq7Tw FjFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=efwidZPVZCF+g+mNAtr/2Rc/bkzjvO+CoFZhgS5gm7c=; b=yR49ZWA8l4Qe6TjxSbYU6gxQS9rzV6LUhx5ttpuFHoX43WMZ0OPReM5cERS3soVXxq 1ysHA/XfzF+hAGvwXzeRoHLS6MSgpZ3Y6ANnuzU5GxxigUaybrBNdIaxS4OkM0c58rh2 LO/gHqaPKwRFzMOAgNbpocV0PDkHmIpbucGM5Ti5U+bnN2V7l9wIUjpbZUiuMmrsBgFP zfHEOcVKUQjS8yvz1vf0jtauSmNI+Dl63fl2PpkMmq6W3b/hbvNXT6+gN5n+5DvqL5DO CWIuDzpel6UZmlaY5lcsaSLWLvDGUqJbz/rdJveyWswVP2m3n95LcQkkhDAjRyMpUA47 Xt2g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id v16si980365wmh.1.2021.02.15.01.27.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 15 Feb 2021 01:27:12 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from localhost.localdomain (mm-131-52-214-37.mgts.dynamic.pppoe.byfly.by [37.214.52.131] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 11F9QnrE003503 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 15 Feb 2021 10:27:11 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v2 5/5] ci-build: Add ci_build tests cases to Avocado Date: Mon, 15 Feb 2021 12:26:35 +0300 Message-Id: <20210215092635.24511-6-amikan@ilbers.de> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210215092635.24511-1-amikan@ilbers.de> References: <20210212164115.89196-1-amikan@ilbers.de> <20210215092635.24511-1-amikan@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 5O2BWGswqDkt All test cases were copied as-is without any changes. Signed-off-by: Anton Mikanovich --- testsuite/build_test/build_repro_test.py | 75 +++++++++++++++++++ testsuite/build_test/build_test.py | 58 ++++++++++---- testsuite/build_test/build_wic_test.py | 65 ++++++++++++++++ testsuite/build_test/cibuilder.py | 66 ++++++++++++++++ testsuite/build_test/run.sh | 3 - testsuite/build_test/run_fast.sh | 15 ++++ testsuite/build_test/run_full.sh | 18 +++++ testsuite/build_test/variants.yaml | 45 +++++++++++ testsuite/build_test/variants_cross.yaml | 22 ++++++ testsuite/build_test/variants_repro.yaml | 13 ++++ testsuite/build_test/variants_sdk.yaml | 5 ++ .../build_test/variants_wic_exclude.yaml | 4 + 12 files changed, 373 insertions(+), 16 deletions(-) create mode 100644 testsuite/build_test/build_repro_test.py create mode 100644 testsuite/build_test/build_wic_test.py create mode 100644 testsuite/build_test/cibuilder.py delete mode 100755 testsuite/build_test/run.sh create mode 100755 testsuite/build_test/run_fast.sh create mode 100755 testsuite/build_test/run_full.sh create mode 100644 testsuite/build_test/variants.yaml create mode 100644 testsuite/build_test/variants_cross.yaml create mode 100644 testsuite/build_test/variants_repro.yaml create mode 100644 testsuite/build_test/variants_sdk.yaml create mode 100644 testsuite/build_test/variants_wic_exclude.yaml diff --git a/testsuite/build_test/build_repro_test.py b/testsuite/build_test/build_repro_test.py new file mode 100644 index 0000000..32a0a51 --- /dev/null +++ b/testsuite/build_test/build_repro_test.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python3 + +import os +import subprocess32 +import tempfile + +from avocado import Test + +import cibuilder + +class BuildTest(Test): + + def test_repro(self): + isar_root = os.path.dirname(__file__) + '/../..' + + build_dir = self.params.get('build_dir', default=isar_root + '/build') + machine = self.params.get('machine', default='qemuamd64') + distro = self.params.get('distro', default='buster') + image = self.params.get('image', default='isar-image-base') + + bitbake_args = self.params.get('bitbake_args', default='-v') + bitbake_cmd = self.params.get('bitbake_cmd', default=None) + + cross = self.params.get('cross', default=None) + debug = self.params.get('debug', default=None) + quiet = self.params.get('quiet', default=None) + signed = self.params.get('signed', default=None) + compat_arch = self.params.get('compat_arch', default=1) + debsrc_cache = self.params.get('debsrc_cache', default=1) + + target = self.name.variant.rsplit('-', 1)[0] + + self.log.info('===================================================') + self.log.info('Running repro Isar build test for: ' + target) + self.log.info('Isar build folder is: ' + build_dir) + self.log.info('===================================================') + + cibuilder.init(isar_root, build_dir) + cibuilder.confprepare(build_dir, compat_arch, cross, debsrc_cache) + + gpg_pub_key = os.path.dirname(__file__) + '/../base-apt/test_pub.key' + gpg_priv_key = os.path.dirname(__file__) + '/../base-apt/test_priv.key' + + if signed: + with open(build_dir + '/conf/ci_build.conf', 'w') as file: + # Enable use of signed cached base repository + file.write('BASE_REPO_KEY="file://' + gpg_pub_key + '"\n') + + if debug: + bitbake_args += ' -D' + if quiet: + bitbake_args = '' + + os.chdir(build_dir) + + os.environ['GNUPGHOME'] = tempfile.mkdtemp() + result = subprocess32.run(['gpg', '--import', gpg_pub_key, gpg_priv_key]) + + if result.returncode: + self.fail('GPG import failed') + + if cibuilder.bitbake(build_dir, 'mc:' + target + ':' + image, bitbake_cmd, bitbake_args).returncode: + self.fail('Repro test failed') + + cibuilder.deletetmp(build_dir) + with open(build_dir + '/conf/ci_build.conf', 'a') as file: + file.write('ISAR_USE_CACHED_BASE_REPO = "1"\n') + file.write('BB_NO_NETWORK = "1"\n') + + if cibuilder.bitbake(build_dir, 'mc:' + target + ':' + image, bitbake_cmd, bitbake_args).returncode: + self.fail('Repro cached test failed') + + # Cleanup and disable use of signed cached base repository + cibuilder.deletetmp(build_dir) + cibuilder.confcleanup(build_dir) diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py index 7a55c2f..7b1d9bc 100644 --- a/testsuite/build_test/build_test.py +++ b/testsuite/build_test/build_test.py @@ -1,29 +1,61 @@ #!/usr/bin/env python3 import os -import subprocess32 -import sys -from os.path import dirname from avocado import Test +import cibuilder + class BuildTest(Test): def test(self): - # TODO: add default values - build_dir = self.params.get('build_dir', default='.') - arch = self.params.get('arch', default='arm') - distro = self.params.get('distro', default='stretch') + isar_root = os.path.dirname(__file__) + '/../..' + + build_dir = self.params.get('build_dir', default=isar_root + '/build') + machine = self.params.get('machine', default='qemuamd64') + distro = self.params.get('distro', default='buster') + image = self.params.get('image', default='isar-image-base') + + bitbake_args = self.params.get('bitbake_args', default='-v') + bitbake_cmd = self.params.get('bitbake_cmd', default=None) + + cross = self.params.get('cross', default=None) + debug = self.params.get('debug', default=None) + quiet = self.params.get('quiet', default=None) + compat_arch = self.params.get('compat_arch', default=1) + debsrc_cache = self.params.get('debsrc_cache', default=1) + rebuild = self.params.get('rebuild', default=0) + + target = self.name.variant.rsplit('-', 1)[0] self.log.info('===================================================') - self.log.info('Running Isar build test for (' + distro + '-' + arch + ')') + self.log.info('Running Isar build test for: ' + target) self.log.info('Isar build folder is: ' + build_dir) self.log.info('===================================================') - #isar_root = dirname(__file__) + '/..' - os.chdir(build_dir) - cmdline = ['bitbake', 'mc:qemu' + arch + '-' + distro + ':isar-image-base'] - p1 = subprocess32.run(cmdline) + cibuilder.init(isar_root, build_dir) + cibuilder.confprepare(build_dir, compat_arch, cross, debsrc_cache) + + if debug: + bitbake_args += ' -D' + if quiet: + bitbake_args = '' - if p1.returncode: + self.log.info('Starting build of ' + target) + + if cibuilder.bitbake(build_dir, 'mc:' + target + ':' + image, bitbake_cmd, bitbake_args).returncode: self.fail('Test failed') + + if rebuild: + layerdir_core = cibuilder.getlayerdir('core') + + cibuilder.backupfile(layerdir_core + '/classes/dpkg-base.bbclass') + with open(layerdir_core + '/classes/dpkg-base.bbclass', 'a') as file: + file.write('do_fetch_append() {\n\n}\n') + + self.log.info('Starting rebuild of ' + target) + + if cibuilder.bitbake(build_dir, 'mc:' + target + ':' + image, bitbake_cmd, bitbake_args).returncode: + self.fail('Rebuild test failed') + + cibuilder.restorefile(layerdir_core + '/classes/dpkg-base.bbclass') diff --git a/testsuite/build_test/build_wic_test.py b/testsuite/build_test/build_wic_test.py new file mode 100644 index 0000000..b012cac --- /dev/null +++ b/testsuite/build_test/build_wic_test.py @@ -0,0 +1,65 @@ +#!/usr/bin/env python3 + +import os +import re + +from avocado import Test + +import cibuilder + +class BuildTest(Test): + + def test_wic(self): + isar_root = os.path.dirname(__file__) + '/../..' + + build_dir = self.params.get('build_dir', default=isar_root + '/build') + machine = self.params.get('machine', default='qemuamd64') + distro = self.params.get('distro', default='buster') + image = self.params.get('image', default='isar-image-base') + + bitbake_args = self.params.get('bitbake_args', default='-v') + bitbake_cmd = self.params.get('bitbake_cmd', default=None) + + cross = self.params.get('cross', default=None) + debug = self.params.get('debug', default=None) + quiet = self.params.get('quiet', default=None) + compat_arch = self.params.get('compat_arch', default=1) + debsrc_cache = self.params.get('debsrc_cache', default=1) + + target = self.name.variant.rsplit('-', 1)[0] + + self.log.info('===================================================') + self.log.info('Running WIC exclude build test for: ' + target) + self.log.info('Isar build folder is: ' + build_dir) + self.log.info('===================================================') + + cibuilder.init(isar_root, build_dir) + cibuilder.confprepare(build_dir, compat_arch, cross, debsrc_cache) + + layerdir_isar = cibuilder.getlayerdir('isar') + + wks_file = layerdir_isar + '/scripts/lib/wic/canned-wks/sdimage-efi.wks' + wic_img = build_dir + '/tmp/deploy/images/' + machine + '/' + image + '-debian-' + distro + '-' + machine + '.wic.img' + + if not os.path.isfile(wic_img): + self.fail('Build was not performed before: ' + wic_img + ' not exist') + + cibuilder.backupfile(wks_file) + cibuilder.backupmove(wic_img) + + with open(layerdir_isar + '/scripts/lib/wic/canned-wks/sdimage-efi.wks', 'r') as file: + lines = file.readlines() + with open(layerdir_isar + '/scripts/lib/wic/canned-wks/sdimage-efi.wks', 'w') as file: + for line in lines: + file.write(re.sub(r'part \/ ', 'part \/ --exclude-path usr ', line)) + + if debug: + bitbake_args += ' -D' + if quiet: + bitbake_args = '' + + if cibuilder.bitbake(build_dir, 'mc:' + target + ':' + image, bitbake_cmd, bitbake_args).returncode: + self.fail('WIC test failed') + + cibuilder.restorefile(wks_file) + cibuilder.restorefile(wic_img) diff --git a/testsuite/build_test/cibuilder.py b/testsuite/build_test/cibuilder.py new file mode 100644 index 0000000..3d09d26 --- /dev/null +++ b/testsuite/build_test/cibuilder.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 + +import os +import subprocess32 +import shutil + +from avocado import Test + +def init(isar_root, build_dir): + os.chdir(isar_root) + pipe = subprocess32.Popen("source isar-init-build-env %s 2>&1 >/dev/null; env" % build_dir, stdout=subprocess32.PIPE, shell=True, executable="/bin/bash") + output = pipe.communicate()[0].decode('utf-8') + env = dict((line.split("=", 1) for line in output.splitlines())) + os.environ.update(env) + +def confprepare(build_dir, compat_arch, cross, debsrc_cache): + with open(build_dir + '/conf/ci_build.conf', 'w') as file: + if compat_arch: + file.write('ISAR_ENABLE_COMPAT_ARCH_amd64 = "1"\n') + file.write('ISAR_ENABLE_COMPAT_ARCH_arm64 = "1"\n') + file.write('ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0"\n') + if cross: + file.write('ISAR_CROSS_COMPILE = "1"\n') + if debsrc_cache: + file.write('BASE_REPO_FEATURES ?= "cache-deb-src"\n') + + with open(build_dir + '/conf/local.conf', 'r+') as file: + for line in file: + if 'include ci_build.conf' in line: + break + else: + file.write('\ninclude ci_build.conf') + +def confcleanup(build_dir): + open(build_dir + '/conf/ci_build.conf', 'w').close() + +def deletetmp(build_dir): + subprocess32.run('sudo rm -rf ' + build_dir + '/tmp', shell=True, executable="/bin/bash") + +def bitbake(build_dir, target, cmd, args): + os.chdir(build_dir) + if not cmd and not args: + cmdline = ['bitbake', target] + elif not cmd: + cmdline = ['bitbake', args, target] + elif not args: + cmdline = ['bitbake', '-c', cmd, target] + else: + cmdline = ['bitbake', args, '-c', cmd, target] + + return subprocess32.run(cmdline) + +def backupfile(path): + shutil.copy2(path, path + '.ci-backup') + +def backupmove(path): + shutil.move(path, path + '.ci-backup') + +def restorefile(path): + shutil.move(path + '.ci-backup', path) + +def getlayerdir(layer): + pipe = subprocess32.Popen('bitbake -e | grep "^LAYERDIR_.*="', stdout=subprocess32.PIPE, shell=True, executable="/bin/bash") + output = pipe.communicate()[0].decode('utf-8') + bitbake_env = dict((line.split("=", 1) for line in output.splitlines())) + return bitbake_env['LAYERDIR_' + layer].strip('"') diff --git a/testsuite/build_test/run.sh b/testsuite/build_test/run.sh deleted file mode 100755 index a8ea9cc..0000000 --- a/testsuite/build_test/run.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/bash - -avocado run build_test.py --mux-yaml test:variant.yaml --mux-inject build_dir:$BUILDDIR diff --git a/testsuite/build_test/run_fast.sh b/testsuite/build_test/run_fast.sh new file mode 100755 index 0000000..4c75cc0 --- /dev/null +++ b/testsuite/build_test/run_fast.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +export PATH=$PATH:~/.local/bin + +# Test cached base repository +avocado run build_repro_test.py --mux-yaml variants_repro.yaml --mux-inject quiet:1 cross:1 + +# Start cross build for the defined set of configurations +avocado run build_test.py --mux-yaml variants_cross.yaml --mux-inject quiet:1 cross:1 + +# In addition test SDK creation +avocado run build_test.py --mux-yaml variants_sdk.yaml --mux-inject quiet:1 cross:1 + +# Test wic --exclude-path +avocado run build_wic_test.py --mux-yaml variants_wic_exclude.yaml --mux-inject quiet:1 cross:1 diff --git a/testsuite/build_test/run_full.sh b/testsuite/build_test/run_full.sh new file mode 100755 index 0000000..59a87d4 --- /dev/null +++ b/testsuite/build_test/run_full.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +export PATH=$PATH:~/.local/bin + +# Test cached base repository +avocado run build_repro_test.py --mux-yaml variants_repro.yaml --mux-inject quiet:1 + +# Start cross build for the defined set of configurations +avocado run build_test.py --mux-yaml variants_cross.yaml --mux-inject quiet:1 cross:1 + +# In addition test SDK creation +avocado run build_test.py --mux-yaml variants_sdk.yaml --mux-inject quiet:1 cross:1 + +# Start non-cross build for the defined set of configurations +avocado run build_test.py --mux-yaml variants.yaml --mux-inject quiet:1 + +# Test wic --exclude-path +avocado run build_wic_test.py --mux-yaml variants_wic_exclude.yaml --mux-inject quiet:1 diff --git a/testsuite/build_test/variants.yaml b/testsuite/build_test/variants.yaml new file mode 100644 index 0000000..6c60586 --- /dev/null +++ b/testsuite/build_test/variants.yaml @@ -0,0 +1,45 @@ +variants: !mux + qemuarm-stretch: + machine: "qemuarm" + distro: "stretch" + qemuarm-buster: + machine: "qemuarm" + qemuarm64-stretch: + machine: "qemuarm64" + distro: "stretch" + qemui386-stretch: + machine: "qemui386" + distro: "stretch" + qemui386-buster: + machine: "qemui386" + qemuamd64-stretch: + machine: "qemuamd64" + distro: "stretch" + rebuild: 1 + qemuamd64-buster: + machine: "qemuamd64" + qemuamd64-buster-tgz: + machine: "qemuamd64" + qemumipsel-stretch: + machine: "qemumipsel" + distro: "stretch" + qemumipsel-buster: + machine: "qemumipsel" + nand-ubi-demo-buster: + machine: "nand-ubi-demo" + image: "isar-image-ubi" + rpi-stretch: + machine: "rpi" + distro: "stretch" + qemuamd64-bullseye: + machine: "qemuamd64" + distro: "bullseye" + qemuarm-bullseye: + machine: "qemuarm" + distro: "bullseye" + qemui386-bullseye: + machine: "qemui386" + distro: "bullseye" + qemumipsel-bullseye: + machine: "qemumipsel" + distro: "bullseye" diff --git a/testsuite/build_test/variants_cross.yaml b/testsuite/build_test/variants_cross.yaml new file mode 100644 index 0000000..e2732f3 --- /dev/null +++ b/testsuite/build_test/variants_cross.yaml @@ -0,0 +1,22 @@ +variants: !mux + qemuarm-stretch: + machine: "qemuarm" + distro: "stretch" + qemuarm-buster: + machine: "qemuarm" + qemuarm64-stretch: + machine: "qemuarm64" + distro: "stretch" + qemuamd64-stretch: + machine: "qemuamd64" + distro: "stretch" + de0-nano-soc-buster: + machine: "de0-nano-soc" + stm32mp15x-buster: + machine: "stm32mp15x" + rpi-stretch: + machine: "rpi" + distro: "stretch" + qemuarm-bullseye: + machine: "qemuarm" + distro: "bullseye" diff --git a/testsuite/build_test/variants_repro.yaml b/testsuite/build_test/variants_repro.yaml new file mode 100644 index 0000000..652f7c0 --- /dev/null +++ b/testsuite/build_test/variants_repro.yaml @@ -0,0 +1,13 @@ +variants: !mux + de0-nano-soc-buster: + machine: "de0-nano-soc" + signed: 1 + qemuarm64-stretch: + machine: "qemuarm64" + distro: "stretch" + signed: 1 + qemuamd64-stretch: + machine: "qemuamd64" + distro: "stretch" + qemuarm-stretch: + machine: "qemuarm" diff --git a/testsuite/build_test/variants_sdk.yaml b/testsuite/build_test/variants_sdk.yaml new file mode 100644 index 0000000..a38b4ff --- /dev/null +++ b/testsuite/build_test/variants_sdk.yaml @@ -0,0 +1,5 @@ +variants: !mux + qemuarm-stretch: + machine: "qemuarm" + distro: "stretch" + bitbake_cmd: "do_populate_sdk" diff --git a/testsuite/build_test/variants_wic_exclude.yaml b/testsuite/build_test/variants_wic_exclude.yaml new file mode 100644 index 0000000..392036f --- /dev/null +++ b/testsuite/build_test/variants_wic_exclude.yaml @@ -0,0 +1,4 @@ +variants: !mux + qemuamd64-stretch: + machine: "qemuamd64" + distro: "stretch" -- 2.25.1