* [PATCH v4 01/15] CI: Enable downloads dir sharing
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 02/15] CI: Add IMAGE_INSTALL override Uladzimir Bely
` (15 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Test cases performed in different build dir now can share DL_DIR.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 9d1b80f7..8178ea28 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -54,7 +54,8 @@ class CIBuilder(Test):
def configure(self, compat_arch=True, cross=None, debsrc_cache=False,
container=False, ccache=False, sstate=False, offline=False,
- gpg_pub_key=None, wic_deploy_parts=False, source_date_epoch=None, **kwargs):
+ gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
+ source_date_epoch=None, **kwargs):
# write configuration file and set bitbake_args
# can run multiple times per test case
self.check_init()
@@ -64,6 +65,9 @@ class CIBuilder(Test):
if cross is None:
cross = bool(int(self.params.get('cross', default=0)))
+ if dl_dir is None:
+ dl_dir = os.path.join(isar_root, 'downloads')
+
# get parameters from environment
distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS')
@@ -79,6 +83,7 @@ class CIBuilder(Test):
f' gpg_pub_key = {gpg_pub_key}\n'
f' wic_deploy_parts = {wic_deploy_parts}\n'
f' source_date_epoch = {source_date_epoch} \n'
+ f' dl_dir = {dl_dir}\n'
f'===================================================')
# determine bitbake_args
@@ -115,6 +120,8 @@ class CIBuilder(Test):
f.write('CCACHE_TOP_DIR = "${TOPDIR}/ccache"\n')
if source_date_epoch:
f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch)
+ if dl_dir:
+ f.write('DL_DIR = "%s"\n' % dl_dir)
# include ci_build.conf in local.conf
with open(self.build_dir + '/conf/local.conf', 'r+') as f:
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 02/15] CI: Add IMAGE_INSTALL override
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 01/15] CI: Enable downloads dir sharing Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 03/15] CI: Add developers test Uladzimir Bely
` (14 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Allow user to override the list of packages to be built.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 8178ea28..27909d29 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -55,7 +55,7 @@ class CIBuilder(Test):
def configure(self, compat_arch=True, cross=None, debsrc_cache=False,
container=False, ccache=False, sstate=False, offline=False,
gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
- source_date_epoch=None, **kwargs):
+ source_date_epoch=None, image_install=None, **kwargs):
# write configuration file and set bitbake_args
# can run multiple times per test case
self.check_init()
@@ -84,6 +84,7 @@ class CIBuilder(Test):
f' wic_deploy_parts = {wic_deploy_parts}\n'
f' source_date_epoch = {source_date_epoch} \n'
f' dl_dir = {dl_dir}\n'
+ f' image_install = {image_install}\n'
f'===================================================')
# determine bitbake_args
@@ -122,6 +123,8 @@ class CIBuilder(Test):
f.write('SOURCE_DATE_EPOCH = "%s"\n' % source_date_epoch)
if dl_dir:
f.write('DL_DIR = "%s"\n' % dl_dir)
+ if image_install is not None:
+ f.write('IMAGE_INSTALL = "%s"' % image_install)
# include ci_build.conf in local.conf
with open(self.build_dir + '/conf/local.conf', 'r+') as f:
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 03/15] CI: Add developers test
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 01/15] CI: Enable downloads dir sharing Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 02/15] CI: Add IMAGE_INSTALL override Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 04/15] CI: Remove duplicated targets Uladzimir Bely
` (13 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Developers test consist of minimal test cases for patches checking.
To ensure dev test is 'always green' include it into full CI.
Dev test can be executed using 'dev' avocado tag.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 54 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 2b6c0b29..027b965b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -17,6 +17,60 @@ try:
except path.CmdNotFoundError:
SKOPEO_AVAILABLE = False
+class DevTest(CIBaseTest):
+
+ """
+ Developer's test
+
+ :avocado: tags=dev,fast,full
+ """
+ def test_dev(self):
+ targets = [
+ 'mc:qemuamd64-bullseye:isar-image-base',
+ 'mc:qemuarm-bullseye:isar-image-base',
+ 'mc:qemuarm-bullseye:isar-image-base:do_populate_sdk',
+ 'mc:qemuarm64-bullseye:isar-image-base'
+ ]
+
+ self.init()
+ self.perform_build_test(targets, cross=True, image_install="example-raw")
+
+ def test_dev_apps(self):
+ targets = [
+ 'mc:qemuamd64-bullseye:isar-image-base',
+ 'mc:qemuarm64-bullseye:isar-image-base'
+ ]
+
+ self.init()
+ self.perform_build_test(targets)
+
+ def test_dev_rebuild(self):
+ self.init()
+ layerdir_core = self.getlayerdir('core')
+
+ dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass'
+
+ self.backupfile(dpkgbase_file)
+ with open(dpkgbase_file, 'a') as file:
+ file.write('do_fetch:append() {\n\n}')
+
+ try:
+ self.perform_build_test('mc:qemuamd64-bullseye:isar-image-base')
+ finally:
+ self.restorefile(dpkgbase_file)
+
+ def test_dev_run_amd64_bullseye(self):
+ self.init()
+ self.vm_start('amd64', 'bullseye')
+
+ def test_dev_run_arm64_bullseye(self):
+ self.init()
+ self.vm_start('arm64', 'bullseye')
+
+ def test_dev_run_arm_bullseye(self):
+ self.init()
+ self.vm_start('arm', 'bullseye')
+
class ReproTest(CIBaseTest):
"""
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 04/15] CI: Remove duplicated targets
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (2 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 03/15] CI: Add developers test Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 05/15] CI: move to avocado to 99.0 Uladzimir Bely
` (12 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Some targets are already covered by dev tests, so remove retesting.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 49 ---------------------------------------------
1 file changed, 49 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 027b965b..e43004d5 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -125,7 +125,6 @@ class CrossTest(CIBaseTest):
targets = [
'mc:qemuarm-buster:isar-image-base',
'mc:qemuarm-bullseye:isar-image-base',
- 'mc:qemuarm64-bullseye:isar-image-base',
'mc:de0-nano-soc-bullseye:isar-image-base',
'mc:stm32mp15x-buster:isar-image-base'
]
@@ -167,19 +166,6 @@ class CrossTest(CIBaseTest):
except:
self.cancel('KFAIL')
-class SdkTest(CIBaseTest):
-
- """
- In addition test SDK creation
-
- :avocado: tags=sdk,fast,full
- """
- def test_sdk(self):
- targets = ['mc:qemuarm-bullseye:isar-image-base']
-
- self.init()
- self.perform_build_test(targets, bitbake_cmd='do_populate_sdk')
-
class WicTest(CIBaseTest):
"""
@@ -218,7 +204,6 @@ class NoCrossTest(CIBaseTest):
'mc:qemui386-buster:isar-image-base',
'mc:qemui386-bullseye:isar-image-base',
'mc:qemuamd64-buster:isar-image-base',
- 'mc:qemuamd64-bullseye:isar-image-base',
'mc:qemuamd64-bullseye:isar-initramfs',
'mc:qemumipsel-buster:isar-image-base',
'mc:qemumipsel-bullseye:isar-image-base',
@@ -278,28 +263,6 @@ class NoCrossTest(CIBaseTest):
except:
self.cancel('KFAIL')
-class RebuildTest(CIBaseTest):
-
- """
- Test image rebuild
-
- :avocado: tags=rebuild,fast,full
- """
- def test_rebuild(self):
- self.init()
- layerdir_core = self.getlayerdir('core')
-
- dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass'
-
- self.backupfile(dpkgbase_file)
- with open(dpkgbase_file, 'a') as file:
- file.write('do_fetch:append() {\n\n}')
-
- try:
- self.perform_build_test('mc:qemuamd64-bullseye:isar-image-base', debsrc_cache=True)
- finally:
- self.restorefile(dpkgbase_file)
-
class ContainerImageTest(CIBaseTest):
"""
@@ -361,14 +324,6 @@ class VmBootTestFast(CIBaseTest):
self.init()
self.vm_start('arm','buster')
- def test_arm64_bullseye(self):
- self.init()
- self.vm_start('arm64','bullseye')
-
- def test_amd64_bullseye(self):
- self.init()
- self.vm_start('amd64','bullseye')
-
def test_arm_bookworm(self):
self.init()
self.vm_start('arm','bookworm')
@@ -392,10 +347,6 @@ class VmBootTestFull(CIBaseTest):
self.init()
self.vm_start('arm64','bullseye')
- def test_amd64_bullseye(self):
- self.init()
- self.vm_start('amd64','bullseye')
-
def test_i386_buster(self):
self.init()
self.vm_start('i386','buster')
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 05/15] CI: move to avocado to 99.0
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (3 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 04/15] CI: Remove duplicated targets Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 06/15] CI: fix shell coding style Uladzimir Bely
` (11 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/ci_build.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index fa7208c2..84a3e472 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -21,7 +21,7 @@ if ! command -v avocado > /dev/null; then
rm -rf /tmp/avocado_venv
virtualenv --python python3 /tmp/avocado_venv
source /tmp/avocado_venv/bin/activate
- pip install avocado-framework==96.0
+ pip install avocado-framework==99.0
fi
# Get Avocado build tests path
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 06/15] CI: fix shell coding style
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (4 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 05/15] CI: move to avocado to 99.0 Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 07/15] CI: install qemu-system when qemu testing is requested Uladzimir Bely
` (10 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Henning Schild <henning.schild@siemens.com>
Apply some suggestions from shellcheck and make sure that runs fine.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/ci_build.sh | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 84a3e472..e5f20d4f 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -20,6 +20,7 @@ if ! command -v avocado > /dev/null; then
sudo apt-get install -y virtualenv
rm -rf /tmp/avocado_venv
virtualenv --python python3 /tmp/avocado_venv
+ # shellcheck source=/dev/null
source /tmp/avocado_venv/bin/activate
pip install avocado-framework==99.0
fi
@@ -33,8 +34,8 @@ BASE_DIR=./build
# Check dependencies
DEPENDENCIES="umoci skopeo"
for prog in ${DEPENDENCIES} ; do
- if [ ! -x "$(which $prog)" ] ; then
- echo "missing $prog in PATH" >&2
+ if ! command -v "${prog}" > /dev/null; then
+ echo "missing ${prog} in PATH" >&2
fi
done
@@ -132,16 +133,16 @@ fi
mkdir -p .config/avocado
cat <<EOF > .config/avocado/avocado.conf
[datadir.paths]
-base_dir = $(realpath $BASE_DIR)/
-test_dir = $(realpath $BASE_DIR)/tests
-data_dir = $(realpath $BASE_DIR)/data
-logs_dir = $(realpath $BASE_DIR)/job-results
+base_dir = $(realpath "${BASE_DIR}")/
+test_dir = $(realpath "${BASE_DIR}")/tests
+data_dir = $(realpath "${BASE_DIR}")/data
+logs_dir = $(realpath "${BASE_DIR}")/job-results
EOF
export VIRTUAL_ENV="./"
# the real stuff starts here, trace commands from now on
set -x
-avocado $VERBOSE run "$TESTSUITE_DIR/citest.py" \
- -t $TAGS --nrunner-max-parallel-tasks=1 --disable-sysinfo \
- -p quiet=$QUIET -p cross=$CROSS_BUILD -p time_to_wait=$TIMEOUT
+avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
+ -t "${TAGS}" --nrunner-max-parallel-tasks=1 --disable-sysinfo \
+ -p quiet="${QUIET}" -p cross="${CROSS_BUILD}" -p time_to_wait="${TIMEOUT}"
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 07/15] CI: install qemu-system when qemu testing is requested
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (5 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 06/15] CI: fix shell coding style Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 08/15] testsuite: remove tests from "fast" set Uladzimir Bely
` (9 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Henning Schild <henning.schild@siemens.com>
And also change gitlab ci config to request qemu testing.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
.gitlab-ci.yml | 4 ++--
scripts/ci_build.sh | 5 +++++
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fcdefa87..6a49ce43 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,7 +27,7 @@ fast-ci:
except:
- schedules
script:
- - scripts/ci_build.sh -q -f -n
+ - scripts/ci_build.sh -q -f
full-ci:
<<: *common-build
@@ -37,7 +37,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -c -r -n;
+ scripts/ci_build.sh -q -c -r;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index e5f20d4f..85b98f8f 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -127,6 +127,11 @@ fi
if [ -n "$NORUN" ]; then
TAGS="$TAGS,-startvm"
+else
+ if [ ! -f /usr/share/doc/qemu-system/copyright ]; then
+ sudo apt-get update -qq
+ sudo apt-get install -y --no-install-recommends qemu-system ovmf
+ fi
fi
# Provide working path
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 08/15] testsuite: remove tests from "fast" set
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (6 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 07/15] CI: install qemu-system when qemu testing is requested Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 09/15] CI: Get rid of setting cross from outside Uladzimir Bely
` (8 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Henning Schild <henning.schild@siemens.com>
Ccache and Sstate are test infrastructure bits and should not be done
when we try to be "fast".
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/citest.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index e43004d5..0485dd7b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -107,7 +107,7 @@ class CcacheTest(CIBaseTest):
"""
Test rebuild speed improve with ccache
- :avocado: tags=ccache,fast,full
+ :avocado: tags=ccache,full
"""
def test_ccache_rebuild(self):
targets = ['mc:qemuamd64-bullseye:hello-isar']
@@ -300,7 +300,7 @@ class SstateTest(CIBaseTest):
"""
Test builds with artifacts taken from sstate cache
- :avocado: tags=sstate,fast,full
+ :avocado: tags=sstate,full
"""
def test_sstate(self):
image_target = 'mc:qemuamd64-bullseye:isar-image-base'
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 09/15] CI: Get rid of setting cross from outside
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (7 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 08/15] testsuite: remove tests from "fast" set Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 10/15] CI: Make cross compile to be default Uladzimir Bely
` (7 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
There is no need in setting cross compilation mode from outside
because it was already done inside test cases.
This also makes test cases more unified.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
.gitlab-ci.yml | 2 +-
scripts/ci_build.sh | 13 +++++--------
testsuite/cibuilder.py | 4 +---
3 files changed, 7 insertions(+), 12 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 6a49ce43..d1d22a6f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -37,7 +37,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -c -r;
+ scripts/ci_build.sh -q -r;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 85b98f8f..20c6d9e9 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -48,7 +48,6 @@ show_help() {
echo "Parameters:"
echo " -b, --base BASE_DIR set path to base directory. If not set,"
echo " the tests will be started in current path."
- echo " -c, --cross enable cross-compilation."
echo " -d, --debug enable debug bitbake output."
echo " -f, --fast cross build reduced set of configurations."
echo " -q, --quiet suppress verbose bitbake output."
@@ -64,7 +63,6 @@ show_help() {
}
TAGS="full"
-CROSS_BUILD="0"
QUIET="0"
TIMEOUT=300
@@ -82,17 +80,12 @@ do
BASE_DIR="$2"
shift
;;
- -c|--cross)
- CROSS_BUILD="1"
- ;;
-d|--debug)
VERBOSE="--show=app,test"
;;
-f|--fast)
# Start build for the reduced set of configurations
- # Enforce cross-compilation to speed up the build
TAGS="fast"
- CROSS_BUILD="1"
;;
-q|--quiet)
QUIET="1"
@@ -112,6 +105,10 @@ do
TIMEOUT=$2
shift
;;
+ -c|--cross)
+ # Just not to cause CI failures on legacy configs
+ echo "warning: deprecated parameter '$key'"
+ ;;
*)
echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
exit $ES_BUG
@@ -150,4 +147,4 @@ set -x
avocado ${VERBOSE} run "${TESTSUITE_DIR}/citest.py" \
-t "${TAGS}" --nrunner-max-parallel-tasks=1 --disable-sysinfo \
- -p quiet="${QUIET}" -p cross="${CROSS_BUILD}" -p time_to_wait="${TIMEOUT}"
+ -p quiet="${QUIET}" -p time_to_wait="${TIMEOUT}"
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 27909d29..a9d50d42 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -52,7 +52,7 @@ class CIBuilder(Test):
if not hasattr(self, 'build_dir'):
self.error("Broken test implementation: need to call init().")
- def configure(self, compat_arch=True, cross=None, debsrc_cache=False,
+ def configure(self, compat_arch=True, cross=False, debsrc_cache=False,
container=False, ccache=False, sstate=False, offline=False,
gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
source_date_epoch=None, image_install=None, **kwargs):
@@ -62,8 +62,6 @@ class CIBuilder(Test):
# get parameters from avocado cmdline
quiet = bool(int(self.params.get('quiet', default=0)))
- if cross is None:
- cross = bool(int(self.params.get('cross', default=0)))
if dl_dir is None:
dl_dir = os.path.join(isar_root, 'downloads')
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 10/15] CI: Make cross compile to be default
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (8 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 09/15] CI: Get rid of setting cross from outside Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 11/15] CI: Get rid of setting repro from outside Uladzimir Bely
` (6 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Cross compile is stable enough to be enabled by default.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 2 +-
testsuite/citest.py | 14 +++++++-------
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index a9d50d42..18ca4919 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -52,7 +52,7 @@ class CIBuilder(Test):
if not hasattr(self, 'build_dir'):
self.error("Broken test implementation: need to call init().")
- def configure(self, compat_arch=True, cross=False, debsrc_cache=False,
+ def configure(self, compat_arch=True, cross=True, debsrc_cache=False,
container=False, ccache=False, sstate=False, offline=False,
gpg_pub_key=None, wic_deploy_parts=False, dl_dir=None,
source_date_epoch=None, image_install=None, **kwargs):
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 0485dd7b..f5cead59 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -33,7 +33,7 @@ class DevTest(CIBaseTest):
]
self.init()
- self.perform_build_test(targets, cross=True, image_install="example-raw")
+ self.perform_build_test(targets, image_install="example-raw")
def test_dev_apps(self):
targets = [
@@ -86,7 +86,7 @@ class ReproTest(CIBaseTest):
self.init()
try:
- self.perform_repro_test(targets, signed=True, cross=True)
+ self.perform_repro_test(targets, signed=True)
finally:
self.move_in_build_dir('tmp', 'tmp_repro_signed')
@@ -98,7 +98,7 @@ class ReproTest(CIBaseTest):
self.init()
try:
- self.perform_repro_test(targets)
+ self.perform_repro_test(targets, cross=False)
finally:
self.move_in_build_dir('tmp', 'tmp_repro_unsigned')
@@ -130,7 +130,7 @@ class CrossTest(CIBaseTest):
]
self.init()
- self.perform_build_test(targets, cross=True, debsrc_cache=True)
+ self.perform_build_test(targets, debsrc_cache=True)
def test_cross_rpi(self):
targets = [
@@ -139,7 +139,7 @@ class CrossTest(CIBaseTest):
self.init()
try:
- self.perform_build_test(targets, cross=True, debsrc_cache=True)
+ self.perform_build_test(targets, debsrc_cache=True)
except:
self.cancel('KFAIL')
@@ -150,7 +150,7 @@ class CrossTest(CIBaseTest):
self.init()
try:
- self.perform_build_test(targets, cross=True)
+ self.perform_build_test(targets)
except:
self.cancel('KFAIL')
@@ -162,7 +162,7 @@ class CrossTest(CIBaseTest):
self.init()
try:
- self.perform_build_test(targets, cross=True)
+ self.perform_build_test(targets)
except:
self.cancel('KFAIL')
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 11/15] CI: Get rid of setting repro from outside
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (9 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 10/15] CI: Make cross compile to be default Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:33 ` [PATCH v4 12/15] CI: Allow example module checking disabling Uladzimir Bely
` (5 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
To make test cases more unified do not allow to change repro test
execution. This test case will always run for "full" now.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
.gitlab-ci.yml | 2 +-
scripts/ci_build.sh | 15 +--------------
2 files changed, 2 insertions(+), 15 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d1d22a6f..d1e82492 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -37,7 +37,7 @@ full-ci:
- PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
- if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
- scripts/ci_build.sh -q -r;
+ scripts/ci_build.sh -q;
fi
cache:
key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 20c6d9e9..5350ad85 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -51,7 +51,6 @@ show_help() {
echo " -d, --debug enable debug bitbake output."
echo " -f, --fast cross build reduced set of configurations."
echo " -q, --quiet suppress verbose bitbake output."
- echo " -r, --repro enable use of cached base repository."
echo " -n, --norun do not execute QEMU run tests."
echo " -t, --timeout SEC specify time in seconds to wait before stop QEMU."
echo " The default is: 300"
@@ -90,14 +89,6 @@ do
-q|--quiet)
QUIET="1"
;;
- -r|--repro)
- REPRO_BUILD="1"
- # This switch is deprecated, just here to not cause failing CI on
- # legacy configs
- case "$2" in
- -s|--sign) shift ;;
- esac
- ;;
-n|--norun)
NORUN="1"
;;
@@ -105,7 +96,7 @@ do
TIMEOUT=$2
shift
;;
- -c|--cross)
+ -c|--cross|-r|--repro|-s|--sign)
# Just not to cause CI failures on legacy configs
echo "warning: deprecated parameter '$key'"
;;
@@ -118,10 +109,6 @@ do
shift
done
-if [ -z "$REPRO_BUILD" ]; then
- TAGS="$TAGS,-repro"
-fi
-
if [ -n "$NORUN" ]; then
TAGS="$TAGS,-startvm"
else
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 12/15] CI: Allow example module checking disabling
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (10 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 11/15] CI: Get rid of setting repro from outside Uladzimir Bely
@ 2023-02-06 12:33 ` Uladzimir Bely
2023-02-06 12:34 ` [PATCH v4 13/15] testsuite: Fix terminal broken after run test executed Uladzimir Bely
` (4 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:33 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
If the target do not have example-module package compiled it can fail
on startvm test currently. Provide internal testsuite API for skipping
this test.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibuilder.py | 6 ++++--
testsuite/citest.py | 2 +-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 18ca4919..c6a1eb67 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -209,7 +209,8 @@ class CIBuilder(Test):
return env['LAYERDIR_' + layer].strip('"')
- def vm_start(self, arch='amd64', distro='buster', enforce_pcbios=False):
+ def vm_start(self, arch='amd64', distro='buster', enforce_pcbios=False,
+ skip_modulecheck=False):
time_to_wait = self.params.get('time_to_wait', default=60)
self.log.info('===================================================')
@@ -291,7 +292,8 @@ class CIBuilder(Test):
if os.path.exists(output_file) and os.path.getsize(output_file) > 0:
with open(output_file, "rb") as f1:
data = f1.read()
- if module_output in data and login_prompt in data:
+ if (module_output in data or skip_modulecheck) \
+ and login_prompt in data:
if resize_output:
if resize_output in data:
return
diff --git a/testsuite/citest.py b/testsuite/citest.py
index f5cead59..cdbede05 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -69,7 +69,7 @@ class DevTest(CIBaseTest):
def test_dev_run_arm_bullseye(self):
self.init()
- self.vm_start('arm', 'bullseye')
+ self.vm_start('arm', 'bullseye', skip_modulecheck=True)
class ReproTest(CIBaseTest):
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 13/15] testsuite: Fix terminal broken after run test executed
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (11 preceding siblings ...)
2023-02-06 12:33 ` [PATCH v4 12/15] CI: Allow example module checking disabling Uladzimir Bely
@ 2023-02-06 12:34 ` Uladzimir Bely
2023-02-06 12:34 ` [PATCH v4 14/15] CI: Update testsuite documentation Uladzimir Bely
` (3 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:34 UTC (permalink / raw)
To: isar-users
If some of run tests executed from interactive terminal by hand,
user input goes broken after VM powered off. Commands user enters
become invisible and terminal can be fixed only by executing
either 'reset' or 'stty echo' commands.
Fix this by specifying 'stdin' value for qemu subprocess.
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
testsuite/cibuilder.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index c6a1eb67..72522f4d 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -262,8 +262,9 @@ class CIBuilder(Test):
timeout = time.time() + int(time_to_wait)
- p1 = subprocess.Popen(cmdline, stdout=subprocess.PIPE,
- stderr=subprocess.PIPE, universal_newlines=True)
+ p1 = subprocess.Popen(cmdline,
+ stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+ universal_newlines=True)
try:
poller = select.poll()
poller.register(p1.stdout, select.POLLIN)
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 14/15] CI: Update testsuite documentation
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (12 preceding siblings ...)
2023-02-06 12:34 ` [PATCH v4 13/15] testsuite: Fix terminal broken after run test executed Uladzimir Bely
@ 2023-02-06 12:34 ` Uladzimir Bely
2023-02-06 12:34 ` [PATCH v4 15/15] CI: Add single target test case Uladzimir Bely
` (2 subsequent siblings)
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:34 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/README.md | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/testsuite/README.md b/testsuite/README.md
index b83a8f98..90ba5c3d 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -4,34 +4,42 @@ The framework could be installed by using standard HOWTO:
https://github.com/avocado-framework/avocado#installing-with-standard-python-tools
-## For Debian (tested on Debian 10.x)
+## For Debian (tested on Debian 11.x)
- $ sudo dpkg -i avocado_91.0_all.deb
+```
+$ pip install avocado-framework==99.0
+```
# Run test
+## Quick developers test
+
+```
+$ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1
+```
+
## Fast build test
```
-$ avocado run build_test.py -t fast -p quiet=1 -p cross=1
+$ avocado run ../testsuite/citest.py -t fast --nrunner-max-parallel-tasks=1 -p quiet=1
```
## Full build test
```
-$ avocado run build_test.py -t full -p quiet=1
+$ avocado run ../testsuite/citest.py -t full --nrunner-max-parallel-tasks=1 -p quiet=1
```
## Fast boot test
```
-$ avocado run vm_boot_test.py -t fast -p build_dir="$BUILDDIR" -p time_to_wait=300
+$ avocado run ../testsuite/citest.py -t startvm,fast -p time_to_wait=300
```
## Full boot test
```
-$ avocado run vm_boot_test.py -t full -p build_dir="$BUILDDIR" -p time_to_wait=300
+$ avocado run ../testsuite/citest.py -t startvm,full -p time_to_wait=300
```
# Other
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH v4 15/15] CI: Add single target test case
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (13 preceding siblings ...)
2023-02-06 12:34 ` [PATCH v4 14/15] CI: Update testsuite documentation Uladzimir Bely
@ 2023-02-06 12:34 ` Uladzimir Bely
2023-02-09 8:39 ` [PATCH v4 00/15] Add developers test Uladzimir Bely
2023-02-14 5:43 ` Uladzimir Bely
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-06 12:34 UTC (permalink / raw)
To: isar-users
From: Anton Mikanovich <amikan@ilbers.de>
Add special test case for single target build and run testing.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/README.md | 6 ++++++
testsuite/citest.py | 22 ++++++++++++++++++++++
2 files changed, 28 insertions(+)
diff --git a/testsuite/README.md b/testsuite/README.md
index 90ba5c3d..4f2a8da8 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -18,6 +18,12 @@ $ pip install avocado-framework==99.0
$ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1
```
+## Single target test
+
+```
+$ avocado run ../testsuite/citest.py -t single --nrunner-max-parallel-tasks=1 -p machine=qemuamd64 -p distro=bullseye
+```
+
## Fast build test
```
diff --git a/testsuite/citest.py b/testsuite/citest.py
index cdbede05..975cb064 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -309,6 +309,28 @@ class SstateTest(CIBaseTest):
self.init('build-sstate')
self.perform_sstate_test(image_target, package_target)
+class SingleTest(CIBaseTest):
+
+ """
+ Single test for selected target
+
+ :avocado: tags=single
+ """
+ def test_single_build(self):
+ self.init()
+ machine = self.params.get('machine', default='qemuamd64')
+ distro = self.params.get('distro', default='bullseye')
+ image = self.params.get('image', default='isar-image-base')
+
+ self.perform_build_test('mc:%s-%s:%s' % (machine, distro, image))
+
+ def test_single_run(self):
+ self.init()
+ machine = self.params.get('machine', default='qemuamd64')
+ distro = self.params.get('distro', default='bullseye')
+
+ self.vm_start(machine.removeprefix('qemu'), distro)
+
class VmBootTestFast(CIBaseTest):
"""
--
2.20.1
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v4 00/15] Add developers test
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (14 preceding siblings ...)
2023-02-06 12:34 ` [PATCH v4 15/15] CI: Add single target test case Uladzimir Bely
@ 2023-02-09 8:39 ` Uladzimir Bely
2023-02-09 15:46 ` Henning Schild
2023-02-14 5:43 ` Uladzimir Bely
16 siblings, 1 reply; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-09 8:39 UTC (permalink / raw)
To: isar-users, Schild, Henning
In the email from Monday, 6 February 2023 15:33:47 +03 user Uladzimir Bely
wrote:
> Add developers test
Hi all.
Patchset passes our internal CI (Jenkins fast / full, also internal Gitlab).
We are planning to merge this at ~Tue, Feb 14, if there are no any objections.
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v4 00/15] Add developers test
2023-02-09 8:39 ` [PATCH v4 00/15] Add developers test Uladzimir Bely
@ 2023-02-09 15:46 ` Henning Schild
2023-02-10 2:42 ` Uladzimir Bely
0 siblings, 1 reply; 28+ messages in thread
From: Henning Schild @ 2023-02-09 15:46 UTC (permalink / raw)
To: Uladzimir Bely; +Cc: isar-users
Am Thu, 09 Feb 2023 11:39:24 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:
> In the email from Monday, 6 February 2023 15:33:47 +03 user Uladzimir
> Bely wrote:
> > Add developers test
>
> Hi all.
>
> Patchset passes our internal CI (Jenkins fast / full, also internal
> Gitlab).
i tested something called "fe689990a9f42242401e00af2e2c7699dde862be"
that i once saw on ubely/devel
that went red but i did not yet look why.
Is that what we are talking about? For this one it would really help if
i could pull it somewhere .. long series i just have to push to a
mirror.
> We are planning to merge this at ~Tue, Feb 14, if there are no any
> objections.
lovely date for patches instead of chocolates and flowers
Henning
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v4 00/15] Add developers test
2023-02-09 15:46 ` Henning Schild
@ 2023-02-10 2:42 ` Uladzimir Bely
2023-02-10 9:44 ` Henning Schild
0 siblings, 1 reply; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-10 2:42 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users
In the email from Thursday, 9 February 2023 18:46:02 +03 user Henning Schild
wrote:
> Am Thu, 09 Feb 2023 11:39:24 +0300
>
> schrieb Uladzimir Bely <ubely@ilbers.de>:
> > In the email from Monday, 6 February 2023 15:33:47 +03 user Uladzimir
> >
> > Bely wrote:
> > > Add developers test
> >
> > Hi all.
> >
> > Patchset passes our internal CI (Jenkins fast / full, also internal
> > Gitlab).
>
> i tested something called "fe689990a9f42242401e00af2e2c7699dde862be"
> that i once saw on ubely/devel
>
Yes, that was a combination of several patchsets
- (merged) fix cyclic dep on crossbuild-essential-riscv64
- (merged) Revert "dpkg: sbuild allows extra arguments via
DPKG_SBUILD_EXTRA_ARGS"
- (merged) initramfs: Add missing TASK_USE_SUDO
- (v2) deb-dl-dir: Fallback to copying when import-export
- (v4) Add developers tests
- (pre-v2) Support running custom commands in VM
It passed full/fast Jenkins CI, but failed in Gitlab ("Operation not
permitted" for ssh private key, so new vm_run tests failed. Fixed in v2)
> that went red but i did not yet look why.
>
Probably, you also faced "Operation not permitted". It seems in case of Gitlab
we can't do any modifications on cloned repo itself, including "chmod 600
testsute/keys/ssh/id_rsa" that is required for non-interactive ssh access
working.
Unfortunally, Git doesn't allow to store files with permissions different than
0755 and 0644.
> Is that what we are talking about? For this one it would really help if
> i could pull it somewhere .. long series i just have to push to a
> mirror.
To test, you can pick `948af8e7` (I've just pushed it to ubely/devel_runtest)
that includes both "(v4) Add developers tests" and "(v2) Support running
custom commands in VM". Or take just `30dadcb` from the same branch, that is
only "(v4) Add developers tests".
>
> > We are planning to merge this at ~Tue, Feb 14, if there are no any
> > objections.
>
> lovely date for patches instead of chocolates and flowers
>
> Henning
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v4 00/15] Add developers test
2023-02-10 2:42 ` Uladzimir Bely
@ 2023-02-10 9:44 ` Henning Schild
2023-02-10 9:51 ` Uladzimir Bely
0 siblings, 1 reply; 28+ messages in thread
From: Henning Schild @ 2023-02-10 9:44 UTC (permalink / raw)
To: Uladzimir Bely; +Cc: isar-users
Too much to read. Give me a branch on github. I trigger a pipeline and
give you the result. I just push buttons and need to know/understand
nothing ... that is my offer.
If i have to know and understand ... i have no time and can not help at
the moment.
Henning
Am Fri, 10 Feb 2023 05:42:40 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:
> In the email from Thursday, 9 February 2023 18:46:02 +03 user Henning
> Schild wrote:
> > Am Thu, 09 Feb 2023 11:39:24 +0300
> >
> > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > In the email from Monday, 6 February 2023 15:33:47 +03 user
> > > Uladzimir
> > >
> > > Bely wrote:
> > > > Add developers test
> > >
> > > Hi all.
> > >
> > > Patchset passes our internal CI (Jenkins fast / full, also
> > > internal Gitlab).
> >
> > i tested something called "fe689990a9f42242401e00af2e2c7699dde862be"
> > that i once saw on ubely/devel
> >
>
> Yes, that was a combination of several patchsets
> - (merged) fix cyclic dep on crossbuild-essential-riscv64
> - (merged) Revert "dpkg: sbuild allows extra arguments via
> DPKG_SBUILD_EXTRA_ARGS"
> - (merged) initramfs: Add missing TASK_USE_SUDO
> - (v2) deb-dl-dir: Fallback to copying when import-export
> - (v4) Add developers tests
> - (pre-v2) Support running custom commands in VM
>
> It passed full/fast Jenkins CI, but failed in Gitlab ("Operation not
> permitted" for ssh private key, so new vm_run tests failed. Fixed in
> v2)
>
> > that went red but i did not yet look why.
> >
>
> Probably, you also faced "Operation not permitted". It seems in case
> of Gitlab we can't do any modifications on cloned repo itself,
> including "chmod 600 testsute/keys/ssh/id_rsa" that is required for
> non-interactive ssh access working.
>
> Unfortunally, Git doesn't allow to store files with permissions
> different than 0755 and 0644.
>
> > Is that what we are talking about? For this one it would really
> > help if i could pull it somewhere .. long series i just have to
> > push to a mirror.
>
> To test, you can pick `948af8e7` (I've just pushed it to
> ubely/devel_runtest) that includes both "(v4) Add developers tests"
> and "(v2) Support running custom commands in VM". Or take just
> `30dadcb` from the same branch, that is only "(v4) Add developers
> tests".
>
> >
> > > We are planning to merge this at ~Tue, Feb 14, if there are no any
> > > objections.
> >
> > lovely date for patches instead of chocolates and flowers
> >
> > Henning
>
>
>
>
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v4 00/15] Add developers test
2023-02-10 9:44 ` Henning Schild
@ 2023-02-10 9:51 ` Uladzimir Bely
2023-02-10 10:19 ` Henning Schild
0 siblings, 1 reply; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-10 9:51 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users
In the email from Friday, 10 February 2023 12:44:16 +03 user Henning Schild
wrote:
> Too much to read. Give me a branch on github. I trigger a pipeline and
> give you the result. I just push buttons and need to know/understand
> nothing ... that is my offer.
ubely/devel_runtest (948af8e7) - includes both "(v4) Add developers tests" and
"(v2) Support running custom commands in VM" patchsets
>
> If i have to know and understand ... i have no time and can not help at
> the moment.
>
> Henning
>
> Am Fri, 10 Feb 2023 05:42:40 +0300
>
> schrieb Uladzimir Bely <ubely@ilbers.de>:
> > In the email from Thursday, 9 February 2023 18:46:02 +03 user Henning
> >
> > Schild wrote:
> > > Am Thu, 09 Feb 2023 11:39:24 +0300
> > >
> > > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > > In the email from Monday, 6 February 2023 15:33:47 +03 user
> > > > Uladzimir
> > > >
> > > > Bely wrote:
> > > > > Add developers test
> > > >
> > > > Hi all.
> > > >
> > > > Patchset passes our internal CI (Jenkins fast / full, also
> > > > internal Gitlab).
> > >
> > > i tested something called "fe689990a9f42242401e00af2e2c7699dde862be"
> > > that i once saw on ubely/devel
> >
> > Yes, that was a combination of several patchsets
> > - (merged) fix cyclic dep on crossbuild-essential-riscv64
> > - (merged) Revert "dpkg: sbuild allows extra arguments via
> > DPKG_SBUILD_EXTRA_ARGS"
> > - (merged) initramfs: Add missing TASK_USE_SUDO
> > - (v2) deb-dl-dir: Fallback to copying when import-export
> > - (v4) Add developers tests
> > - (pre-v2) Support running custom commands in VM
> >
> > It passed full/fast Jenkins CI, but failed in Gitlab ("Operation not
> > permitted" for ssh private key, so new vm_run tests failed. Fixed in
> > v2)
> >
> > > that went red but i did not yet look why.
> >
> > Probably, you also faced "Operation not permitted". It seems in case
> > of Gitlab we can't do any modifications on cloned repo itself,
> > including "chmod 600 testsute/keys/ssh/id_rsa" that is required for
> > non-interactive ssh access working.
> >
> > Unfortunally, Git doesn't allow to store files with permissions
> > different than 0755 and 0644.
> >
> > > Is that what we are talking about? For this one it would really
> > > help if i could pull it somewhere .. long series i just have to
> > > push to a mirror.
> >
> > To test, you can pick `948af8e7` (I've just pushed it to
> > ubely/devel_runtest) that includes both "(v4) Add developers tests"
> > and "(v2) Support running custom commands in VM". Or take just
> > `30dadcb` from the same branch, that is only "(v4) Add developers
> > tests".
> >
> > > > We are planning to merge this at ~Tue, Feb 14, if there are no any
> > > > objections.
> > >
> > > lovely date for patches instead of chocolates and flowers
> > >
> > > Henning
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v4 00/15] Add developers test
2023-02-10 9:51 ` Uladzimir Bely
@ 2023-02-10 10:19 ` Henning Schild
2023-02-10 17:13 ` Baurzhan Ismagulov
2023-02-10 17:36 ` Henning Schild
0 siblings, 2 replies; 28+ messages in thread
From: Henning Schild @ 2023-02-10 10:19 UTC (permalink / raw)
To: Uladzimir Bely; +Cc: isar-users
Am Fri, 10 Feb 2023 12:51:25 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:
> In the email from Friday, 10 February 2023 12:44:16 +03 user Henning
> Schild wrote:
> > Too much to read. Give me a branch on github. I trigger a pipeline
> > and give you the result. I just push buttons and need to
> > know/understand nothing ... that is my offer.
>
> ubely/devel_runtest (948af8e7) - includes both "(v4) Add developers
> tests" and "(v2) Support running custom commands in VM" patchsets
Your pipeline is running.
Henning
> >
> > If i have to know and understand ... i have no time and can not
> > help at the moment.
> >
> > Henning
> >
> > Am Fri, 10 Feb 2023 05:42:40 +0300
> >
> > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > In the email from Thursday, 9 February 2023 18:46:02 +03 user
> > > Henning
> > >
> > > Schild wrote:
> > > > Am Thu, 09 Feb 2023 11:39:24 +0300
> > > >
> > > > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > > > In the email from Monday, 6 February 2023 15:33:47 +03 user
> > > > > Uladzimir
> > > > >
> > > > > Bely wrote:
> > > > > > Add developers test
> > > > >
> > > > > Hi all.
> > > > >
> > > > > Patchset passes our internal CI (Jenkins fast / full, also
> > > > > internal Gitlab).
> > > >
> > > > i tested something called
> > > > "fe689990a9f42242401e00af2e2c7699dde862be" that i once saw on
> > > > ubely/devel
> > >
> > > Yes, that was a combination of several patchsets
> > > - (merged) fix cyclic dep on crossbuild-essential-riscv64
> > > - (merged) Revert "dpkg: sbuild allows extra arguments via
> > > DPKG_SBUILD_EXTRA_ARGS"
> > > - (merged) initramfs: Add missing TASK_USE_SUDO
> > > - (v2) deb-dl-dir: Fallback to copying when import-export
> > > - (v4) Add developers tests
> > > - (pre-v2) Support running custom commands in VM
> > >
> > > It passed full/fast Jenkins CI, but failed in Gitlab ("Operation
> > > not permitted" for ssh private key, so new vm_run tests failed.
> > > Fixed in v2)
> > >
> > > > that went red but i did not yet look why.
> > >
> > > Probably, you also faced "Operation not permitted". It seems in
> > > case of Gitlab we can't do any modifications on cloned repo
> > > itself, including "chmod 600 testsute/keys/ssh/id_rsa" that is
> > > required for non-interactive ssh access working.
> > >
> > > Unfortunally, Git doesn't allow to store files with permissions
> > > different than 0755 and 0644.
> > >
> > > > Is that what we are talking about? For this one it would really
> > > > help if i could pull it somewhere .. long series i just have to
> > > > push to a mirror.
> > >
> > > To test, you can pick `948af8e7` (I've just pushed it to
> > > ubely/devel_runtest) that includes both "(v4) Add developers
> > > tests" and "(v2) Support running custom commands in VM". Or take
> > > just `30dadcb` from the same branch, that is only "(v4) Add
> > > developers tests".
> > >
> > > > > We are planning to merge this at ~Tue, Feb 14, if there are
> > > > > no any objections.
> > > >
> > > > lovely date for patches instead of chocolates and flowers
> > > >
> > > > Henning
>
>
>
>
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH v4 00/15] Add developers test
2023-02-10 10:19 ` Henning Schild
@ 2023-02-10 17:13 ` Baurzhan Ismagulov
2023-02-10 19:54 ` Henning Schild
2023-02-10 17:36 ` Henning Schild
1 sibling, 1 reply; 28+ messages in thread
From: Baurzhan Ismagulov @ 2023-02-10 17:13 UTC (permalink / raw)
To: isar-users; +Cc: Henning Schild
On Fri, Feb 10, 2023 at 11:19:57AM +0100, Henning Schild wrote:
> > > Too much to read. Give me a branch on github. I trigger a pipeline
> > > and give you the result. I just push buttons and need to
> > > know/understand nothing ... that is my offer.
> >
> > ubely/devel_runtest (948af8e7) - includes both "(v4) Add developers
> > tests" and "(v2) Support running custom commands in VM" patchsets
>
> Your pipeline is running.
This patchset drops ccache and sstate testcases from "fast". The next step is
to add the necessary coverage to "dev" and switch to it by default. You also
suggested to drop sdk, containerbuild, containersdk -- we will consider it in
this next step.
With kind regards,
Baurzhan
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v4 00/15] Add developers test
2023-02-10 17:13 ` Baurzhan Ismagulov
@ 2023-02-10 19:54 ` Henning Schild
2023-02-10 20:04 ` Henning Schild
2023-02-12 17:16 ` Baurzhan Ismagulov
0 siblings, 2 replies; 28+ messages in thread
From: Henning Schild @ 2023-02-10 19:54 UTC (permalink / raw)
To: Baurzhan Ismagulov; +Cc: isar-users
Am Fri, 10 Feb 2023 18:13:40 +0100
schrieb Baurzhan Ismagulov <ibr@radix50.net>:
> On Fri, Feb 10, 2023 at 11:19:57AM +0100, Henning Schild wrote:
> > > > Too much to read. Give me a branch on github. I trigger a
> > > > pipeline and give you the result. I just push buttons and need
> > > > to know/understand nothing ... that is my offer.
> > >
> > > ubely/devel_runtest (948af8e7) - includes both "(v4) Add
> > > developers tests" and "(v2) Support running custom commands in
> > > VM" patchsets
> >
> > Your pipeline is running.
>
> This patchset drops ccache and sstate testcases from "fast". The next
> step is to add the necessary coverage to "dev" and switch to it by
> default. You also suggested to drop sdk, containerbuild, containersdk
> -- we will consider it in this next step.
I did not even look. I just triggered and delivered the result.
We will have to see how we get our both CI results public where
triggers probably have to stay manual.
I think ilbers is much better at the triggers for "trusted
contributors" at the moment and maybe ilbers will also be faster on the
public view.
Getting this done in Siemens might be a longer story. But we will keep
an eye on "gitlab-ci" doing what "ilbers"/"jenkins" does to get the
same output even if we can not publish or allow external triggers.
This is all going in the right direction, now at least we have the same
results and the duration in a shape that Siemens infra allows.
Henning
> With kind regards,
> Baurzhan
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v4 00/15] Add developers test
2023-02-10 19:54 ` Henning Schild
@ 2023-02-10 20:04 ` Henning Schild
2023-02-12 17:16 ` Baurzhan Ismagulov
1 sibling, 0 replies; 28+ messages in thread
From: Henning Schild @ 2023-02-10 20:04 UTC (permalink / raw)
To: Baurzhan Ismagulov; +Cc: isar-users
Am Fri, 10 Feb 2023 20:54:14 +0100
schrieb Henning Schild <henning.schild@siemens.com>:
> Am Fri, 10 Feb 2023 18:13:40 +0100
> schrieb Baurzhan Ismagulov <ibr@radix50.net>:
>
> > On Fri, Feb 10, 2023 at 11:19:57AM +0100, Henning Schild wrote:
> > > > > Too much to read. Give me a branch on github. I trigger a
> > > > > pipeline and give you the result. I just push buttons and need
> > > > > to know/understand nothing ... that is my offer.
> > > >
> > > > ubely/devel_runtest (948af8e7) - includes both "(v4) Add
> > > > developers tests" and "(v2) Support running custom commands in
> > > > VM" patchsets
> > >
> > > Your pipeline is running.
> >
> > This patchset drops ccache and sstate testcases from "fast". The
> > next step is to add the necessary coverage to "dev" and switch to
> > it by default. You also suggested to drop sdk, containerbuild,
> > containersdk -- we will consider it in this next step.
>
> I did not even look. I just triggered and delivered the result.
>
> We will have to see how we get our both CI results public where
> triggers probably have to stay manual.
>
> I think ilbers is much better at the triggers for "trusted
> contributors" at the moment and maybe ilbers will also be faster on
> the public view.
>
> Getting this done in Siemens might be a longer story. But we will keep
> an eye on "gitlab-ci" doing what "ilbers"/"jenkins" does to get the
> same output even if we can not publish or allow external triggers.
>
> This is all going in the right direction, now at least we have the
> same results and the duration in a shape that Siemens infra allows.
The infra we currently run on in Siemens might never allow "external
triggers" or "sharing reports" unfortunately. And getting/maintaining
infra for public use might be complicated as well.
So i think the "ilbers" does it in public and Siemens does it
internally as well ... might just be what we get.
But that does not seem too bad, because others might also have a hard
time to "publish" and we can learn from that limitation to help
"others".
Henning
> Henning
>
> > With kind regards,
> > Baurzhan
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v4 00/15] Add developers test
2023-02-10 19:54 ` Henning Schild
2023-02-10 20:04 ` Henning Schild
@ 2023-02-12 17:16 ` Baurzhan Ismagulov
1 sibling, 0 replies; 28+ messages in thread
From: Baurzhan Ismagulov @ 2023-02-12 17:16 UTC (permalink / raw)
To: isar-users; +Cc: Henning Schild
On Fri, Feb 10, 2023 at 08:54:14PM +0100, Henning Schild wrote:
> I think ilbers is much better at the triggers for "trusted
> contributors" at the moment and maybe ilbers will also be faster on the
> public view.
We've tested reporting job status on github and could hopefully deploy that
soon. We've also evaluated job log publishing, this looks possible as well.
> This is all going in the right direction, now at least we have the same
> results and the duration in a shape that Siemens infra allows.
Thanks for the feedback, good to know.
With kind regards,
Baurzhan
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v4 00/15] Add developers test
2023-02-10 10:19 ` Henning Schild
2023-02-10 17:13 ` Baurzhan Ismagulov
@ 2023-02-10 17:36 ` Henning Schild
1 sibling, 0 replies; 28+ messages in thread
From: Henning Schild @ 2023-02-10 17:36 UTC (permalink / raw)
To: Uladzimir Bely; +Cc: isar-users
Am Fri, 10 Feb 2023 11:19:57 +0100
schrieb Henning Schild <henning.schild@siemens.com>:
> Am Fri, 10 Feb 2023 12:51:25 +0300
> schrieb Uladzimir Bely <ubely@ilbers.de>:
>
> > In the email from Friday, 10 February 2023 12:44:16 +03 user Henning
> > Schild wrote:
> > > Too much to read. Give me a branch on github. I trigger a pipeline
> > > and give you the result. I just push buttons and need to
> > > know/understand nothing ... that is my offer.
> >
> > ubely/devel_runtest (948af8e7) - includes both "(v4) Add developers
> > tests" and "(v2) Support running custom commands in VM" patchsets
>
> Your pipeline is running.
Your pipeline is green.
All tests passed, reach out if you need the reports. Will not share
them in public.
Henning
> Henning
>
> > >
> > > If i have to know and understand ... i have no time and can not
> > > help at the moment.
> > >
> > > Henning
> > >
> > > Am Fri, 10 Feb 2023 05:42:40 +0300
> > >
> > > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > > In the email from Thursday, 9 February 2023 18:46:02 +03 user
> > > > Henning
> > > >
> > > > Schild wrote:
> > > > > Am Thu, 09 Feb 2023 11:39:24 +0300
> > > > >
> > > > > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > > > > In the email from Monday, 6 February 2023 15:33:47 +03 user
> > > > > > Uladzimir
> > > > > >
> > > > > > Bely wrote:
> > > > > > > Add developers test
> > > > > >
> > > > > > Hi all.
> > > > > >
> > > > > > Patchset passes our internal CI (Jenkins fast / full, also
> > > > > > internal Gitlab).
> > > > >
> > > > > i tested something called
> > > > > "fe689990a9f42242401e00af2e2c7699dde862be" that i once saw on
> > > > > ubely/devel
> > > >
> > > > Yes, that was a combination of several patchsets
> > > > - (merged) fix cyclic dep on crossbuild-essential-riscv64
> > > > - (merged) Revert "dpkg: sbuild allows extra arguments via
> > > > DPKG_SBUILD_EXTRA_ARGS"
> > > > - (merged) initramfs: Add missing TASK_USE_SUDO
> > > > - (v2) deb-dl-dir: Fallback to copying when import-export
> > > > - (v4) Add developers tests
> > > > - (pre-v2) Support running custom commands in VM
> > > >
> > > > It passed full/fast Jenkins CI, but failed in Gitlab ("Operation
> > > > not permitted" for ssh private key, so new vm_run tests failed.
> > > > Fixed in v2)
> > > >
> > > > > that went red but i did not yet look why.
> > > >
> > > > Probably, you also faced "Operation not permitted". It seems in
> > > > case of Gitlab we can't do any modifications on cloned repo
> > > > itself, including "chmod 600 testsute/keys/ssh/id_rsa" that is
> > > > required for non-interactive ssh access working.
> > > >
> > > > Unfortunally, Git doesn't allow to store files with permissions
> > > > different than 0755 and 0644.
> > > >
> > > > > Is that what we are talking about? For this one it would
> > > > > really help if i could pull it somewhere .. long series i
> > > > > just have to push to a mirror.
> > > >
> > > > To test, you can pick `948af8e7` (I've just pushed it to
> > > > ubely/devel_runtest) that includes both "(v4) Add developers
> > > > tests" and "(v2) Support running custom commands in VM". Or take
> > > > just `30dadcb` from the same branch, that is only "(v4) Add
> > > > developers tests".
> > > >
> > > > > > We are planning to merge this at ~Tue, Feb 14, if there are
> > > > > > no any objections.
> > > > >
> > > > > lovely date for patches instead of chocolates and flowers
> > > > >
> > > > > Henning
> >
> >
> >
> >
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v4 00/15] Add developers test
2023-02-06 12:33 [PATCH v4 00/15] Add developers test Uladzimir Bely
` (15 preceding siblings ...)
2023-02-09 8:39 ` [PATCH v4 00/15] Add developers test Uladzimir Bely
@ 2023-02-14 5:43 ` Uladzimir Bely
16 siblings, 0 replies; 28+ messages in thread
From: Uladzimir Bely @ 2023-02-14 5:43 UTC (permalink / raw)
To: isar-users
In the email from Monday, 6 February 2023 15:33:47 +03 user Uladzimir Bely
wrote:
> Add separate quick 'developers' test for fast checking new patches.
> It will test only general Isar logic on basic targets without dealing with
> all the functionality.
> To execute dev test run avocado testsuite like:
>
> $ avocado run ../testsuite/citest.py -t dev --nrunner-max-parallel-tasks=1
>
> Estimated execution time for dev test is ~1.5h.
>
> Changes since v3:
> - More correct check of `qemu-system` package installed in p7.
> - Don't install `qemu-system-gui` and deps in p7 (use --no-recommends).
>
> Changes since v2:
> - Rebase on next (with bitbake 2.0).
> - Don't fail on deprecated params, show warning if used.
>
> Changes sinse v1:
> - Rebase on next.
> - Integrate some changes from other patchsets.
> - Remove ci_build.sh params.
> - Improve testsuite API.
> - Improve testsuite readme.
> - Add single target test case.
>
> Anton Mikanovich (10):
> CI: Enable downloads dir sharing
> CI: Add IMAGE_INSTALL override
> CI: Add developers test
> CI: Remove duplicated targets
> CI: Get rid of setting cross from outside
> CI: Make cross compile to be default
> CI: Get rid of setting repro from outside
> CI: Allow example module checking disabling
> CI: Update testsuite documentation
> CI: Add single target test case
>
> Henning Schild (4):
> CI: move to avocado to 99.0
> CI: fix shell coding style
> CI: install qemu-system when qemu testing is requested
> testsuite: remove tests from "fast" set
>
> Uladzimir Bely (1):
> testsuite: Fix terminal broken after run test executed
>
> .gitlab-ci.yml | 4 +-
> scripts/ci_build.sh | 50 ++++++---------
> testsuite/README.md | 26 ++++++--
> testsuite/cibuilder.py | 27 +++++---
> testsuite/citest.py | 141 ++++++++++++++++++++++++-----------------
> 5 files changed, 145 insertions(+), 103 deletions(-)
Applied to next, thanks.
^ permalink raw reply [flat|nested] 28+ messages in thread