From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7008446828784910336 X-Received: by 2002:adf:b748:: with SMTP id n8mr13286790wre.133.1631898094476; Fri, 17 Sep 2021 10:01:34 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1:: with SMTP id g1ls324482wmc.3.canary-gmail; Fri, 17 Sep 2021 10:01:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzfTOvxP0S3thAgzl/SIkwMTrkYT7YFrDN1/A3JwIMzfG5HJ8gSCDuS1GMz4GejWCpLHj0r X-Received: by 2002:a05:600c:4c09:: with SMTP id d9mr2903353wmp.194.1631898093427; Fri, 17 Sep 2021 10:01:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631898093; cv=none; d=google.com; s=arc-20160816; b=sw/uUSWA2vAUsP2Y8ZO2t/QMQUHHhYLLECZd7Br/mq48xA7ZVFqh5G2hyOC0PLevzT VQSDe3ewI2DMLwpsUoQae0sD1u9DqmLB3Zg79sUfY8LjMB7P5n/zfkDfymSolXtXzpEz gJA6jJVhyRGxez2TGZuRdIPibDI2lfKD/is+mHgKe8eIjFnG86CKHqQac6qK6vVrOYKU bvWEsJV4t3TtkcqZG3eksRs4H4tnf2KLnc3NthtxctPRTiKPukeEhwABVigcRor0at3l oieDrS8mGP7p48qABugesj3I047N9gk0hWzjYY99lpemvb0eUfFcN29y/KjxLo0yifmd zu7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=BxMzjqXX8pmpE3YVrhTpuECT+Tn1ilxC72PeJ1LzfwQ=; b=wOeAA5A9BWNxz7gBSkE38/RYAqyntxTnJ5dnnEpfvy8ixLJlrnoXJ4cAE3lvayT8dk dO1iXAapXyhPeKF3eyuxa1BQT6MgD1+0/HvivlprY58AznAjn0x6wF3dugawRpLn8Wci 7XQZmrMzXE5qilQdBsRuqzGiw9PMFnMhRpfOjEf1vIBMwyooLa/eyFpT1qliTO84LMpb 3hWezU0co2JhXtbItTEJYcx7nzwk5V8hj0p37w+TIE52f0o/gthpeGARylVAOKDTK7Nf twJcSMF2hulPNkIql4+fxqtRXT3DJuo7vzB6SD7jdFq2SIWFlYSBGt322h10X6hR5jyz fydQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 193si1023869wme.0.2021.09.17.10.01.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 17 Sep 2021 10:01:33 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from localhost.localdomain (mm-240-58-214-37.mgts.dynamic.pppoe.byfly.by [37.214.58.240] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 18HH0iND028039 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 17 Sep 2021 19:01:30 +0200 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Anton Mikanovich Subject: [PATCH v12 15/16] ci_build: Migrate to Avocado Date: Fri, 17 Sep 2021 20:00:32 +0300 Message-Id: <20210917170033.74797-16-amikan@ilbers.de> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210917170033.74797-1-amikan@ilbers.de> References: <20210917170033.74797-1-amikan@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: Gvj4+T4SEuqE Call Avocado test cases instead of shell based. Build artifacts are placed in build directory. Signed-off-by: Anton Mikanovich --- scripts/ci_build.sh | 208 +++------------------- scripts/ci_build_legacy.sh | 275 +++++++++++++++++++++++++++++ testsuite/build_test/build_test.py | 4 +- testsuite/build_test/cibase.py | 10 +- testsuite/build_test/cibuilder.py | 4 +- 5 files changed, 316 insertions(+), 185 deletions(-) create mode 100755 scripts/ci_build_legacy.sh diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index 48a51f2..4534957 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -14,6 +14,9 @@ export PATH=$PATH:/sbin # Go to Isar root cd "$(dirname "$0")/.." +# Get Avocado build tests path +BUILD_TEST_DIR="$(pwd)/testsuite/build_test" + # Start build in Isar tree by default BUILD_DIR=./build @@ -21,70 +24,10 @@ BUILD_DIR=./build DEPENDENCIES="umoci skopeo" for prog in ${DEPENDENCIES} ; do if [ ! -x "$(which $prog)" ] ; then - echo "missing $prog in PATH, exiting" >&2 - exit 1 + echo "missing $prog in PATH" >&2 fi done -BB_ARGS="-v" - -TARGETS_SET="\ - mc:qemuarm-stretch:isar-image-base \ - mc:qemuarm-buster:isar-image-base \ - mc:qemuarm64-stretch:isar-image-base \ - mc:qemui386-stretch:isar-image-base \ - mc:qemui386-buster:isar-image-base \ - mc:qemuamd64-stretch:isar-image-base \ - mc:qemuamd64-buster:isar-image-base \ - mc:qemuamd64-buster-tgz:isar-image-base \ - mc:qemuamd64-buster-cpiogz:isar-image-base \ - mc:qemuamd64-buster:isar-initramfs \ - mc:qemumipsel-stretch:isar-image-base \ - mc:qemumipsel-buster:isar-image-base \ - mc:nand-ubi-demo-buster:isar-image-ubi \ - mc:rpi-stretch:isar-image-base \ - mc:qemuamd64-focal:isar-image-base \ - mc:virtualbox-ova-buster:isar-image-base \ - " - # qemu-user-static of <= buster too old to build that - # mc:qemuarm64-buster:isar-image-base - # mc:qemuarm64-bullseye:isar-image-base - -TARGETS_SET_BULLSEYE="\ - mc:qemuamd64-bullseye:isar-image-base \ - mc:qemuarm-bullseye:isar-image-base \ - mc:qemui386-bullseye:isar-image-base \ - mc:qemumipsel-bullseye:isar-image-base \ -" - -TARGETS_CONTAINERS="\ - mc:container-amd64-stretch:isar-image-base \ - mc:container-amd64-buster:isar-image-base \ - mc:container-amd64-bullseye:isar-image-base \ -" - -CROSS_TARGETS_SET="\ - mc:qemuarm-stretch:isar-image-base \ - mc:qemuarm-buster:isar-image-base \ - mc:qemuarm64-stretch:isar-image-base \ - mc:qemuamd64-stretch:isar-image-base \ - mc:de0-nano-soc-buster:isar-image-base \ - mc:stm32mp15x-buster:isar-image-base \ - mc:rpi-stretch:isar-image-base \ - mc:qemuarm64-focal:isar-image-base" - -CROSS_TARGETS_SET_BULLSEYE="\ - mc:qemuarm-bullseye:isar-image-base \ -" - -REPRO_TARGETS_SET_SIGNED="\ - mc:de0-nano-soc-buster:isar-image-base \ - mc:qemuarm64-stretch:isar-image-base" - -REPRO_TARGETS_SET="\ - mc:qemuamd64-stretch:isar-image-base \ - mc:qemuarm-buster:isar-image-base" - show_help() { echo "This script builds the default Isar images." echo @@ -106,6 +49,10 @@ show_help() { echo " 3 if invalid parameters are passed." } +TAGS="full" +CROSS_BUILD="0" +QUIET="0" + # Parse command line to get user configuration while [ $# -gt 0 ] do @@ -124,16 +71,16 @@ do CROSS_BUILD="1" ;; -d|--debug) - BB_ARGS="$BB_ARGS -D" + VERBOSE="--show=app,test" ;; -f|--fast) # Start build for the reduced set of configurations # Enforce cross-compilation to speed up the build - FAST_BUILD="1" + TAGS="fast" CROSS_BUILD="1" ;; -q|--quiet) - BB_ARGS="" + QUIET="1" ;; -r|--repro) REPRO_BUILD="1" @@ -152,125 +99,24 @@ do shift done +if [ -z "$REPRO_BUILD" ]; then + TAGS="$TAGS,-repro" +fi + # the real stuff starts here, trace commands from now on set -x -# Setup build folder for the current build -if [ ! -d "$BUILD_DIR" ]; then - mkdir -p "$BUILD_DIR" -fi -source isar-init-build-env "$BUILD_DIR" - -cat >>conf/local.conf < .config/avocado/avocado.conf +[datadir.paths] +base_dir = $BUILD_DIR/ +test_dir = $BUILD_DIR/tests +data_dir = $BUILD_DIR/data +logs_dir = $BUILD_DIR/job-results EOF +export VIRTUAL_ENV="./" -if [ -n "$DISTRO_APT_PREMIRRORS" ]; then - echo "DISTRO_APT_PREMIRRORS = \"$DISTRO_APT_PREMIRRORS\"" >> conf/local.conf -fi - -if [ -n "$CROSS_BUILD" ]; then - sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf -fi - -if [ -n "$REPRO_BUILD" ]; then - ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key" - ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key" - export GNUPGHOME=$(mktemp -d) - gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE - - # Enable use of signed cached base repository - echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf - bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED - while [ -e bitbake.sock ]; do sleep 1; done - sudo rm -rf tmp - sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf - sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf - bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED - while [ -e bitbake.sock ]; do sleep 1; done - # Cleanup and disable use of signed cached base repository - sudo rm -rf tmp - sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf - sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf - sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf - bitbake $BB_ARGS $REPRO_TARGETS_SET - while [ -e bitbake.sock ]; do sleep 1; done - # Enable use of unsigned cached base repository - sudo rm -rf tmp - sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf - sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf - bitbake $BB_ARGS $REPRO_TARGETS_SET - while [ -e bitbake.sock ]; do sleep 1; done - # Disable use of unsigned cached base repository - sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf - sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf - # Try to build with changed configuration with no cleanup - bitbake $BB_ARGS $REPRO_TARGETS_SET - while [ -e bitbake.sock ]; do sleep 1; done - # Cleanup - sudo rm -rf tmp -fi - -sed -i -e 's/^#BASE_REPO_FEATURES ?= "cache-deb-src"/BASE_REPO_FEATURES ?= "cache-deb-src"/g' conf/local.conf -# Start cross build for the defined set of configurations -sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf -bitbake $BB_ARGS $CROSS_TARGETS_SET -while [ -e bitbake.sock ]; do sleep 1; done -if bitbake $BB_ARGS $CROSS_TARGETS_SET_BULLSEYE; then - echo "bullseye cross: PASSED" -else - echo "bullseye cross: KFAIL" -fi -# In addition test SDK creation -bitbake $BB_ARGS -c do_populate_sdk mc:qemuarm-stretch:isar-image-base -while [ -e bitbake.sock ]; do sleep 1; done - -if [ -z "$FAST_BUILD" ]; then - # Cleanup and disable cross build - sudo rm -rf tmp - sed -i -e 's/ISAR_CROSS_COMPILE ?= "1"/ISAR_CROSS_COMPILE ?= "0"/g' conf/local.conf - bitbake $BB_ARGS $TARGETS_SET - while [ -e bitbake.sock ]; do sleep 1; done - - if bitbake $BB_ARGS $TARGETS_SET_BULLSEYE; then - echo "bullseye: PASSED" - else - echo "bullseye: KFAIL" - fi - while [ -e bitbake.sock ]; do sleep 1; done -fi - -eval $(bitbake -e | grep "^LAYERDIR_core=") -eval $(bitbake -e | grep "^LAYERDIR_isar=") -cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" -echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/classes/dpkg-base.bbclass" - -bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base - -mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/classes/dpkg-base.bbclass" - -# Test wic --exclude-path -cp -a "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" -mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \ - ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup -sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" - -bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base - -mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" -mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \ - ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img - -# Finalize with containerized images, since they remove some not-needed packages from the local.conf -sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf -sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf -bitbake $BB_ARGS $TARGETS_CONTAINERS -while [ -e bitbake.sock ]; do sleep 1; done -# and SDK container image creation -echo 'SDK_FORMATS = "docker-archive"' >> conf/local.conf -bitbake $BB_ARGS -c do_populate_sdk mc:container-amd64-stretch:isar-image-base -while [ -e bitbake.sock ]; do sleep 1; done - +avocado $VERBOSE run "$BUILD_TEST_DIR/build_test.py" \ + -t $TAGS --test-runner=runner --disable-sysinfo \ + -p build_dir="$BUILD_DIR" -p quiet=$QUIET -p cross=$CROSS_BUILD diff --git a/scripts/ci_build_legacy.sh b/scripts/ci_build_legacy.sh new file mode 100755 index 0000000..d8418a9 --- /dev/null +++ b/scripts/ci_build_legacy.sh @@ -0,0 +1,275 @@ +#!/usr/bin/env bash +# Script for CI system build +# +# Alexander Smirnov +# Copyright (c) 2016-2018 ilbers GmbH + +set -e + +ES_BUG=3 + +# Export $PATH to use 'parted' tool +export PATH=$PATH:/sbin + +# Go to Isar root +cd "$(dirname "$0")/.." + +# Start build in Isar tree by default +BUILD_DIR=./build + +# Check dependencies +DEPENDENCIES="umoci skopeo" +for prog in ${DEPENDENCIES} ; do + if [ ! -x "$(which $prog)" ] ; then + echo "missing $prog in PATH, exiting" >&2 + exit 1 + fi +done + +BB_ARGS="-v" + +TARGETS_SET="\ + mc:qemuarm-stretch:isar-image-base \ + mc:qemuarm-buster:isar-image-base \ + mc:qemuarm64-stretch:isar-image-base \ + mc:qemui386-stretch:isar-image-base \ + mc:qemui386-buster:isar-image-base \ + mc:qemuamd64-stretch:isar-image-base \ + mc:qemuamd64-buster:isar-image-base \ + mc:qemuamd64-buster-tgz:isar-image-base \ + mc:qemuamd64-buster-cpiogz:isar-image-base \ + mc:qemuamd64-buster:isar-initramfs \ + mc:qemumipsel-stretch:isar-image-base \ + mc:qemumipsel-buster:isar-image-base \ + mc:nand-ubi-demo-buster:isar-image-ubi \ + mc:rpi-stretch:isar-image-base \ + mc:qemuamd64-focal:isar-image-base \ + mc:virtualbox-ova-buster:isar-image-base \ + " + # qemu-user-static of <= buster too old to build that + # mc:qemuarm64-buster:isar-image-base + # mc:qemuarm64-bullseye:isar-image-base + +TARGETS_SET_BULLSEYE="\ + mc:qemuamd64-bullseye:isar-image-base \ + mc:qemuarm-bullseye:isar-image-base \ + mc:qemui386-bullseye:isar-image-base \ + mc:qemumipsel-bullseye:isar-image-base \ +" + +TARGETS_CONTAINERS="\ + mc:container-amd64-stretch:isar-image-base \ + mc:container-amd64-buster:isar-image-base \ + mc:container-amd64-bullseye:isar-image-base \ +" + +CROSS_TARGETS_SET="\ + mc:qemuarm-stretch:isar-image-base \ + mc:qemuarm-buster:isar-image-base \ + mc:qemuarm64-stretch:isar-image-base \ + mc:qemuamd64-stretch:isar-image-base \ + mc:de0-nano-soc-buster:isar-image-base \ + mc:stm32mp15x-buster:isar-image-base \ + mc:rpi-stretch:isar-image-base \ + mc:qemuarm64-focal:isar-image-base" + +CROSS_TARGETS_SET_BULLSEYE="\ + mc:qemuarm-bullseye:isar-image-base \ +" + +REPRO_TARGETS_SET_SIGNED="\ + mc:de0-nano-soc-buster:isar-image-base \ + mc:qemuarm64-stretch:isar-image-base" + +REPRO_TARGETS_SET="\ + mc:qemuamd64-stretch:isar-image-base \ + mc:qemuarm-buster:isar-image-base" + +show_help() { + echo "This script builds the default Isar images." + echo + echo "Usage:" + echo " $0 [params]" + echo + echo "Parameters:" + echo " -b, --build BUILD_DIR set path to build directory. If not set," + echo " the build will be started in current path." + echo " -c, --cross enable cross-compilation." + echo " -d, --debug enable debug bitbake output." + echo " -f, --fast cross build reduced set of configurations." + echo " -q, --quiet suppress verbose bitbake output." + echo " -r, --repro enable use of cached base repository." + echo " --help display this message and exit." + echo + echo "Exit status:" + echo " 0 if OK," + echo " 3 if invalid parameters are passed." +} + +# Parse command line to get user configuration +while [ $# -gt 0 ] +do + key="$1" + + case $key in + -h|--help) + show_help + exit 0 + ;; + -b|--build) + BUILD_DIR="$2" + shift + ;; + -c|--cross) + CROSS_BUILD="1" + ;; + -d|--debug) + BB_ARGS="$BB_ARGS -D" + ;; + -f|--fast) + # Start build for the reduced set of configurations + # Enforce cross-compilation to speed up the build + FAST_BUILD="1" + CROSS_BUILD="1" + ;; + -q|--quiet) + BB_ARGS="" + ;; + -r|--repro) + REPRO_BUILD="1" + # This switch is deprecated, just here to not cause failing CI on + # legacy configs + case "$2" in + -s|--sign) shift ;; + esac + ;; + *) + echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters" + exit $ES_BUG + ;; + esac + + shift +done + +# the real stuff starts here, trace commands from now on +set -x + +# Setup build folder for the current build +if [ ! -d "$BUILD_DIR" ]; then + mkdir -p "$BUILD_DIR" +fi +source isar-init-build-env "$BUILD_DIR" + +cat >>conf/local.conf <> conf/local.conf +fi + +if [ -n "$CROSS_BUILD" ]; then + sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf +fi + +if [ -n "$REPRO_BUILD" ]; then + ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$TESTSUITEDIR/base-apt/test_pub.key" + ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$TESTSUITEDIR/base-apt/test_priv.key" + export GNUPGHOME=$(mktemp -d) + gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE + + # Enable use of signed cached base repository + echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf + bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED + while [ -e bitbake.sock ]; do sleep 1; done + sudo rm -rf tmp + sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf + sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf + bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED + while [ -e bitbake.sock ]; do sleep 1; done + # Cleanup and disable use of signed cached base repository + sudo rm -rf tmp + sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf + sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf + sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf + bitbake $BB_ARGS $REPRO_TARGETS_SET + while [ -e bitbake.sock ]; do sleep 1; done + # Enable use of unsigned cached base repository + sudo rm -rf tmp + sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf + sed -i -e 's/^#BB_NO_NETWORK/BB_NO_NETWORK/g' conf/local.conf + bitbake $BB_ARGS $REPRO_TARGETS_SET + while [ -e bitbake.sock ]; do sleep 1; done + # Disable use of unsigned cached base repository + sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf + sed -i -e 's/^BB_NO_NETWORK/#BB_NO_NETWORK/g' conf/local.conf + # Try to build with changed configuration with no cleanup + bitbake $BB_ARGS $REPRO_TARGETS_SET + while [ -e bitbake.sock ]; do sleep 1; done + # Cleanup + sudo rm -rf tmp +fi + +sed -i -e 's/^#BASE_REPO_FEATURES ?= "cache-deb-src"/BASE_REPO_FEATURES ?= "cache-deb-src"/g' conf/local.conf +# Start cross build for the defined set of configurations +sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf +bitbake $BB_ARGS $CROSS_TARGETS_SET +while [ -e bitbake.sock ]; do sleep 1; done +if bitbake $BB_ARGS $CROSS_TARGETS_SET_BULLSEYE; then + echo "bullseye cross: PASSED" +else + echo "bullseye cross: KFAIL" +fi +# In addition test SDK creation +bitbake $BB_ARGS -c do_populate_sdk mc:qemuarm-stretch:isar-image-base +while [ -e bitbake.sock ]; do sleep 1; done + +if [ -z "$FAST_BUILD" ]; then + # Cleanup and disable cross build + sudo rm -rf tmp + sed -i -e 's/ISAR_CROSS_COMPILE ?= "1"/ISAR_CROSS_COMPILE ?= "0"/g' conf/local.conf + bitbake $BB_ARGS $TARGETS_SET + while [ -e bitbake.sock ]; do sleep 1; done + + if bitbake $BB_ARGS $TARGETS_SET_BULLSEYE; then + echo "bullseye: PASSED" + else + echo "bullseye: KFAIL" + fi + while [ -e bitbake.sock ]; do sleep 1; done +fi + +eval $(bitbake -e | grep "^LAYERDIR_core=") +eval $(bitbake -e | grep "^LAYERDIR_isar=") +cp -a "${LAYERDIR_core}/classes/dpkg-base.bbclass" "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" +echo -e "do_fetch_append() {\n\n}" >> "${LAYERDIR_core}/classes/dpkg-base.bbclass" + +bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base + +mv "${LAYERDIR_core}/classes/dpkg-base.bbclass.ci-backup" "${LAYERDIR_core}/classes/dpkg-base.bbclass" + +# Test wic --exclude-path +cp -a "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" +mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img \ + ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup +sed -i -e 's/part \/ /part \/ --exclude-path usr /g' "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" + +bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base + +mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks" +mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \ + ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img + +# Finalize with containerized images, since they remove some not-needed packages from the local.conf +sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf +sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf +bitbake $BB_ARGS $TARGETS_CONTAINERS +while [ -e bitbake.sock ]; do sleep 1; done +# and SDK container image creation +echo 'SDK_FORMATS = "docker-archive"' >> conf/local.conf +bitbake $BB_ARGS -c do_populate_sdk mc:container-amd64-stretch:isar-image-base +while [ -e bitbake.sock ]; do sleep 1; done diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py index 63ea301..d39c10c 100644 --- a/testsuite/build_test/build_test.py +++ b/testsuite/build_test/build_test.py @@ -109,12 +109,14 @@ class NoCrossTest(CIBaseTest): 'mc:qemuamd64-stretch:isar-image-base', 'mc:qemuamd64-buster:isar-image-base', 'mc:qemuamd64-buster-tgz:isar-image-base', + 'mc:qemuamd64-buster-cpiogz:isar-image-base', 'mc:qemuamd64-buster:isar-initramfs', 'mc:qemumipsel-stretch:isar-image-base', 'mc:qemumipsel-buster:isar-image-base', 'mc:nand-ubi-demo-buster:isar-image-ubi', 'mc:rpi-stretch:isar-image-base', - 'mc:qemuamd64-focal:isar-image-base' + 'mc:qemuamd64-focal:isar-image-base', + 'mc:virtualbox-ova-buster:isar-image-base' ] # Cleanup after cross build diff --git a/testsuite/build_test/cibase.py b/testsuite/build_test/cibase.py index 1a89f43..78d7bdb 100644 --- a/testsuite/build_test/cibase.py +++ b/testsuite/build_test/cibase.py @@ -67,10 +67,16 @@ class CIBaseTest(CIBuilder): self.bitbake(build_dir, targets, None, bb_args) - # Cleanup and disable use of signed cached base repository - self.deletetmp(build_dir) + # Disable use of cached base repository self.confcleanup(build_dir) + if not signed: + # Try to build with changed configuration with no cleanup + self.bitbake(build_dir, targets, None, bb_args) + + # Cleanup + self.deletetmp(build_dir) + def perform_wic_test(self, targets, wks_path, wic_path): cross = int(self.params.get('cross', default=0)) build_dir, bb_args = self.prep('WIC exclude build', targets, cross, 1) diff --git a/testsuite/build_test/cibuilder.py b/testsuite/build_test/cibuilder.py index 187363f..baa7185 100644 --- a/testsuite/build_test/cibuilder.py +++ b/testsuite/build_test/cibuilder.py @@ -48,6 +48,9 @@ class CIBuilder(Test): f.write('ISAR_CROSS_COMPILE = "1"\n') if debsrc_cache: f.write('BASE_REPO_FEATURES = "cache-deb-src"\n') + distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS') + if distro_apt_premir: + f.write('DISTRO_APT_PREMIRRORS = "%s"\n' % distro_apt_premir) with open(build_dir + '/conf/local.conf', 'r+') as f: for line in f: @@ -127,4 +130,3 @@ class CIBuilder(Test): for x in output.splitlines() if x != '')) return env['LAYERDIR_' + layer].strip('"') - -- 2.25.1