From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7039647964761948160 X-Received: by 2002:a5d:5643:: with SMTP id j3mr12601664wrw.138.1639124146185; Fri, 10 Dec 2021 00:15:46 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:4092:: with SMTP id o18ls139428wrp.1.gmail; Fri, 10 Dec 2021 00:15:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJxO1Ze2yC+n3AtwFrDAV/NSgRytAq9rXQ+FRHOWzNTchMS8rg5b5q6ZDuOG4lR7xu+VTlmf X-Received: by 2002:a5d:5850:: with SMTP id i16mr12427131wrf.197.1639124145238; Fri, 10 Dec 2021 00:15:45 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1639124145; cv=pass; d=google.com; s=arc-20160816; b=L2o218AyYKgSNeZi9TuqGDWmkc7fEm1DJma6sIJ87Lj3tUeGTcH6Hoc9mQE/DcMH/z //mVTjkIXSNSMjhHQtYKdtgl/D7IKYiVD56idF5gP9FkGSVmgKffy03ugA0YNUrNJ2h7 CQ+7rY4lSKYy3XT0mlq8vcQ/3Wan/pIdwaJhNRTA/DBAJk+N/jszpaGT5u0EyekSLnJc TvZB5VaeRVogcKH0w7DkAd5c/+46x7laf7hYv7kaHhv4ZvnE+rsE1ztXhw0omWkk/D+s N0QnbGr0ABaFX8NpEaehoGsoyuEKrTMo3WaprKvo9RiygLCsfEN/OIWzYPVKL5yY6l8j 8vuA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=8TyJyQCE69HfN3a6LvAPnOKnTZnBUZptgbLD25p6KS0=; b=VEQYWDyMORRsQtGRcjQm5BtUhZYNEcI7daizCYUfH6d+i2+ui7IYBBfjiDWvvp0AgN RQDHpa+ATKXH/9WDDsY7T+Y6jjCfr8dGw0ui8oxrU8h0+/yd5ECeReknnNaUQHxsA6op no3dAqVP1Vh+nIzpGThbR+lQNq0A2Bm9CTZsPuZHi1MXCDVbrdT7dcNgAiPJNmKeYglH nAuudRdA5rToLDrl/qYB4PetHJCjGGCDNIyOcumE8OSc4LBrxvn9FBbZzuv/Rm1wm2G9 iMimmyM1Dds69PmKt7NKtKxjV5rqQD6/bQOqQ/xfgN2JriKEaHYUo4iSYpoPegP8+MHX e98Q== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=TIGsIAP1; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.8.77 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80077.outbound.protection.outlook.com. [40.107.8.77]) by gmr-mx.google.com with ESMTPS id s138si774602wme.1.2021.12.10.00.15.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Dec 2021 00:15:45 -0800 (PST) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.8.77 as permitted sender) client-ip=40.107.8.77; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=TIGsIAP1; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 40.107.8.77 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RHwK+iCNIT+PTkty11QH8wSft4/u/Ibl7AdJ2trIX2yGaWQD5kBM+3fti1AT+1COKatJQ/zL4thRktMtFDgIg0b2BV8JyTnCvWsfbfJ5umZCEqIRBbndOCapmmEY08MT1hR2bCuv6DJVGWgxh1MrpTfTCvjEZYXpTtuwMu/7YgUGhEwuqfDv+1GrgW00OCCCtmxzzWIQ8z9NZm2Tme5niFg4xyF7D44IIF0fJAt6FFkPMDEhex7NRF0rFBJd/HtOI4rQ2/pqVzXBxUqRsLB7simCluK7blDplNMa9mxOZPvDNrasOJUBsrVfffUwxgnwXAjgolYRNWIbn1GPP0Rr2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8TyJyQCE69HfN3a6LvAPnOKnTZnBUZptgbLD25p6KS0=; b=NdJ2YnDkHah1pB4sv1QjOoBZTkzeRyanedZUyojJrcowbwD4rHoQnv2TCS663/9cxlSxCWnLdsj4clrk/YY5x1G4c4BQ7GDRRLudVy8NqWbth0/ah+NtKD5v7hkWaaV9ZRNsutV002FuiABd3FvgAmbSdEpW3WepGwPLWL1uPwqJeMX+I59iFRi28I/wc9rr2VqiwIbRI5nvwNv92fzM5rTySXQuOquQXIyhjJ3YmJGZ3RiUJ4f1yVeM1OTVIa46J1EDv2b+0ZFZceb4tbROkX+/xiRpS/24Vb8DywbPUYf2YLF8PY3D+taLpU2SSqpcVyUa4AlD1kyVPUAykeQoeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.72) smtp.rcpttodomain=ilbers.de smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8TyJyQCE69HfN3a6LvAPnOKnTZnBUZptgbLD25p6KS0=; b=TIGsIAP1lPu2JCdz1ucrnqLj9s9FKPBORVERD7O9K67SyInIXK4o05mBP8kOdFkNMffYJMu/6at9bwPEULiun3ate1MOTFzKirBmQTNtaYqG7ef2vOxo3pBAsP50iupw58UQ01TdRHDaXv8EJ1u+qF8MG+t1Tn0aYf+YAltNNHRGAqMusvvlcBXz38YF6WjuRHfsQdG5RPSp3rrv5m5JqWvmic5lhU1q/tNLPIUnDaqcv6vhGA4HqEMp+JAPRZ5TXpSbNtSjLA2DxQHVfZqPIYuUcT9hBfVB8IQooYo+LX4S8Y5M0dsRyp1sJ/DjiJBxUHXgpY7VpaTe28StZ+Iaow== Received: from SV0P279CA0007.NORP279.PROD.OUTLOOK.COM (2603:10a6:f10:11::12) by AS8PR10MB4711.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:318::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.21; Fri, 10 Dec 2021 08:15:43 +0000 Received: from HE1EUR01FT005.eop-EUR01.prod.protection.outlook.com (2603:10a6:f10:11:cafe::46) by SV0P279CA0007.outlook.office365.com (2603:10a6:f10:11::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.17 via Frontend Transport; Fri, 10 Dec 2021 08:15:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.72) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.72 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.72; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.72) by HE1EUR01FT005.mail.protection.outlook.com (10.152.1.229) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4778.12 via Frontend Transport; Fri, 10 Dec 2021 08:15:43 +0000 Received: from DEMCHDC89YA.ad011.siemens.net (139.25.226.104) by DEMCHDC9SMA.ad011.siemens.net (194.138.21.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Fri, 10 Dec 2021 09:15:42 +0100 Received: from random.ppmd.siemens.net (139.25.68.25) by DEMCHDC89YA.ad011.siemens.net (139.25.226.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Fri, 10 Dec 2021 09:15:42 +0100 From: Adriaan Schmidt To: , CC: Adriaan Schmidt Subject: [RFC PATCH v2] testsuite: refactor Date: Fri, 10 Dec 2021 09:15:26 +0100 Message-ID: <20211210081526.2944853-1-adriaan.schmidt@siemens.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain Return-Path: adriaan.schmidt@siemens.com X-Originating-IP: [139.25.68.25] X-ClientProxiedBy: DEMCHDC89YA.ad011.siemens.net (139.25.226.104) To DEMCHDC89YA.ad011.siemens.net (139.25.226.104) X-TM-AS-Product-Ver: SMEX-14.0.0.3080-8.6.1018-26580.006 X-TM-AS-Result: No-10--9.839000-8.000000 X-TMASE-MatchedRID: iRwsnLDGEHeQLaoaEKRbU8ZPPXOsW6nbyHwwnnWuWmvo5fsYXP0fUBiD IOPlOJG11YzbHoRn9L0E8cQsUQdAHwQy4qivAwYgSsX+ZURUowlmZsAqgZ7gH2wTEruL9ObTpWO BfK9L1z/An6/WnA76fvM+9Fw01I7GfGzuoVn0Vs6PQi9XuOWoOMWo6kJnduXJKrDHzH6zmUUEa8 g1x8eqF9JO+sYlhfSotVS7djV62e+CukcfrdcoGZctTH4XjCd+brPIUt6+GzgtvPXhZLGnkTuDa yXS0LqBCgNjqBgJU5O0gcB1l1Fz4ady/Us1uHCq4dJOQj1ZX0rFlh6wZqSS6+pKEpRpjeYiUXlp 1FHYSPVf0yVReoVM7WULGsUmVy58eCGIi8zhAhWAqyvkiZGeGWOEtO7K/2e5ZacDbE73ZSktOI9 ZW/K3Op+4ziUPq4LxzKhI/PM+wq96QAuR8hT0ILlUFWrj927bMbO7pRnSf75v+ggm5QAi4fE/cU LMuCxwCuSPuSVW5+66Ij5glj2cugXsO/8AMIjNWa/mmNeQ2z8+KbeGe6bo/fgAhuaFie7S1K3bs e3FjTs+argXbuCfh8qZ/PwTGIUMUtTgmoNQjI2MqybMD87ynPV1MnnwgooQggra2NOo2i2CeHfY ZgVO5plTEz+bsQajoIXZ6Oi7T50OjITYH5OIYCASu+a9XHN6nM8FMFzcocJ1e7Xbb6Im2uHv2XT +zK5Wa0TOsL14A2l0t9tHJjfNd2/eSPFWFCWrBKn0MEJ4VA8/4zhj2dSlgcduhdtUlQVLfS0Ip2 eEHnzFf/Ie0cs/wMFwgTvxipFajoczmuoPCq0wIPYySOiKVp+pKH/mB6AhmtQ4brv6ggU4dJHu8 MpeMA82OQH2Hrfu X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--9.839000-8.000000 X-TMASE-Version: SMEX-14.0.0.3080-8.6.1018-26580.006 X-TM-SNTS-SMTP: 5D1A24107559EDD4506B1CE2E33B780AECB3A8970DBAB364F7ED4386E21AEF1C2000:8 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4bd417b5-49fe-498e-34a0-08d9bbb542d3 X-MS-TrafficTypeDiagnostic: AS8PR10MB4711:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MO+YyISlRGxV3iKGptDc4SzkQLzeMFi9BoBrCK2B/gwQBEIv8sBVqDlQoEDwLwHTuXYqILC6J0wAytVshpEE014WJZWDyVT5K81160xxQ0LwLDQd2elbcNIkIiE2BlDqsYdQ56HHM5ULaO+qjZQQqMqUDcvRx4Tv47dTf4uC9nohuxiMPDj5TIj6ekrZjCiAN4DWeMBqqOfGZewAqgLLi5SHnqSTg8K9r/AxmUymBMw8gtdfm6qKicTaLxWazAdRBYzMTeEFfu1TcYy5B/Oe88uuBzk30u7Iyvq+ND+QLSY7G9CNlGdBrzvs6TqHHLIcUdEd4nT5H/TuZXc2ssgLnaTODt+agh4YTWGpBYOKiQmQq2UyjCT1VWx2kCqsBqMvKl9bLTKCbG1ywicd9bXM0OkepD5pZSH74nHo6lGN2jw8NFa08Ae9SLROvD0lR9urxfV4HCfCutorqAGD8yOk+IKe81ooFawhmUk+12E4DOQK8sPnyiPlMTksdxStG7L9cjOvKm/I2rPKqRvHi5HSMFtRUcKCA/19LMSCxHU/Ui3g/OYwWnYnwQ7OjD2/SxiA793yNL5yWpMY1YYCjXtYwxV3hqgHDe9x/l5N90CnX3T0JZ/JJRuQ4P6fHE1/oZ4NLVzNZsyXdRoKF3Y6Hw8ruZsdl3FUMx9KWrK0YIfyVSbczitpUUyqxwiT/gnut+vRbN3ZwFw/09RVtT7fsFO1hhV4xjGblj1td3sExknXDZ8= X-Forefront-Antispam-Report: CIP:194.138.21.72;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(36756003)(316002)(36860700001)(356005)(86362001)(70206006)(110136005)(83380400001)(2616005)(4326008)(336012)(6666004)(26005)(82960400001)(47076005)(186003)(107886003)(82310400004)(44832011)(16526019)(30864003)(1076003)(8676002)(8936002)(70586007)(2906002)(508600001)(956004)(81166007)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Dec 2021 08:15:43.3364 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4bd417b5-49fe-498e-34a0-08d9bbb542d3 X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.72];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: HE1EUR01FT005.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR10MB4711 X-TUID: fhiCgdK1NdHN - make test initialization more explicit. tests now call init() to initialize build_dir and environment, and configure() to generate the config file and bitbake_args - only configure() touches the config file. if config needs to change during one test case, it is created from scratch (no appending by the test case itself). - build_dir is not passed via avocado parameter. each test can set it in the call to init(). this makes dependencies between tests explicit and permits parallelization. - in the main invocation script, rename --build to --base. what we're setting here is not the bitbake build_dir but the base dir for avocado test output. Signed-off-by: Adriaan Schmidt --- scripts/ci_build.sh | 22 +++--- testsuite/build_test/build_test.py | 35 +++++----- testsuite/build_test/cibase.py | 103 +++++++++-------------------- testsuite/build_test/cibuilder.py | 98 +++++++++++++++++++-------- 4 files changed, 128 insertions(+), 130 deletions(-) diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index e9ba034..339ebca 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -27,8 +27,8 @@ fi # Get Avocado build tests path BUILD_TEST_DIR="$(pwd)/testsuite/build_test" -# Start build in Isar tree by default -BUILD_DIR=./build +# Start tests in current path by default +BASE_DIR=./build # Check dependencies DEPENDENCIES="umoci skopeo" @@ -45,8 +45,8 @@ show_help() { echo " $0 [params]" echo echo "Parameters:" - echo " -b, --build BUILD_DIR set path to build directory. If not set," - echo " the build will be started in current path." + echo " -b, --base BASE_DIR set path to base directory. If not set," + echo " the tests will be started in current path." echo " -c, --cross enable cross-compilation." echo " -d, --debug enable debug bitbake output." echo " -f, --fast cross build reduced set of configurations." @@ -73,8 +73,8 @@ do show_help exit 0 ;; - -b|--build) - BUILD_DIR="$2" + -b|--base) + BASE_DIR="$2" shift ;; -c|--cross) @@ -117,10 +117,10 @@ fi mkdir -p .config/avocado cat < .config/avocado/avocado.conf [datadir.paths] -base_dir = $(realpath $BUILD_DIR)/ -test_dir = $(realpath $BUILD_DIR)/tests -data_dir = $(realpath $BUILD_DIR)/data -logs_dir = $(realpath $BUILD_DIR)/job-results +base_dir = $(realpath $BASE_DIR)/ +test_dir = $(realpath $BASE_DIR)/tests +data_dir = $(realpath $BASE_DIR)/data +logs_dir = $(realpath $BASE_DIR)/job-results EOF export VIRTUAL_ENV="./" @@ -129,4 +129,4 @@ set -x avocado $VERBOSE run "$BUILD_TEST_DIR/build_test.py" \ -t $TAGS --test-runner=runner --disable-sysinfo \ - -p build_dir="$BUILD_DIR" -p quiet=$QUIET -p cross=$CROSS_BUILD + -p quiet=$QUIET -p cross=$CROSS_BUILD diff --git a/testsuite/build_test/build_test.py b/testsuite/build_test/build_test.py index de9e3fc..7359295 100644 --- a/testsuite/build_test/build_test.py +++ b/testsuite/build_test/build_test.py @@ -3,7 +3,7 @@ import os from avocado import skipUnless -from avocado.utils import path +from avocado.utils import path, process from cibase import CIBaseTest UMOCI_AVAILABLE = True @@ -30,7 +30,7 @@ class ReproTest(CIBaseTest): 'mc:qemuarm64-stretch:isar-image-base' ] - self.perform_repro_test(targets, 1) + self.perform_repro_test(targets, signed=True) def test_repro_unsigned(self): targets = [ @@ -38,7 +38,7 @@ class ReproTest(CIBaseTest): 'mc:qemuarm-stretch:isar-image-base' ] - self.perform_repro_test(targets, 0) + self.perform_repro_test(targets) class CcacheTest(CIBaseTest): @@ -69,7 +69,7 @@ class CrossTest(CIBaseTest): 'mc:rpi-stretch:isar-image-base' ] - self.perform_build_test(targets, 1, None) + self.perform_build_test(targets, cross=True) def test_cross_ubuntu(self): targets = [ @@ -77,7 +77,7 @@ class CrossTest(CIBaseTest): ] try: - self.perform_build_test(targets, 1, None) + self.perform_build_test(targets, cross=True) except: self.cancel('KFAIL') @@ -87,7 +87,7 @@ class CrossTest(CIBaseTest): ] try: - self.perform_build_test(targets, 1, None) + self.perform_build_test(targets, cross=True) except: self.cancel('KFAIL') @@ -101,7 +101,7 @@ class SdkTest(CIBaseTest): def test_sdk(self): targets = ['mc:qemuarm-stretch:isar-image-base'] - self.perform_build_test(targets, 1, 'do_populate_sdk') + self.perform_build_test(targets, bitbake_cmd='do_populate_sdk') class NoCrossTest(CIBaseTest): @@ -131,10 +131,9 @@ class NoCrossTest(CIBaseTest): ] # Cleanup after cross build - self.deletetmp(self.params.get('build_dir', - default=os.path.dirname(__file__) + '/../../build')) - - self.perform_build_test(targets, 0, None) + self.init() + self.delete_from_build_dir('tmp') + self.perform_build_test(targets, cross=False) def test_nocross_bullseye(self): targets = [ @@ -146,7 +145,7 @@ class NoCrossTest(CIBaseTest): ] try: - self.perform_build_test(targets, 0, None) + self.perform_build_test(targets, cross=False) except: self.cancel('KFAIL') @@ -158,19 +157,16 @@ class RebuildTest(CIBaseTest): :avocado: tags=rebuild,fast,full """ def test_rebuild(self): - is_cross_build = int(self.params.get('cross', default=0)) - + self.init() layerdir_core = self.getlayerdir('core') dpkgbase_file = layerdir_core + '/classes/dpkg-base.bbclass' - self.backupfile(dpkgbase_file) with open(dpkgbase_file, 'a') as file: file.write('do_fetch_append() {\n\n}') try: - self.perform_build_test('mc:qemuamd64-stretch:isar-image-base', - is_cross_build, None) + self.perform_build_test('mc:qemuamd64-stretch:isar-image-base') finally: self.restorefile(dpkgbase_file) @@ -189,6 +185,7 @@ class WicTest(CIBaseTest): self.perform_wic_test('mc:qemuamd64-stretch:isar-image-base', wks_path, wic_path) + class ContainerImageTest(CIBaseTest): """ @@ -204,7 +201,7 @@ class ContainerImageTest(CIBaseTest): 'mc:container-amd64-bullseye:isar-image-base' ] - self.perform_container_test(targets, None) + self.perform_build_test(targets, container=True) class ContainerSdkTest(CIBaseTest): @@ -217,4 +214,4 @@ class ContainerSdkTest(CIBaseTest): def test_container_sdk(self): targets = ['mc:container-amd64-stretch:isar-image-base'] - self.perform_container_test(targets, 'do_populate_sdk') + self.perform_build_test(targets, bitbake_cmd='do_populate_sdk', container=True) diff --git a/testsuite/build_test/cibase.py b/testsuite/build_test/cibase.py index 0b053aa..9c06c94 100644 --- a/testsuite/build_test/cibase.py +++ b/testsuite/build_test/cibase.py @@ -8,50 +8,24 @@ import time from cibuilder import CIBuilder from avocado.utils import process -isar_root = os.path.dirname(__file__) + '/../..' class CIBaseTest(CIBuilder): - - def prep(self, testname, targets, cross, debsrc_cache): - build_dir = self.params.get('build_dir', default=isar_root + '/build') - build_dir = os.path.realpath(build_dir) - quiet = int(self.params.get('quiet', default=0)) - bitbake_args = '-v' - - if quiet: - bitbake_args = '' - - self.log.info('===================================================') - self.log.info('Running ' + testname + ' test for:') - self.log.info(targets) - self.log.info('Isar build folder is: ' + build_dir) - self.log.info('===================================================') - - self.init(build_dir) - self.confprepare(build_dir, 1, cross, debsrc_cache) - - return build_dir, bitbake_args; - - def perform_build_test(self, targets, cross, bitbake_cmd): - build_dir, bb_args = self.prep('Isar build', targets, cross, 1) + def perform_build_test(self, targets, **kwargs): + self.init(**kwargs) + self.configure(**kwargs) self.log.info('Starting build...') - self.bitbake(build_dir, targets, bitbake_cmd, bb_args) - - def perform_repro_test(self, targets, signed): - cross = int(self.params.get('cross', default=0)) - build_dir, bb_args = self.prep('repro Isar build', targets, cross, 0) + self.bitbake(targets, **kwargs) + def perform_repro_test(self, targets, signed=False, **kwargs): gpg_pub_key = os.path.dirname(__file__) + '/../base-apt/test_pub.key' gpg_priv_key = os.path.dirname(__file__) + '/../base-apt/test_priv.key' - if signed: - with open(build_dir + '/conf/ci_build.conf', 'a') as file: - # Enable use of signed cached base repository - file.write('BASE_REPO_KEY="file://' + gpg_pub_key + '"\n') + self.init(**kwargs) + self.configure(gpg_pub_key=gpg_pub_key if signed else None, **kwargs) - os.chdir(build_dir) + os.chdir(self.build_dir) os.environ['GNUPGHOME'] = tempfile.mkdtemp() result = process.run('gpg --import %s %s' % (gpg_pub_key, gpg_priv_key)) @@ -59,33 +33,31 @@ class CIBaseTest(CIBuilder): if result.exit_status: self.fail('GPG import failed') - self.bitbake(build_dir, targets, None, bb_args) + self.bitbake(targets, **kwargs) - self.deletetmp(build_dir) - with open(build_dir + '/conf/ci_build.conf', 'a') as file: - file.write('ISAR_USE_CACHED_BASE_REPO = "1"\n') - file.write('BB_NO_NETWORK = "1"\n') + self.delete_from_build_dir('tmp') + self.configure(gpg_pub_key=gpg_pub_key if signed else None, offline=True, **kwargs) - self.bitbake(build_dir, targets, None, bb_args) + self.bitbake(targets, **kwargs) # Disable use of cached base repository - self.confcleanup(build_dir) + self.unconfigure() if not signed: # Try to build with changed configuration with no cleanup - self.bitbake(build_dir, targets, None, bb_args) + self.bitbake(targets, **kwargs) # Cleanup - self.deletetmp(build_dir) + self.delete_from_build_dir('tmp') - def perform_wic_test(self, targets, wks_path, wic_path): - cross = int(self.params.get('cross', default=0)) - build_dir, bb_args = self.prep('WIC exclude build', targets, cross, 1) + def perform_wic_test(self, targets, wks_path, wic_path, **kwargs): + self.init(**kwargs) + self.configure(**kwargs) layerdir_isar = self.getlayerdir('isar') wks_file = layerdir_isar + wks_path - wic_img = build_dir + wic_path + wic_img = self.build_dir + wic_path if not os.path.isfile(wic_img): self.fail('No build started before: ' + wic_img + ' not exist') @@ -99,44 +71,30 @@ class CIBaseTest(CIBuilder): for line in lines: file.write(re.sub(r'part \/ ', 'part \/ --exclude-path usr ', line)) - try: - self.bitbake(build_dir, targets, None, bb_args) + self.bitbake(targets, **kwargs) finally: self.restorefile(wks_file) - self.restorefile(wic_img) - def perform_container_test(self, targets, bitbake_cmd): - cross = int(self.params.get('cross', default=0)) - build_dir, bb_args = self.prep('Isar Container', targets, cross, 1) - - self.containerprep(build_dir) - - self.bitbake(build_dir, targets, bitbake_cmd, bb_args) - + def perform_ccache_test(self, targets, **kwargs): + self.init(**kwargs) + self.configure(ccache=True, **kwargs) - def perform_ccache_test(self, targets): - build_dir, bb_args = self.prep('Isar ccache build', targets, 0, 0) - - self.deletetmp(build_dir) - process.run('rm -rf ' + build_dir + '/ccache', sudo=True) - - with open(build_dir + '/conf/ci_build.conf', 'a') as file: - file.write('USE_CCACHE = "1"\n') - file.write('CCACHE_TOP_DIR = "${TOPDIR}/ccache"') + self.delete_from_build_dir('tmp') + self.delete_from_build_dir('ccache') self.log.info('Starting build and filling ccache dir...') start = time.time() - self.bitbake(build_dir, targets, None, bb_args) + self.bitbake(targets, **kwargs) first_time = time.time() - start self.log.info('Non-cached build: ' + str(round(first_time)) + 's') - self.deletetmp(build_dir) + self.delete_from_build_dir('tmp') self.log.info('Starting build and using ccache dir...') start = time.time() - self.bitbake(build_dir, targets, None, bb_args) + self.bitbake(targets, **kwargs) second_time = time.time() - start self.log.info('Cached build: ' + str(round(second_time)) + 's') @@ -145,5 +103,6 @@ class CIBaseTest(CIBuilder): self.fail('No speedup after rebuild with ccache') # Cleanup - self.deletetmp(build_dir) - process.run('rm -rf ' + build_dir + '/ccache', sudo=True) + self.delete_from_build_dir('tmp') + self.delete_from_build_dir('ccache') + diff --git a/testsuite/build_test/cibuilder.py b/testsuite/build_test/cibuilder.py index 94786c7..6a9f6a0 100644 --- a/testsuite/build_test/cibuilder.py +++ b/testsuite/build_test/cibuilder.py @@ -2,7 +2,6 @@ import logging import os -import re import select import shutil import subprocess @@ -11,9 +10,8 @@ from avocado import Test from avocado.utils import path from avocado.utils import process -isar_root = os.path.dirname(__file__) + '/../..' +isar_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..')) backup_prefix = '.ci-backup' - app_log = logging.getLogger("avocado.app") class CIBuilder(Test): @@ -28,17 +26,55 @@ class CIBuilder(Test): self._file_handler.setFormatter(formatter) app_log.addHandler(self._file_handler) - def init(self, build_dir): + + def init(self, build_dir='build', **kwargs): + # initialize build_dir and setup environment + # needs to run once (per test case) + self.build_dir = os.path.join(isar_root, build_dir) os.chdir(isar_root) - path.usable_rw_dir(build_dir) + path.usable_rw_dir(self.build_dir) output = process.getoutput('/bin/bash -c "source isar-init-build-env \ - %s 2>&1 >/dev/null; env"' % build_dir) + %s 2>&1 >/dev/null; env"' % self.build_dir) env = dict(((x.split('=', 1) + [''])[:2] \ for x in output.splitlines() if x != '')) os.environ.update(env) - def confprepare(self, build_dir, compat_arch, cross, debsrc_cache): - with open(build_dir + '/conf/ci_build.conf', 'w') as f: + + def configure(self, compat_arch=True, cross=None, debsrc_cache=True, + container=False, ccache=False, sstate=False, offline=False, + gpg_pub_key=None, **kwargs): + # write configuration file and set bitbake_args + # can run multiple times per test case + + # get parameters from avocado cmdline + quiet = bool(int(self.params.get('quiet', default=0))) + if cross is None: + cross = bool(int(self.params.get('cross', default=0))) + + # get parameters from environment + distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS') + + self.log.info(f'===================================================\n' + f'Configuring build_dir {self.build_dir}\n' + f' compat_arch = {compat_arch}\n' + f' cross = {cross}\n' + f' debsrc_cache = {debsrc_cache}\n' + f' offline = {offline}\n' + f' container = {container}\n' + f' ccache = {ccache}\n' + f' sstate = {sstate}\n' + f' gpg_pub_key = {gpg_pub_key}\n' + f'===================================================') + + # determine bitbake_args + self.bitbake_args = [] + if not quiet: + self.bitbake_args.append('-v') + if not sstate: + self.bitbake_args.append('--no-setscene') + + # write ci_build.conf + with open(self.build_dir + '/conf/ci_build.conf', 'w') as f: if compat_arch: f.write('ISAR_ENABLE_COMPAT_ARCH_amd64 = "1"\n') f.write('ISAR_ENABLE_COMPAT_ARCH_arm64 = "1"\n') @@ -48,36 +84,42 @@ class CIBuilder(Test): f.write('ISAR_CROSS_COMPILE = "1"\n') if debsrc_cache: f.write('BASE_REPO_FEATURES = "cache-deb-src"\n') - distro_apt_premir = os.getenv('DISTRO_APT_PREMIRRORS') + if offline: + f.write('ISAR_USE_CACHED_BASE_REPO = "1"\n') + f.write('BB_NO_NETWORK = "1"\n') + if container: + f.write('SDK_FORMATS = "docker-archive"\n') + f.write('IMAGE_INSTALL_remove = "example-module-${KERNEL_NAME} enable-fsck"\n') + if gpg_pub_key: + f.write('BASE_REPO_KEY="file://' + gpg_pub_key + '"\n') if distro_apt_premir: f.write('DISTRO_APT_PREMIRRORS = "%s"\n' % distro_apt_premir) - with open(build_dir + '/conf/local.conf', 'r+') as f: + # include ci_build.conf in local.conf + with open(self.build_dir + '/conf/local.conf', 'r+') as f: for line in f: if 'include ci_build.conf' in line: break else: f.write('\ninclude ci_build.conf') - def containerprep(self, build_dir): - with open(build_dir + '/conf/ci_build.conf', 'a') as f: - f.write('SDK_FORMATS = "docker-archive"\n') - f.write('IMAGE_INSTALL_remove = "example-module-${KERNEL_NAME} enable-fsck"\n') - - def confcleanup(self, build_dir): - open(build_dir + '/conf/ci_build.conf', 'w').close() + def unconfigure(self): + open(self.build_dir + '/conf/ci_build.conf', 'w').close() - def deletetmp(self, build_dir): - process.run('rm -rf ' + build_dir + '/tmp', sudo=True) + def delete_from_build_dir(self, path): + process.run('rm -rf ' + self.build_dir + '/' + path, sudo=True) - def bitbake(self, build_dir, target, cmd, args): - os.chdir(build_dir) + def bitbake(self, target, bitbake_cmd=None, **kwargs): + self.log.info('===================================================') + self.log.info('Building ' + str(target)) + self.log.info('===================================================') + os.chdir(self.build_dir) cmdline = ['bitbake'] - if args: - cmdline.append(args) - if cmd: + if self.bitbake_args: + cmdline.extend(self.bitbake_args) + if bitbake_cmd: cmdline.append('-c') - cmdline.append(cmd) + cmdline.append(bitbake_cmd) if isinstance(target, list): cmdline.extend(target) else: @@ -124,9 +166,9 @@ class CIBuilder(Test): try: path.find_command('bitbake') except path.CmdNotFoundError: - build_dir = self.params.get('build_dir', - default=isar_root + '/build') - self.init(build_dir) + self.log.error("Broken test implementation: must call init() before " + "using getlayerdir()!") + raise output = process.getoutput('bitbake -e | grep "^LAYERDIR_.*="') env = dict(((x.split('=', 1) + [''])[:2] \ for x in output.splitlines() if x != '')) -- 2.30.2