From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7205110283261444096 X-Received: by 2002:a62:cdcc:0:b0:590:3183:60eb with SMTP id o195-20020a62cdcc000000b00590318360ebmr687724pfg.3.1677570464729; Mon, 27 Feb 2023 23:47:44 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90b:224d:b0:233:cf4f:61f4 with SMTP id hk13-20020a17090b224d00b00233cf4f61f4ls16983517pjb.1.-pod-canary-gmail; Mon, 27 Feb 2023 23:47:43 -0800 (PST) X-Google-Smtp-Source: AK7set/yixuNwHe9t+1Ypy79Ql6XjK7c37tUsUl5bheBDwzYU5AlTuWODP5EQR2yOKC82sxQ0TNT X-Received: by 2002:a17:90b:3e86:b0:234:b964:570d with SMTP id rj6-20020a17090b3e8600b00234b964570dmr2469328pjb.13.1677570463860; Mon, 27 Feb 2023 23:47:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677570463; cv=none; d=google.com; s=arc-20160816; b=x0+MM79ERsrUjoJySYjpLoqo33hRxrcogwBTwXyZ0n2qo+kAMzHb2KpI9RWo3ld1U0 q1X4/YKyeg69EcIHkvQWrpRCxO2vcgbRCp1cep510stQDqx6Xj4OHo3xSE0BB9spUIXl qulLxB2NwZ39bdqA3WVpKBo8Dkk8/qd+C0YEKpWdoFOdJbeMla4DgklXKHzcgidBXY2m tKFfkcrAyyydKD1axv716qBJsrWTXdh/j4ySXAwtJwTDqwQH3FJR/Op1JF+sM+5i7A2c JijDqylHu6O/OkOKXrhsM3VZkRt0cZVNb2dfQD9UhbTxoEC/dRbyRYjSZtboT14uApSE Kdkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=vw6uLbjMnc7cq/oVJ+2TOja22i8lBQK76j811Xu0HeY=; b=T+0UlQz7IOWBbO+BNR2x17pK6vgfT9cUOZaHsAhKAOODS677HWS4vnQzaTC0Huynzn O4oZxHniCnAbZu2EDzQFGaR09UOJv5wGJ2Py6SHCkAjLnHTiRJEltWeeWtXMc5z//mCN 4QMLYLjRXWUculHDAdE17dWYT5huJqPNAWwnY0gi5PRayBfIVNsc90YarlQ4SSB3lfUH YJcDqqbbMSCmSlTDKJZqzHmr4a2UwB5ZdN5GCqvHE4i3VI5gonfBo68orSq4h7N0SaOB rYBVWK+U2tSiUMKlNKDMEEvKOkuBUXbWdiuVGGBsZ3qt+UxQr0r/dpm3LSfZfTAzGATs 3Xqw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id v3-20020a17090a6b0300b002373b032314si64665pjj.0.2023.02.27.23.47.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 27 Feb 2023 23:47:43 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 31S7lcoP010874 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 28 Feb 2023 08:47:39 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH 3/3] testsuite: Document custom testcase for downstreams Date: Tue, 28 Feb 2023 08:47:38 +0100 Message-Id: <20230228074738.16304-4-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230228074738.16304-1-ubely@ilbers.de> References: <20230228074738.16304-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: HdOLrmV55Ulb Provide a complete example of testsuite implementation on the downstream side in their own testcase directory. Mention `PYTHONPATH` that needs to be adjusted. Signed-off-by: Uladzimir Bely --- testsuite/README.md | 101 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) diff --git a/testsuite/README.md b/testsuite/README.md index 476b8607..28036bcb 100644 --- a/testsuite/README.md +++ b/testsuite/README.md @@ -122,3 +122,104 @@ And to execute this example: $ avocado run sample.py:SampleTest.test_sample ``` +## Using a different directory for custom testcases + +Downstreams may want to keep their testcases in a different directory +(e.g. `./test/sample.py` as top-level with test description) but reuse +classes implemented in Isar testsuite (e.g. `./isar/testsuite/*.py`). This is +a common case for downstream that use `kas` to handle layers they use. + +In this case it's important to adjust `PYTHONPATH` variable before running +avocado so that isar testsuite files could be found: + +``` +# TESTSUITEDIR="/work/isar/testsuite" +export PYTHONPATH=${PYTHONPATH}:${TESTSUITEDIR} +``` + +# Example of the downstream testcase + +Let's assume the downstream builds `custom_image_name` image for `qemuarm64` +target in order to running custom tests using qemu. + +Let's assume the [kas](https://github.com/siemens/kas) is used by +the downstream with the following directory structure: + +``` +... +├── build/ +├── isar/ +├── kas +│ ├── qemuarm64.yml +│ └── kas-container +├── meta-custom-layer +└── test + ├── run_test.sh + ├── scripts + │ └── sample_script.sh + └── sample.py +``` + +### test/sample.py: + +``` +#!/usr/bin/env python3 + +from cibase import CIBaseTest + +class SampleTest(CIBaseTest): + def test_sample_script(self): + self.init("/build") + self.vm_start('arm64','bullseye', image='custom_image_name', \ + script='sample_script.sh') +``` + +### test/run_test.sh: + +``` +#!/usr/bin/env bash + +# Make Isar testsuite accessable +export PYTHONPATH=${PYTHONPATH}:${TESTSUITEDIR} + +# install avocado in virtualenv in case it is not there already +if ! command -v avocado > /dev/null; then + sudo apt-get -y update + sudo apt-get -y install virtualenv + rm -rf /tmp/avocado_venv + virtualenv --python python3 /tmp/avocado_venv + source /tmp/avocado_venv/bin/activate + pip install avocado-framework==100.1 +fi + +# Install qemu +if ! command -v qemu-system-aarch64 > /dev/null; then + sudo apt-get -y update + sudo apt-get -y install --no-install-recommends qemu-system-aarch64 ipxe-qemu +fi + +# Start tests in this dir +BASE_DIR="/build/avocado" + +# Provide working path +mkdir -p .config/avocado +cat < .config/avocado/avocado.conf +[datadir.paths] +base_dir = ${BASE_DIR}/ +data_dir = ${BASE_DIR}/data +logs_dir = ${BASE_DIR}/logs +test_dir = ${BASE_DIR}/test +EOF +export VIRTUAL_ENV="./" + +tsd=$(dirname $(realpath $0))/scripts + +# Run SSH tests +avocado run --max-parallel-tasks=1 /work/test/test.py -p test_script_dir=${tsd} +``` + +Running testcase using `kas shell` may be done as: + +``` +./kas/kas-container shell kas/qemuarm64.yml -c '/work/test/run_test.sh' +``` -- 2.20.1