From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7159568422443941888 X-Received: by 2002:a2e:a602:0:b0:264:5132:f59 with SMTP id v2-20020a2ea602000000b0026451320f59mr9700568ljp.0.1667394970240; Wed, 02 Nov 2022 06:16:10 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:5d3:b0:4a2:3951:eac8 with SMTP id o19-20020a05651205d300b004a23951eac8ls4270331lfo.0.-pod-prod-gmail; Wed, 02 Nov 2022 06:16:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6EjZNpwDgErWOZgpf9h4iq5j1YFuP4U8/RTQusutzoMtb56vOg0Qd37Wa36Z+6HRS78fbu X-Received: by 2002:a05:6512:3b95:b0:4a2:7409:b2ac with SMTP id g21-20020a0565123b9500b004a27409b2acmr8723396lfv.448.1667394968689; Wed, 02 Nov 2022 06:16:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667394968; cv=none; d=google.com; s=arc-20160816; b=X21XRwoTIGGwPvpegu1sIjaSpjn28O+mS/zvjXLZt8wJ76uGJ3dJWoJ9KUlrymxIP0 t0a3at3rF0kcEYOoRj8sMyblcv76Zwjn0PrgtoW9m8DPmVbt3X6gcLPGQ35dve7CUjnf Bbyrt+7hTJ6HXimgZxbmmuEc4nZnV7knUCWClhHeBBFDOfMBmVZ5nwMXKpLWLX8j1L7A JDSMlRGSY4bINyA5IBxqVa02ZB2Wx5+UsMPfJr6bp55aXfbs4MgK9aC1aKeEGQ16GHKC 86LvXZCSisL0bXaSVmuX0udeSJhDv/c1LlR1SBfsg9n4sUyR/Big8hVUW7Sp7QTixp8W Eenw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=C/6qVqzrS7mp9UDauQXVkqgTHFfZ/5sTEY4UCb+XVec=; b=Z5QRFNZ3wYVzNJYOnfxyKgwukCAQvGMbO39Q5SAGgUxAMa8NWJ9KBmo6r0Rj/cF9g7 ZX70xnQ55CRLfDw695Kny3Q3D8qERcYxw2sVhyM/LgEw/l5B1q8L64B9wbjjafdsZ75K Wk4TZUYe20cdIP+5bS4A/4I8GIg+aacfvukvG4VBQJEftfDWOXUiSYzza/yFdMyOxrUY KU6WZSvs/Egy1qMp+4YWRUjnUqw4UADrkP2Cg+tEHiY+wlSybNPj50klADZ2WU0d0wd0 0iHcQlwqOKLZ4t/vZ+Hl2bXorv/BBdh0ZKxB/A8pC3JlFYMVTd0Z364Bso9MvR9HEefN DQgQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id p16-20020a2ea4d0000000b002772ff50da3si503335ljm.7.2022.11.02.06.16.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 02 Nov 2022 06:16:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2A2DG69O028652 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 2 Nov 2022 14:16:07 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com, Henning Schild Subject: Re: [PATCH 2/2] CI: install expand-on-first-boot in all tests and also test Date: Wed, 02 Nov 2022 16:16:04 +0300 Message-ID: <5420887.rdbgypaU67@home> In-Reply-To: <20221028163230.58f4be4d@md1za8fc.ad001.siemens.net> References: <20221028142145.4428-1-henning.schild@siemens.com> <20221028142145.4428-3-henning.schild@siemens.com> <20221028163230.58f4be4d@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: z7q0DRDHyU4c In the email from Friday, 28 October 2022 17:32:30 +03 user Henning Schild wrote: > I am not sure i like this one. Keep reading inline. > > Am Fri, 28 Oct 2022 16:21:45 +0200 > > schrieb Henning Schild : > > Install the package in any rootfs we build. And also resize a few > > images when we boot them and check the output. > > > > Signed-off-by: Henning Schild > > --- > > > > meta-isar/conf/local.conf.sample | 2 +- > > meta-isar/conf/machine/qemuamd64.conf | 1 + > > testsuite/cibuilder.py | 18 +++++++++++++++--- > > testsuite/start_vm.py | 11 +++++++++++ > > 4 files changed, 28 insertions(+), 4 deletions(-) > > > > diff --git a/meta-isar/conf/local.conf.sample > > b/meta-isar/conf/local.conf.sample index 57d062025f21..27becf19aefd > > 100644 --- a/meta-isar/conf/local.conf.sample > > +++ b/meta-isar/conf/local.conf.sample > > @@ -198,7 +198,7 @@ CONF_VERSION = "1" > > > > # > > # The default list of extra packages to be installed. > > > > -IMAGE_INSTALL = "hello-isar example-raw > > example-module-${KERNEL_NAME} enable-fsck isar-exclude-docs samefile > > hello isar-disable-apt-cache cowsay example-prebuilt" +IMAGE_INSTALL > > = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck > > isar-exclude-docs samefile hello isar-disable-apt-cache cowsay > > example-prebuilt expand-on-first-boot" # # Enable cross-compilation > > support diff --git a/meta-isar/conf/machine/qemuamd64.conf > > b/meta-isar/conf/machine/qemuamd64.conf index > > eca2628c4ae5..6e3561901447 100644 --- > > a/meta-isar/conf/machine/qemuamd64.conf +++ > > b/meta-isar/conf/machine/qemuamd64.conf @@ -19,6 +19,7 @@ QEMU_ARCH > > ?= "x86_64" QEMU_MACHINE ?= "q35" QEMU_CPU ?= "" > > > > QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE## -bios > > > > /usr/share/ovmf/OVMF.fd" +QEMU_DISK_RESIZE ?= "1" > > I hate that i need this. But it seems to be the way to talk to the > testsuite. Has nothing to do with the machine otherwise. > Probably, it is better to place this in local.conf using machine/distro overrides. So, just selected (overrided) targets would be affected and ubuntu won't require any workarounds. > > MACHINE_SERIAL ?= "ttyS0" > > BAUDRATE_TTY ?= "115200" > > > > diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py > > index 07a9edc56d9c..fb0daedc7d49 100755 > > --- a/testsuite/cibuilder.py > > +++ b/testsuite/cibuilder.py > > > > @@ -231,6 +231,15 @@ class CIBuilder(Test): > > login_prompt = b'isar login:' > > # the printk of recipes-kernel/example-module > > module_output = b'Just an example' > > > > + # output we see when expand-on-first-boot runs on ext4 > > + resize_output = b'resized filesystem to' > > + expecting_resize = False > > + for arg in cmdline: > > > + if arg.endswith(".wic.resized"): > Here i do not have that magic variable. So i guess from the filename. > cibuilder.py imports start_vm.py, which has some helper functions to get variables from "bitbake -e" output parsing. So, technically the variable should be available through get_bitbake_env/get_bitbake_var functions. > > + # in ubuntu the resize works but no trace in boot log > > > + if not 'ubuntu' in arg: > same here, i guess from the filename which distro my suite is, while > there is a variable distro ... which is a suite > > There is a similar guess somewhere in the code > > >> base = 'ubuntu' if distro in ['focal', 'bionic'] else 'debian' > > > > + expecting_resize = True > > + break > > > > timeout = time.time() + int(time_to_wait) > > > > @@ -265,8 +274,11 @@ class CIBuilder(Test): > > with open(output_file, "rb") as f1: > > data = f1.read() > > > > if module_output in data and login_prompt in data: > > - return > > - else: > > - app_log.error(data.decode(errors='replace')) > > + if expecting_resize: > > + if resize_output in data: > > + return > > + else: > > + return > > + app_log.error(data.decode(errors='replace')) > > > > self.fail('Log ' + output_file) > > > > diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py > > index f761a8bda789..2262af557a32 100755 > > --- a/testsuite/start_vm.py > > +++ b/testsuite/start_vm.py > > @@ -5,6 +5,7 @@ > > > > import argparse > > import os > > > > +import shutil > > > > import subprocess > > import sys > > import time > > > > @@ -57,6 +58,7 @@ def format_qemu_cmdline(arch, build, distro, out, > > pid, enforce_pcbios=False): qemu_machine = get_bitbake_var(bb_output, > > 'QEMU_MACHINE') qemu_cpu = get_bitbake_var(bb_output, 'QEMU_CPU') > > > > qemu_disk_args = get_bitbake_var(bb_output, 'QEMU_DISK_ARGS') > > > > + qemu_disk_resize = get_bitbake_var(bb_output, > > 'QEMU_DISK_RESIZE') == "1" > > > > if out: > > extra_args.extend(['-chardev','stdio,id=ch0,logfile=' + out]) > > > > @@ -65,6 +67,15 @@ def format_qemu_cmdline(arch, build, distro, out, > > > > pid, enforce_pcbios=False): if pid: > > extra_args.extend(['-pidfile', pid]) > > > > + if qemu_disk_resize: > Here i smuggle in a file copy in what should just give me a cmdline for > qemu. But is is the only place where all information seems available. > > And i leave that magic copy forever, wasting space and use a magic > suffix for the caller to know to look for log traces of resizing. > > As i said. I am not happy. But wanted to show that for discussion and > maybe get feedback on how to do things better. > > The patches ran successfully on ilbers-ci. It might not be pretty, but > it works. And might help us get that expand thingy more stable and keep > it stable. > > Henning > > > + suffix = ".resized" > > + abs_src = os.path.join(deploy_dir_image, rootfs_image) > > + abs_dst = abs_src + suffix > > + if not os.path.exists(abs_dst): > > + shutil.copy(abs_src, abs_dst) > > + subprocess.run(["qemu-img", "resize", abs_dst, "+2G"]) > > + rootfs_image += suffix > > + > > Is this ".resized" image just a something allowed to be "spoiled" by qemu in order to have original file untouched? If yes, it might be OK, but if it's just a hint for the code above, I would prefer to remove this code and parse "bitbake -e" in "cbuilder.py". > > qemu_disk_args = qemu_disk_args.replace('##ROOTFS_IMAGE##', > > > > deploy_dir_image + '/' + rootfs_image).split() if enforce_pcbios and > > '-bios' in qemu_disk_args: bios_idx = qemu_disk_args.index('-bios') -- Uladzimir Bely