From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7176574728906211328 X-Received: by 2002:a0d:d944:0:b0:3c7:edf8:6eb with SMTP id b65-20020a0dd944000000b003c7edf806ebmr132296ywe.9.1671618334219; Wed, 21 Dec 2022 02:25:34 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a81:57d7:0:b0:393:f144:5d4d with SMTP id l206-20020a8157d7000000b00393f1445d4dls8721149ywb.5.-pod-prod-gmail; Wed, 21 Dec 2022 02:25:33 -0800 (PST) X-Google-Smtp-Source: AMrXdXsO7G4ytkD2VgMVoRbIs8NfPU66ciyrgUCCi3waRs13Z/bnNxnSe2DiEZW4HkLz8ph62/+y X-Received: by 2002:a0d:dd88:0:b0:45b:9813:5daf with SMTP id g130-20020a0ddd88000000b0045b98135dafmr825509ywe.45.1671618333462; Wed, 21 Dec 2022 02:25:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671618333; cv=none; d=google.com; s=arc-20160816; b=P35cM0E4aPDisvNyOrdsBNHRElkx5Nt/xyQC8IzmysqIfoHv3i0FGb+71ggHT0ZPGy 6Z137YcExnKTHA2vcliYd7cgdXXgevoa93JwyOGfk1OCYHkgipQ9Ucm+CBj8b0otULBT GisX888ym+HBfur3cUC7mld93xFtPL7qWdwU6D5gCDXGYOQefjsuR3zi8zDCBsKSP0Kv RhufspI3VE5t/yZicdDvTamyH5mjVQIbLeRp+APPXsku556w3UQ+gkzrrqTIIY9zOSAt bHS/KYPPQUWJTbS1IXTrFaNLL7qcllWqpGbVz8OSI1on4A4JkbxGR0l/ykaLoN2WPf26 NdDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=DGu3B51ZT1ClyWDmdR1mQvqcoyhm08sIOOCDjr86Jag=; b=0Wbrq8IYE3bc4mu3hkNNaIweYoknzHQ40MPxhIY2deMvQ3MLEJM+ci3We7BwVSszE1 khcfvFiKnsMO2hAQCQEgWqj9AKr8LQQnT5nvM1Z2h6BqhMl3XDdgdHiRYfpWa6jwW6Lw ar01VpDIctjzXBUBL5Re0MDE3aBg89aPtCp+W1py8g6sPpcb48GK7b4lHMxuFPL8tBmS HReLHVaNLZ0IqQoCgJZOtUxtd/nQ/NTVWDnESxbsTpD/ruPhzC3VDE+uZWza9KkPAvmh Lf6DK76uGZBuNcd1XmyJmDRmR0zLeis6wmwHk5g+aiL4iVodLaR92jPpeZT+5wzVPhBt x24Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id z145-20020a814c97000000b003d82e3c1d09si1290277ywa.4.2022.12.21.02.25.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 21 Dec 2022 02:25:33 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2BLAPUsB005189 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 21 Dec 2022 11:25:31 +0100 From: Uladzimir Bely To: Henning Schild Cc: isar-users@googlegroups.com Subject: Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing Date: Wed, 21 Dec 2022 13:25:29 +0300 Message-ID: <3377421.LZWGnKmheA@home> In-Reply-To: <20221220160849.2726fd9e@md1za8fc.ad001.siemens.net> References: <20221213101509.535-1-henning.schild@siemens.com> <2618987.lGaqSPkdTl@home> <20221220160849.2726fd9e@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 8bzaTG3XkBYr In the email from Tuesday, 20 December 2022 18:08:49 +03 user Henning Schild wrote: > Am Tue, 20 Dec 2022 12:37:24 +0300 > > schrieb Uladzimir Bely : > > In the email from Tuesday, 13 December 2022 13:25:26 +03 user Henning > > > > Schild wrote: > > > Hi all, > > > > > > i decided to merge the re-introduction of btrfs with the CI series i > > > had going on. Because that is all so important yet seemingly fragile > > > that we really do not want any big change without CI. > > > > > > this is based on > > > "expand-on-first-boot: switch back away from systemd-growfs" > > > which should be merged because it breaks several machines like rpis, > > > with the udev race we found lately > > > > > > It is tested manually in qemu, especially debian-bullseye (btrfs > > > candidate) and on that formerly affected rpi. > > > In addition i pushed it here to trigger CI runs at ilbers already: > > > https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci > > > > > > I get the feeling we really need a way to log in to the images > > > running in CI and run test code there. Only waiting for the login > > > prompt and hoping for bits in the kernel log is not powerful enough > > > do perform real tests. That is something to improve at some point. > > > > Hello Henning. > > > > We are currently working on the downstream that requires such kind of > > access to the machine running under qemu. We want to implement the > > essential parts of functionality in Isar in the nearest future: > > - remove scripts/start_vm in favor of testsuite/start_vm.py (or, at > > least, scripts/start_vm will be just a wrapper) > > - allow custom image names (not only hardcoded 'isar-image-base') to > > be tested > > - support checking the image for service status (at least) > > - support any custom commands execution (at most) > > > > Actually, the latest thing should also cover previous "service status > > checking". Also, when implemented, we will be able to completely > > remove boot log parsing that is used currently. > > > > The main idea of the upcoming implementation is executing custom > > commands over SSH in "test" thread while the machine is kept running > > in "main" thread. > > For a truly generic way i strongly suggest looking into lava or tbot2. > We use lava in many downstream projects. > I've looked at some of them and found that lava seems to be used in `xenomai- images`. But I didn't find anything using tbot2. Could you point at some example projects (if they are not closed-source)? And saying "tbot2" did you mean https://tbot.tools/ or something else? At first glance it looks something promising and able to be integrated to our python-based testsuite. While lava looks more complicated. > For the mainly CI topics that might be overkill, maybe OEs scripts > might be the better option. Not sure how they do that but for some of > the fstab changes we got merged there lately we had to adopt test code > that was running in the context of the newly assembled rootfs. > > Henning > > > > regards, > > > Henning > > > > > > Am Tue, 13 Dec 2022 11:15:04 +0100 > > > > > > schrieb henning.schild@siemens.com: > > > > From: Henning Schild > > > > > > > > changes since v3: > > > > - rebased on "expand-on-first-boot: switch back away from > > > > > > > > systemd-growfs" > > > > > > > > - implement btrfs resizing and CI testing it > > > > - put the padding wks entry into an include > > > > > > > > changes since v2: > > > > - add python namespace prefix for the bitbake get functions > > > > - install only in images that use WKS_FILEs that have the "gap" > > > > > > > > changes since v1: > > > > - complete rewrite to not focus only on the tests but rather the > > > > > > > > feature for all examples > > > > > > > > This first was only about testing but meanwhile has a focus on > > > > enabling the expand feature in all example images anyhow, and > > > > later look for traces of that happening in CI test where we can. > > > > We install the application in any image and in the wic images add > > > > some scratch space at the end. That will enable improved > > > > interactive use of all those wic images, so makes sense apart > > > > from any testing. Later use the log reading feature of the qemu > > > > tests to look for traces of successful resizing for non ubuntu > > > > images. In ubuntu the console_loglevel is not verbose enough to > > > > find traces. > > > > > > > > Henning Schild (5): > > > > expand-on-first-boot: support resizing a btrfs > > > > meta-isar: introduce an example to use btrfs > > > > CI: improve cibuilder readability > > > > meta-isar: install expand-on-first-boot in most images and add > > > > > > > > space CI: expect a message about filesystem resize vom expand > > > > script > > > > > > > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > > > > meta-isar/conf/machine/virtualbox.conf | 1 + > > > > meta-isar/conf/machine/vmware.conf | 1 + > > > > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > > > > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > > > > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 ++++++ > > > > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > > > > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > > > > .../files/expand-last-partition.sh | 35 > > > > > > > > ++++++++++++++++--- testsuite/cibuilder.py > > > > > > > > | 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9 > > > > > > > > deletions(-) > > > > > > > > create mode 100644 > > > > > > > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create > > > > mode 100644 > > > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks -- Uladzimir Bely