From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7176574728906211328 X-Received: by 2002:a17:902:e54f:b0:189:6a7f:3046 with SMTP id n15-20020a170902e54f00b001896a7f3046mr65079501plf.88.1671548934976; Tue, 20 Dec 2022 07:08:54 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:902:bd86:b0:178:3881:c7e3 with SMTP id q6-20020a170902bd8600b001783881c7e3ls13780520pls.11.-pod-prod-gmail; Tue, 20 Dec 2022 07:08:54 -0800 (PST) X-Google-Smtp-Source: AA0mqf4SS6yAGnrHbcCXzMXFq82qkPHGd9Sy8OgD050lj2umVxYZbtHn994EYPZ8OeN8ZOa2y2e8 X-Received: by 2002:a17:903:2449:b0:190:c219:ae1e with SMTP id l9-20020a170903244900b00190c219ae1emr41471971pls.31.1671548934031; Tue, 20 Dec 2022 07:08:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1671548934; cv=pass; d=google.com; s=arc-20160816; b=KaWVCBkCWvTMRUaHcLjBbJJWGOSv5fwg7ohBf1ldAHnQJdn6URFElasCa5vivQ0x4V AMrrut3fuW1TaY0AG4FjK4fRu0XLHhe2s//D65vF/uHU6Dz6S2/CJhtlTocTtyp5tHlZ M87bTu2OookvReQ2b0rl1pk9bJj4vKf/j9Ipx4z0sL2fUduYntZfWyrWg5Rg4dce66NP co2KOPt6Qy0irzVNzez1orKGwc5A1k60VRdxYwnAAq4szSzGSI1mjJs3BCDc0y2apDVx pmW8mM94d2+AF9ZWTatqnuK5AnJjxSS62qXZXF8Q3omi2xQDQHCWlwurZ7UEOL5Kr4YC kqnQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=3aSL3D86Y1vsd8f6kELyB/HSaEUX7rg68d6xGnNdo6U=; b=cQ5PPGzoVpWkbqOLTxTaYgQ4q8P+8oX91mE/GnWd0bnNHEip15eUxjwC2xTj4Bfy8G jv5xevTUiV0yjSJ7qVkP5DCrlQx+qi6vgCeFa10V/wITlxyNTmij5Ey/vil1ZGynWvy6 TiHcjan2BXLgQK0ELFFaWVx1E9XO10F1VZtYw854ufnNs2mpepzn6RKaXDaP79WPL80v 0LJf+EBqLi8uNqFNgfLX0kg+K+8RBJKrSjQuzoY48Ww5WEo0HG2y0CyXMJxUzYZFY4xK YVEFEBTErHcD0OYHZCF7BP3aZZPseBzWOGOkH0QihF34z7G1OT5uexagBZBGecYjgTwX Jj6w== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=lkTZDaC4; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.43 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2043.outbound.protection.outlook.com. [40.107.22.43]) by gmr-mx.google.com with ESMTPS id o17-20020a170902d4d100b00188a88cc62fsi1355451plg.12.2022.12.20.07.08.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Dec 2022 07:08:53 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.43 as permitted sender) client-ip=40.107.22.43; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=lkTZDaC4; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.43 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZcU+MCvT8phoqIQzcU08a8K8pu4oovHuS1kc7RNhthCyTljqRlwZQ5WK9S55RhTD/neDYXAPSBSPR7ZjUnNFe8AP94wdiEEep6ZWsSBfxql1EU8M7NhA66S1uW9PytSktIvIECAH4hR4ZIkm3B/Z60YNXUO5MIQ9IrL4FM2j9kTz2ndgpsDJ8WU30NFrusoZRlWO/0T8/ToqS+Ocb78mvjwbpf3V+rimQndx2LxYbgYUrUNKZlnd/uM5YnId4wBSCsroBj8MfJVZl52+qbYckQfe4AxWHlzQzDHOEiWF2BjN84PR5nil05+iIxiLi4yQsTj8z5/h85CQllEkOOo2VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3aSL3D86Y1vsd8f6kELyB/HSaEUX7rg68d6xGnNdo6U=; b=Fxusvo5aYLWedOjZoXGnFa1vLD9LLKi3u2D+u5kC7Oy/M8kPLoViD+zjuYPUVI2YZYXfsncYPYwX+eMRt2PSPxdDHQWgrPJmsyPPNfCbSkOmEZwDenQWgbbwg4Lxteiolwi9k0s3NYRPnXs91feomXXr+cMotzlxOEVDAxygdlxTxlqHWOuZETYu5VZyWuZEQFE/RHYFEGF/gX+11eCxKuaO3KV3eqZ5R0NJ+br/yd673NRqn5o0f5pnLfbwK1tokWNqI1BojDgGvaAfKBGMd9Z+TxDIZDX8U75m4uHjHzN7qQ0b0AuOf4kt1VZ2O0RbfxqoAL23h8xARBl3Zjzidw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3aSL3D86Y1vsd8f6kELyB/HSaEUX7rg68d6xGnNdo6U=; b=lkTZDaC4C0Y2TQX7krILobPyYDsFNpWgEP4QVj+KGs75pgPOzGcFeVL8ByQuLisGDURicMHEIwnjBA51m80LFxdIcODc6XsIOLhkq5n3x/dUPfKgVvelmiJby/YkVu2yH5Qjrj+J+3OFy/cZxN4wkaBLVBvzzmV70R+XzpxNlpxxqHHFhudTxGerhYB5vxTKgQ0lhH5MzNg2gFfWTtX9nELz0zBjxfdNRK9hnWErqR5e1bvdA7ZxTZivpYzParbY07UFU5gO7LCl/7kJp213mfp4c+9JnVnGCyNRfRuErDudejPSW6WxGzPuoZx+efp7m3akig4xkzHC32+oC8QFfQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by PAWPR10MB7389.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:2e6::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5924.16; Tue, 20 Dec 2022 15:08:51 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79%3]) with mapi id 15.20.5924.016; Tue, 20 Dec 2022 15:08:51 +0000 Date: Tue, 20 Dec 2022 16:08:49 +0100 From: Henning Schild To: Uladzimir Bely Cc: isar-users@googlegroups.com Subject: Re: [PATCH v4 0/5] expand-on-first-boot btrfs and CI testing Message-ID: <20221220160849.2726fd9e@md1za8fc.ad001.siemens.net> In-Reply-To: <2618987.lGaqSPkdTl@home> References: <20221213101509.535-1-henning.schild@siemens.com> <20221213112526.3e2bcae8@md1za8fc.ad001.siemens.net> <2618987.lGaqSPkdTl@home> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR0P281CA0068.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:49::13) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|PAWPR10MB7389:EE_ X-MS-Office365-Filtering-Correlation-Id: 8248b6ce-4f1b-4515-b5eb-08dae29c1a96 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tChB88MFZGevM3fmxkiCpH77RWsuerKlN+f4D6jN/RSfuOyI94DVq6BrJfSBhkSeNPlunQGN17X3P5w05DEPZvgBCvzy78wjVrJEa2bZY+VJQ5QaDu9wSbEClN91KkhP6Nwi+VZHZV22rkvsYl3t5h2V8UqfRnxBIiG6SldCAeTRypmbpuBs9T+qlaepXYapRhQqcHbn1LfwezRPzVkQGfxSKNccnvRa9KysGO3IHqfl0BOX2tSCLWbh9JDrOgnifQSSGh8Pg82MooALkhS2Sfm+VU1EtHlHlbeLHUeFflGCyH7wErmeSaDLvUgDPDfQ4ldiPOCJV8yGzxaQqVMO+LEmeOlrMR2r2hUn8yE/ocI/lzJIneGgl/Zd+INqn/4gaIzZF4z4oxluOnGSwe4+MRf4GJ0z6E9AZamAVTUl0tiWTEc1r9P+fah+MDkZ5UfQq9p/1Hfinq2CJGJdqt5U4iRGjry+WfvozMcrLgWI39e4re7TfsPLNXFWM3vyQ/2L0zelCHqr5a3vAS7PFm5nHH9sk7jdx9fDUK3DZf/YwtYLa0Z3a+I+O3410TZVXMLVcpykVg5p+O5oRcz8VGxpJbi9v2PHlblgcio8Pjr5U8f/fZynoNaNrD23v4ubI+9w6Ub80NdNxoDxCy0cmrehmUouVWfxXmZwYa9mW1nYhf0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230022)(4636009)(366004)(136003)(346002)(376002)(396003)(39860400002)(451199015)(9686003)(966005)(478600001)(6486002)(2906002)(6916009)(316002)(83380400001)(38100700002)(82960400001)(86362001)(1076003)(6506007)(6512007)(186003)(26005)(8676002)(4326008)(66946007)(5660300002)(8936002)(66476007)(41300700001)(66556008)(44832011);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?bm9K3AWZ3fChFmVaDmQf8i+P7YPX1aeVl2QerlVX0C/Sz7qnDX17V3dtf1n6?= =?us-ascii?Q?2iZ9STsaYLmCD0SZCOdL651HJBWNIQY0QE/MdUNGz4aYgNefbC3Msox/+I7p?= =?us-ascii?Q?qUq1lryo80lfBIGwBEsh5tq/95+rPywiNvH8pJjQ1zmJKwEE6SPEPPpEswbv?= =?us-ascii?Q?28e3yKDmtQp6JtKLwxcHOSFwZBhrY0BFDo8kX00sDe7zuT6Xhu7g9dwNGwc9?= =?us-ascii?Q?e2KZM/A1qmuTHeO0SBSi7Za+bKfYQhM+TXYZ5RJUHW5Vrvyz6HSN+L79XqU8?= =?us-ascii?Q?PIMYpnRW48tvi7wQui3G9oZjxCM7z1KDm9dtGnO0e3mbMZrbC/2PNKeWviD/?= =?us-ascii?Q?r5l44S1YJ1avdGEP9VuYmSOtVWdWpT3i4R9d0ofGcLQem48k2xZ/YFVKEyBr?= =?us-ascii?Q?N1To/WzFKdJUZp+y3XHA6NInzB414sCnqqzL1LG2xO9XhJWpIUlfTrio0QKy?= =?us-ascii?Q?59OMPGuFW9nhimFFtJkzvUtO8m98X3VzyydZlde6GAFpujHQAxzMkL7R5Ze+?= =?us-ascii?Q?HB8Mn2ZdBXFtEZW+2Bj6TW2VeGOQU7HSN7XOQTMPhMfe+LHbPNCueOGy2CxO?= =?us-ascii?Q?BJJkaeHigtzC9z+2XJx07g9eLHgyFIiwfqv6im8l8iLK4ecO0FYHohmMZXst?= =?us-ascii?Q?7jb9U8GLRBW4CniDFAt2o5I71FenbOYoi2eMHHiaA14qA7qc6JaQQQz+tg1T?= =?us-ascii?Q?TxBGuhrelAq3BDtPwg+qY2C3cj6CbNbVlUJ/0RVvJ+eo1lBaP13XyOUJRIcV?= =?us-ascii?Q?DawGFVOmjx00yvhFo9BSVIqztjpKLJx6uIk4dkmbADq0jWMfFixZ91xGFvi4?= =?us-ascii?Q?iCfgf472KPlxbOjf3eL48ufPQwEAC6QPNn+J+6MB89aWwDtmI5BROK5Ieqww?= =?us-ascii?Q?UaihqgD2PJmz2IPkDxgUSPGV+8oDw6U5BEDT+hu4c3eNa4Zx2pYLzq+ikLsX?= =?us-ascii?Q?0uZkTXuSVjrecqNCf+mCHzRqQv3QXf0iJCS/7BGkkl2vVzLZ4f5nij7KE//c?= =?us-ascii?Q?CLOOtGL9hM0LhNsSH5/U8Hw+nOa9DF184qb4gm7x8fzENvPL1n7HE/CLCuC4?= =?us-ascii?Q?veLndGat7p4M9sT9r5CQjtsZgR9g9YoWrrP1W0Y4J2g83TGeTYL9pTsQO/py?= =?us-ascii?Q?m4ycRf8v9qAT3MwPk8hQBOv+Mvdi7ZBuGPoX5liG4c6uBDTjHFAoLgN+W2Za?= =?us-ascii?Q?TL1jjwbdsvZ6EZY02R3F+7XnWBPc4L2fGLGUstu8acrf7+EvwdZ7lWcsGklB?= =?us-ascii?Q?lr2YOAuGjrDhSxp07M1GCz0Fga50LJ/SqscQlQWQC4AQF2DWGZM3W4t4MCaL?= =?us-ascii?Q?Ux0LfdCPyNnJivnPEjzcWuRrCWpy888E71XCQGkZUzIuxSMEDR4PFoOM7un2?= =?us-ascii?Q?bmGXvgUr/GCbbMq61Jj9D6pr/QSNVDy7FxiMoLCqsfXel0FnK8VgrR6mA9Fs?= =?us-ascii?Q?gBGsaTPGv8YWG4DMheIJQK2q1V9USQiQ2BJLHELSYr1vDP4zkloLmnisCEuh?= =?us-ascii?Q?js0tjHtIDE+xeENub0atolPOUYeEUVoFSe7lxb4Zh39OJ7IRq1Q9gwOJaJFO?= =?us-ascii?Q?0jdOqP4rxWBktcAswzmFz53c7rD2vUtchO9cZbaBKeMq8YOGOIrbhOiv5hCr?= =?us-ascii?Q?pg=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8248b6ce-4f1b-4515-b5eb-08dae29c1a96 X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2022 15:08:51.6300 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: B4a/9YpjoVfjLbW+0NIwX892Vr5B58rphqNoaZaYFfOtDGT0hfMSLgNePZFluD2MJHfS6satXAAv/7ut4EUry8Lj1p2rplEkB5vHgg2GwuA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR10MB7389 X-TUID: THg0Q6YksQnb Am Tue, 20 Dec 2022 12:37:24 +0300 schrieb Uladzimir Bely : > In the email from Tuesday, 13 December 2022 13:25:26 +03 user Henning > Schild wrote: > > Hi all, > > > > i decided to merge the re-introduction of btrfs with the CI series i > > had going on. Because that is all so important yet seemingly fragile > > that we really do not want any big change without CI. > > > > this is based on > > "expand-on-first-boot: switch back away from systemd-growfs" > > which should be merged because it breaks several machines like rpis, > > with the udev race we found lately > > > > It is tested manually in qemu, especially debian-bullseye (btrfs > > candidate) and on that formerly affected rpi. > > In addition i pushed it here to trigger CI runs at ilbers already: > > https://github.com/henning-schild-work/isar/tree/henning/ilbers-ci > > > > I get the feeling we really need a way to log in to the images > > running in CI and run test code there. Only waiting for the login > > prompt and hoping for bits in the kernel log is not powerful enough > > do perform real tests. That is something to improve at some point. > > > > Hello Henning. > > We are currently working on the downstream that requires such kind of > access to the machine running under qemu. We want to implement the > essential parts of functionality in Isar in the nearest future: > - remove scripts/start_vm in favor of testsuite/start_vm.py (or, at > least, scripts/start_vm will be just a wrapper) > - allow custom image names (not only hardcoded 'isar-image-base') to > be tested > - support checking the image for service status (at least) > - support any custom commands execution (at most) > > Actually, the latest thing should also cover previous "service status > checking". Also, when implemented, we will be able to completely > remove boot log parsing that is used currently. > > The main idea of the upcoming implementation is executing custom > commands over SSH in "test" thread while the machine is kept running > in "main" thread. For a truly generic way i strongly suggest looking into lava or tbot2. We use lava in many downstream projects. For the mainly CI topics that might be overkill, maybe OEs scripts might be the better option. Not sure how they do that but for some of the fstab changes we got merged there lately we had to adopt test code that was running in the context of the newly assembled rootfs. Henning > > regards, > > Henning > > > > Am Tue, 13 Dec 2022 11:15:04 +0100 > > > > schrieb henning.schild@siemens.com: > > > From: Henning Schild > > > > > > changes since v3: > > > - rebased on "expand-on-first-boot: switch back away from > > > > > > systemd-growfs" > > > > > > - implement btrfs resizing and CI testing it > > > - put the padding wks entry into an include > > > > > > changes since v2: > > > - add python namespace prefix for the bitbake get functions > > > - install only in images that use WKS_FILEs that have the "gap" > > > > > > changes since v1: > > > - complete rewrite to not focus only on the tests but rather the > > > > > > feature for all examples > > > > > > This first was only about testing but meanwhile has a focus on > > > enabling the expand feature in all example images anyhow, and > > > later look for traces of that happening in CI test where we can. > > > We install the application in any image and in the wic images add > > > some scratch space at the end. That will enable improved > > > interactive use of all those wic images, so makes sense apart > > > from any testing. Later use the log reading feature of the qemu > > > tests to look for traces of successful resizing for non ubuntu > > > images. In ubuntu the console_loglevel is not verbose enough to > > > find traces. > > > > > > Henning Schild (5): > > > expand-on-first-boot: support resizing a btrfs > > > meta-isar: introduce an example to use btrfs > > > CI: improve cibuilder readability > > > meta-isar: install expand-on-first-boot in most images and add > > > space CI: expect a message about filesystem resize vom expand > > > script > > > meta-isar/conf/machine/qemuamd64.conf | 6 ++++ > > > meta-isar/conf/machine/virtualbox.conf | 1 + > > > meta-isar/conf/machine/vmware.conf | 1 + > > > .../conf/multiconfig/qemuarm-bookworm.conf | 1 + > > > .../lib/wic/canned-wks/expand-padding.wks.inc | 5 +++ > > > .../lib/wic/canned-wks/sdimage-efi-btrfs.wks | 11 ++++++ > > > .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 ++ > > > .../lib/wic/canned-wks/sdimage-efi.wks | 2 ++ > > > .../files/expand-last-partition.sh | 35 > > > > > > ++++++++++++++++--- testsuite/cibuilder.py > > > | 28 ++++++++++++--- 10 files changed, 83 insertions(+), 9 > > > deletions(-) > > > > > > create mode 100644 > > > > > > meta-isar/scripts/lib/wic/canned-wks/expand-padding.wks.inc create > > > mode 100644 > > > meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-btrfs.wks > >