From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Thu, 25 Sep 2025 08:55:02 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-pj1-f63.google.com (mail-pj1-f63.google.com [209.85.216.63]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 58P6t02c002370 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 25 Sep 2025 08:55:01 +0200 Received: by mail-pj1-f63.google.com with SMTP id 98e67ed59e1d1-33428befc08sf1321968a91.2 for ; Wed, 24 Sep 2025 23:55:01 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1758783295; cv=pass; d=google.com; s=arc-20240605; b=lPY8r6qQHX+ditsbzsxQsbcImxsoxBmh1knwIDfLtcW4cxHJvDIeGWFQgmJptAaLjX W2sMToYYG/BPaYUD5tnmBQ07I4EZ316KxTu0ngjR6lzt22wIGRyqYYS7D/XtQ7NzPB88 7hcP9RnaaYSEJV0vOnRIFWEGwiQI6OIiVZ5sl/ryYEmtcp9z+hWJLC5uXc61MzYLNKxl +ORP7OxNZlicu9wu+XfsoUd7QDdHFpTr/kyPOs4WGeMCXewlwmIgVULCS+2YbeU2X8Mg pNeMBcSCvRVviZz8daM2rss+O5OaCspbipZVp03bYC2q1XtK0mrARSsvQaTFYqSzjYxj wHNQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=08/Ca0V6Y7+rI15jRWfJltWp+WZdM7fpTsDhpVYoPR8=; fh=6glaeKnAldNti+TgSvXqMFjXUGBEkw/6WSroVAr+DZE=; b=aupsLy4eebGrSGQ02oluoIXsbHzPAQUgvrsUfbAgmtpd8vCpCgKz9HDUrPls3qnQfi 193GE1yTMsVnf6mGVXAuGFkYIUNRYnjwssstxsRtXS+CGh/5H/fM+ke/D1dotFFvVCZl j0ku/N79ETs6s/vPaFQ6PhpD92yd4eWAJq7MEqf0dxnSF0yrpvkXN1MMSR8GWljnMXpJ qymc4Y/x0VL67Qucmo9XbX0H809dqsmU8YH5xUzzIqC/ub/7t5mlF+wtChBNiGXgfcbM umuKHdt+8QFOvqrwn3N4HGRQC0ulYnnUQCoj4A5n02dd96dtZYUZoV7gayVEMzZ8W5/4 rtsg==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b="XkOUxEd/"; spf=pass (google.com: domain of fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1758783295; x=1759388095; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=08/Ca0V6Y7+rI15jRWfJltWp+WZdM7fpTsDhpVYoPR8=; b=DvxWAPxwvKqpSGxtZ7f0Dq7dHjvydxNpr6mQYeZdguHIWgHwe4RM/stvgJ1u68+gFc SJlPwL4DfQgmP176WEFCkL9SoCzKzPfgweUo750ltvSjvFkGNpt0OhRZL0QXv3WKbag3 d66nDBcJ3QZHBxzeatvYDdd9u7kK2IisAPd14CFo1BzK85oZl8RNkf65Sv6F8p+rvI9B mYGNxp9E2c0Xq8wmB9GYo50L7vMaUIJNlpnP8zmYokwcj0RjFvfH2kktlFzK84mCS7d0 9kMkqYveWS74c/dknB/pR7dZvtuBgy8qs6nMdu55i8seUMro/mUES7DzoR9R6grm9/FW rk+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758783295; x=1759388095; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=08/Ca0V6Y7+rI15jRWfJltWp+WZdM7fpTsDhpVYoPR8=; b=uowv6OKKzkp0TxTdFGoAZciX2/EznkAYFr5ElzGt/pWFZh0JJpIhAnVtZ0wIDk1vJp 07kb2FnHC4B4y1j4lwRQjpN7qyq2rma3Nn/DU17WN67Td9H2q0asjzc8t2O2/e8Bc8cg SP5V5Woq5cpxXep5VCurmqxGOMvbcRZd/craoBYJm2bOPVGj9COTvI4Pi507JIUbaQYL oQwBSzU87VS7U2xAt97wc7WKCAESQlN2ryYCl7En5WnoY5M2pBrSn9Qw60brD2kUJz2f xzJk0ulSCoRv0YTNnoKpklmzGWdpCd+hlPnnvPEzudeE6yXJ/h+O0EQUJV1oONIguGLR yPcw== X-Forwarded-Encrypted: i=2; AJvYcCUH9RbMZvif2CEddC++hcbfoYO+lWoy73V8MSQR7N8Feo9OxDyZIwtIZP5sOCXb8QQ6qlTY@ilbers.de X-Gm-Message-State: AOJu0Yyps9yR35qNVBPlVY6J8fDkb6UG9TZ6MIEp9+NTREYb+XleWEkj 0UNVi9HjfGyn9DU/hUd4HlPGTJiHFQnvgRvSq/q/FY0v7U9F4buzOXKS X-Google-Smtp-Source: AGHT+IFtrMa2z/Uxu+H3ZA69WCBNSkCQUXoxTaOcbmJSLSoVrIkXLXZQijH/Dj8OXrsxUaMf4qmw8A== X-Received: by 2002:a17:90b:538e:b0:32e:18b2:5a45 with SMTP id 98e67ed59e1d1-3342a20b94dmr2540441a91.5.1758783294866; Wed, 24 Sep 2025 23:54:54 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="ARHlJd5bBqmNejRl/vCMkTcwmZSgncPTsJcupu8b965ddCbsRQ==" Received: by 2002:a17:90b:4ac5:b0:32e:43ae:a453 with SMTP id 98e67ed59e1d1-3342a46b148ls715825a91.0.-pod-prod-06-us; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) X-Received: by 2002:a17:90b:1809:b0:32e:6fae:ba53 with SMTP id 98e67ed59e1d1-3342a20bdd9mr2733937a91.8.1758783284414; Wed, 24 Sep 2025 23:54:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758783284; cv=none; d=google.com; s=arc-20240605; b=SxUK9NZwSO3wwdaurvEBXZH/KgKjK4gMFfo7QKA7hdoxnC9pmoKSvwJ7SxD0RwXmFn HmYjtkX5gIZi1swVhA8nknMlkhGMv8rfpagD8jRNMCr8mjWprgmRZRZR3Nk/iDbg9Adv Tpe1xB2dNt39aYZTd5aoTp1A6Klrcw1KA3rb0bI2P7kqJ1P228yi2cjLgjSgJR35tXhD fV/9CrzBFD56SRp9kKXT/H9MaUeUFLuXzZn9K0wuwSvPyMW8sLo36nkE/pQPhuQgCyoN mbm14IsrivFG+iBfNPIefILk289kpCrefJkld0yOMWEbjncVlAwvEmzSKATt0RHxqJ5F M0RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=yxQOdxi+GJ2vZx2tdncaRSaSCHhlURMpV+33koZIozY=; fh=1z37pEVhqwMLlkT4FDzOga7XBsM6Rzv6sXOq0pipxqo=; b=ZDGWDTWnmxSthzPLvcpEN3c+SVr+2w11mr0IveiTQKbr7mmx2TinUdI9xn4KYzdE3P /d+IdQ0kiqNQ9DGnwfI0gNnMSZlDdXeHzdnR1/W33MnKKF+5dRilxqtCSM9CTkAPXMD1 rsrgbdezU474fM82cEiz/BOawWv258jUxxPMUa61+TRazYyNgKj5NA9ojajMgyFHVQ+G 9E9PIZC9xmfYPnZ8dp1dSukPBXDJXCtpqafvwIEOC0qYqyJRe2VDJQ/ocJF9P9dg+qvo Uu8Hn67sKXZbFT9HuSQUjFM0g6P2nucDNrx3Arsp74XMc02cGYFXDxS72yfCc8fp8/VI SuwQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b="XkOUxEd/"; spf=pass (google.com: domain of fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-64-225.siemens.flowmailer.net (mta-64-225.siemens.flowmailer.net. [185.136.64.225]) by gmr-mx.google.com with ESMTPS id 98e67ed59e1d1-3342a2bd92dsi83296a91.0.2025.09.24.23.54.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Sep 2025 23:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) client-ip=185.136.64.225; Received: by mta-64-225.siemens.flowmailer.net with ESMTPSA id 20250925065442e763ffe25c000207ea for ; Thu, 25 Sep 2025 08:54:42 +0200 From: "'Cedric Hombourger' via isar-users" To: isar-users@googlegroups.com Cc: Cedric Hombourger Subject: [PATCH v4 0/4] non-privileged commands in chroot Date: Thu, 25 Sep 2025 08:54:20 +0200 Message-ID: <20250925065433.4180883-1-cedric.hombourger@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-1212295:519-21489:flowmailer X-Original-Sender: cedric.hombourger@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b="XkOUxEd/"; spf=pass (google.com: domain of fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com designates 185.136.64.225 as permitted sender) smtp.mailfrom=fm-1212295-20250925065442e763ffe25c000207ea-r_w649@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Cedric Hombourger Reply-To: Cedric Hombourger Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED, RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: ckE3I2MST/Wf When building root filesystems for foreign architectures with package source caching enabled, apt operations are executed within the rootfs through QEMU emulation. This results in significantly degraded performance, particularly when downloading source packages sequentially. This patch series introduces a new wrapper function that enables native command execution against a rootfs while preserving special mount points (such as /isar-apt). The approach: - Improves build performance for foreign architecture builds - Maintains filesystem isolation using bubblewrap - Preserves access to special mount points required by isar Additional notes: - rootfs_cmd may be used to run commands from the host root file-system: use with extreme care to avoid host contamination problems. - mmdebstrap already calls apt-get of the host to download packages (in other words, a build of a bookworm image from a trixie host will result in mmdebstrap (from trixie), call apt-get (from trixie) to download bookworm packages. This is the behavior we have before and after these changes. - With these changes and when caching of Debian source packages is enabled/requested, Isar will use apt-get of the host to download source packages (it will however do this from a bubblewrap'ed environment to avoid a non-required privilege elevation; Isar has many but we need to start from somewhere). Testing: - Tested against 9e62337953fbb8371c846c44e8a99d62a8d220ba - Basic smoke tests performed successfully (citest.py -t fast) - Performance improvements observed in source package acquisition - Tested with various foreign architecture configurations Dependencies: - Adds bubblewrap as a new host tool requirement - Uses kas-container 4.8.0 or later (see [1]) Changes since v3 patch: - drop image-postproc-extension patches (refactoring and use of rootfs_cmd). They are not strictly needed and were only meant to provide another potential use of rootfs_cmd. - Rebase changes to RECIPE-API-CHANGELOG.md and added a few extra words about the motivation. Changes since v2 patch: - rootfs_install_pkgs_download will no longer use sudo to run apt-get install --download-only. This was added to further demonstrate/test rootfs_cmd in existing Isar code. Changes since v1 patch: - Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts) - Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes Changes since RFC patch: - Let caller decide where to bind-mount the rootfs to - Make the rootfs argument optional - Support 32-bit rootfs (no lib64 there) Test Results (avocado started from a kas-container version 4.8.1): (01/22) citest.py:DevTest.test_dev: STARTED (01/22) citest.py:DevTest.test_dev: PASS (1132.17 s) (02/22) citest.py:DevTest.test_dev_apps: STARTED (02/22) citest.py:DevTest.test_dev_apps: PASS (845.24 s) (03/22) citest.py:DevTest.test_dev_rebuild: STARTED (03/22) citest.py:DevTest.test_dev_rebuild: PASS (689.53 s) (04/22) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED (04/22) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (53.79 s) (05/22) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED (05/22) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (32.64 s) (06/22) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED (06/22) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (34.15 s) (07/22) citest.py:CrossTest.test_cross: STARTED (07/22) citest.py:CrossTest.test_cross: PASS (488.24 s) (08/22) citest.py:CrossTest.test_cross_debsrc: STARTED (08/22) citest.py:CrossTest.test_cross_debsrc: PASS (1409.06 s) (09/22) citest.py:CrossTest.test_cross_trixie: STARTED (09/22) citest.py:CrossTest.test_cross_trixie: PASS (216.54 s) (10/22) citest.py:CrossTest.test_cross_kselftest: STARTED (10/22) citest.py:CrossTest.test_cross_kselftest: PASS (340.48 s) (11/22) citest.py:CrossTest.test_cross_rpi: STARTED (11/22) citest.py:CrossTest.test_cross_rpi: PASS (1053.48 s) (12/22) citest.py:VmBootTestFast.test_arm_bullseye: STARTED (12/22) citest.py:VmBootTestFast.test_arm_bullseye: PASS (41.03 s) (13/22) citest.py:VmBootTestFast.test_arm_bullseye_example_module: STARTED (13/22) citest.py:VmBootTestFast.test_arm_bullseye_example_module: PASS (7.07 s) (14/22) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: STARTED (14/22) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: PASS (7.82 s) (15/22) citest.py:VmBootTestFast.test_arm_buster: STARTED (15/22) citest.py:VmBootTestFast.test_arm_buster: PASS (37.54 s) (16/22) citest.py:VmBootTestFast.test_arm_buster_getty_target: STARTED (16/22) citest.py:VmBootTestFast.test_arm_buster_getty_target: PASS (6.79 s) (17/22) citest.py:VmBootTestFast.test_arm_buster_example_module: STARTED (17/22) citest.py:VmBootTestFast.test_arm_buster_example_module: PASS (7.57 s) (18/22) citest.py:VmBootTestFast.test_arm_bookworm: STARTED (18/22) citest.py:VmBootTestFast.test_arm_bookworm: PASS (49.58 s) (19/22) citest.py:VmBootTestFast.test_arm_bookworm_example_module: STARTED (19/22) citest.py:VmBootTestFast.test_arm_bookworm_example_module: PASS (8.06 s) (20/22) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: STARTED (20/22) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: PASS (8.18 s) (21/22) citest.py:VmBootTestFast.test_amd64_trixie: STARTED (21/22) citest.py:VmBootTestFast.test_amd64_trixie: PASS (37.14 s) (22/22) citest.py:VmBootTestFast.test_arm64_trixie: STARTED (22/22) citest.py:VmBootTestFast.test_arm64_trixie: PASS (41.79 s) RESULTS : PASS 22 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 6585.87 s cedric.hombourger@siemens.com (4): rootfs: introduce wrapper to run commands against a rootfs deb-dl-dir: optimize caching of source packages using apt natively bootstrap: create lock for downloads/deb without sudo rootfs: do not get elevated privileges when downloading packages RECIPE-API-CHANGELOG.md | 8 ++ doc/user_manual.md | 1 + meta/classes/deb-dl-dir.bbclass | 58 ++++++------- meta/classes/rootfs.bbclass | 83 ++++++++++++++++++- .../isar-mmdebstrap/isar-mmdebstrap.inc | 4 + 5 files changed, 120 insertions(+), 34 deletions(-) -- 2.47.3 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250925065433.4180883-1-cedric.hombourger%40siemens.com.