public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
* [PATCH v3 0/6] non-privileged commands in chroot
@ 2025-06-25 19:37 'Cedric Hombourger' via isar-users
  2025-06-25 19:37 ` [PATCH v3 1/6] rootfs: introduce wrapper to run commands against a rootfs 'Cedric Hombourger' via isar-users
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: 'Cedric Hombourger' via isar-users @ 2025-06-25 19:37 UTC (permalink / raw)
  To: isar-users; +Cc: felix.moessbauer, Cedric Hombourger

When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.

This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:

- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar

Testing:
- Basic smoke tests performed successfully (citest.py -t fast)
- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations

Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])

Changes since v2 patch:
  - rootfs_install_pkgs_download will no longer use sudo to run
    apt-get install --download-only. This was added to further
    demonstrate/test rootfs_cmd in existing Isar code.

Changes since v1 patch:
  - Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
  - Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes

Changes since RFC patch:
  - Let caller decide where to bind-mount the rootfs to
  - Make the rootfs argument optional
  - Support 32-bit rootfs (no lib64 there)

(Re-)validated with "citest.py -t fast" (using kas-container 4.8.1):
 
 JOB ID     : 2724be97c6711e046fbc2169823c293dc99cd97c
 JOB LOG    : avocado/job-results/job-2025-06-25T15.51-2724be9/job.log
  (01/19) citest.py:DevTest.test_dev: STARTED
  (01/19) citest.py:DevTest.test_dev:  PASS (1573.34 s)
  (02/19) citest.py:DevTest.test_dev_apps: STARTED
  (02/19) citest.py:DevTest.test_dev_apps:  PASS (2158.85 s)
  (03/19) citest.py:DevTest.test_dev_rebuild: STARTED
  (03/19) citest.py:DevTest.test_dev_rebuild:  PASS (349.73 s)
  (04/19) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
  (04/19) citest.py:DevTest.test_dev_run_amd64_bookworm:  PASS (77.79 s)
  (05/19) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
  (05/19) citest.py:DevTest.test_dev_run_arm64_bookworm:  PASS (55.12 s)
  (06/19) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
  (06/19) citest.py:DevTest.test_dev_run_arm_bookworm:  PASS (58.94 s)
  (07/19) citest.py:CrossTest.test_cross: STARTED
  (07/19) citest.py:CrossTest.test_cross:  PASS (1912.25 s)
  (08/19) citest.py:CrossTest.test_cross_debsrc: STARTED
  (08/19) citest.py:CrossTest.test_cross_debsrc:  PASS (2933.62 s)
  (09/19) citest.py:CrossTest.test_cross_kselftest: STARTED
  (09/19) citest.py:CrossTest.test_cross_kselftest:  PASS (2024.26 s)
  (10/19) citest.py:CrossTest.test_cross_rpi: STARTED
  (10/19) citest.py:CrossTest.test_cross_rpi:  PASS (1543.77 s)
  (11/19) citest.py:VmBootTestFast.test_arm_bullseye: STARTED
  (11/19) citest.py:VmBootTestFast.test_arm_bullseye:  PASS (64.33 s)
  (12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module: STARTED
  (12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module:  PASS (12.72 s)
  (13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: STARTED
  (13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target:  PASS (10.18 s)
  (14/19) citest.py:VmBootTestFast.test_arm_buster: STARTED
  (14/19) citest.py:VmBootTestFast.test_arm_buster:  PASS (57.01 s)
  (15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target: STARTED
  (15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target:  PASS (9.73 s)
  (16/19) citest.py:VmBootTestFast.test_arm_buster_example_module: STARTED
  (16/19) citest.py:VmBootTestFast.test_arm_buster_example_module:  PASS (10.39 s)
  (17/19) citest.py:VmBootTestFast.test_arm_bookworm: STARTED
  (17/19) citest.py:VmBootTestFast.test_arm_bookworm:  PASS (82.93 s)
  (18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module: STARTED
  (18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module:  PASS (30.40 s)
  (19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: STARTED
  (19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target:  PASS (11.59 s)
 RESULTS    : PASS 19 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
 JOB TIME   : 13003.86 s
 
Cedric Hombourger (5):
  rootfs: introduce wrapper to run commands against a rootfs
  deb-dl-dir: optimize caching of source packages using apt natively
  image-postproc-extension: refactor systemd version checks
  image-postproc-extension: extract systemd's version using rootfs_cmd
  bootstrap: create lock for downloads/deb without sudo
  rootfs: do not get elevated privileges when downloading packages

 RECIPE-API-CHANGELOG.md                       |  7 ++
 doc/user_manual.md                            |  1 +
 meta/classes/deb-dl-dir.bbclass               | 58 ++++++-------
 meta/classes/image-postproc-extension.bbclass | 12 +--
 meta/classes/rootfs.bbclass                   | 83 ++++++++++++++++++-
 .../isar-mmdebstrap/isar-mmdebstrap.inc       |  4 +
 6 files changed, 126 insertions(+), 39 deletions(-)

-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250625193748.2681-1-cedric.hombourger%40siemens.com.

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2025-06-25 19:40 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-06-25 19:37 [PATCH v3 0/6] non-privileged commands in chroot 'Cedric Hombourger' via isar-users
2025-06-25 19:37 ` [PATCH v3 1/6] rootfs: introduce wrapper to run commands against a rootfs 'Cedric Hombourger' via isar-users
2025-06-25 19:37 ` [PATCH v3 2/6] deb-dl-dir: optimize caching of source packages using apt natively 'Cedric Hombourger' via isar-users
2025-06-25 19:37 ` [PATCH v3 3/6] image-postproc-extension: refactor systemd version checks 'Cedric Hombourger' via isar-users
2025-06-25 19:37 ` [PATCH v3 4/6] image-postproc-extension: extract systemd's version using rootfs_cmd 'Cedric Hombourger' via isar-users
2025-06-25 19:37 ` [PATCH v3 5/6] bootstrap: create lock for downloads/deb without sudo 'Cedric Hombourger' via isar-users
2025-06-25 19:37 ` [PATCH v3 6/6] rootfs: do not get elevated privileges when downloading packages 'Cedric Hombourger' via isar-users

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox