[PATCH v3 0/6] non-privileged commands in chroot

["'Cedric Hombourger' via isar-users" <isar-users@googlegroups.com>]

When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.

This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:

- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar

Testing:
- Basic smoke tests performed successfully (citest.py -t fast)
- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations

Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])

Changes since v2 patch:
  - rootfs_install_pkgs_download will no longer use sudo to run
    apt-get install --download-only. This was added to further
    demonstrate/test rootfs_cmd in existing Isar code.

Changes since v1 patch:
  - Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
  - Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes

Changes since RFC patch:
  - Let caller decide where to bind-mount the rootfs to
  - Make the rootfs argument optional
  - Support 32-bit rootfs (no lib64 there)

(Re-)validated with "citest.py -t fast" (using kas-container 4.8.1):
 
 JOB ID     : 2724be97c6711e046fbc2169823c293dc99cd97c
 JOB LOG    : avocado/job-results/job-2025-06-25T15.51-2724be9/job.log
  (01/19) citest.py:DevTest.test_dev: STARTED
  (01/19) citest.py:DevTest.test_dev:  PASS (1573.34 s)
  (02/19) citest.py:DevTest.test_dev_apps: STARTED
  (02/19) citest.py:DevTest.test_dev_apps:  PASS (2158.85 s)
  (03/19) citest.py:DevTest.test_dev_rebuild: STARTED
  (03/19) citest.py:DevTest.test_dev_rebuild:  PASS (349.73 s)
  (04/19) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
  (04/19) citest.py:DevTest.test_dev_run_amd64_bookworm:  PASS (77.79 s)
  (05/19) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
  (05/19) citest.py:DevTest.test_dev_run_arm64_bookworm:  PASS (55.12 s)
  (06/19) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
  (06/19) citest.py:DevTest.test_dev_run_arm_bookworm:  PASS (58.94 s)
  (07/19) citest.py:CrossTest.test_cross: STARTED
  (07/19) citest.py:CrossTest.test_cross:  PASS (1912.25 s)
  (08/19) citest.py:CrossTest.test_cross_debsrc: STARTED
  (08/19) citest.py:CrossTest.test_cross_debsrc:  PASS (2933.62 s)
  (09/19) citest.py:CrossTest.test_cross_kselftest: STARTED
  (09/19) citest.py:CrossTest.test_cross_kselftest:  PASS (2024.26 s)
  (10/19) citest.py:CrossTest.test_cross_rpi: STARTED
  (10/19) citest.py:CrossTest.test_cross_rpi:  PASS (1543.77 s)
  (11/19) citest.py:VmBootTestFast.test_arm_bullseye: STARTED
  (11/19) citest.py:VmBootTestFast.test_arm_bullseye:  PASS (64.33 s)
  (12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module: STARTED
  (12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module:  PASS (12.72 s)
  (13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: STARTED
  (13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target:  PASS (10.18 s)
  (14/19) citest.py:VmBootTestFast.test_arm_buster: STARTED
  (14/19) citest.py:VmBootTestFast.test_arm_buster:  PASS (57.01 s)
  (15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target: STARTED
  (15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target:  PASS (9.73 s)
  (16/19) citest.py:VmBootTestFast.test_arm_buster_example_module: STARTED
  (16/19) citest.py:VmBootTestFast.test_arm_buster_example_module:  PASS (10.39 s)
  (17/19) citest.py:VmBootTestFast.test_arm_bookworm: STARTED
  (17/19) citest.py:VmBootTestFast.test_arm_bookworm:  PASS (82.93 s)
  (18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module: STARTED
  (18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module:  PASS (30.40 s)
  (19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: STARTED
  (19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target:  PASS (11.59 s)
 RESULTS    : PASS 19 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
 JOB TIME   : 13003.86 s
 
Cedric Hombourger (5):
  rootfs: introduce wrapper to run commands against a rootfs
  deb-dl-dir: optimize caching of source packages using apt natively
  image-postproc-extension: refactor systemd version checks
  image-postproc-extension: extract systemd's version using rootfs_cmd
  bootstrap: create lock for downloads/deb without sudo
  rootfs: do not get elevated privileges when downloading packages

 RECIPE-API-CHANGELOG.md                       |  7 ++
 doc/user_manual.md                            |  1 +
 meta/classes/deb-dl-dir.bbclass               | 58 ++++++-------
 meta/classes/image-postproc-extension.bbclass | 12 +--
 meta/classes/rootfs.bbclass                   | 83 ++++++++++++++++++-
 .../isar-mmdebstrap/isar-mmdebstrap.inc       |  4 +
 6 files changed, 126 insertions(+), 39 deletions(-)

-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250625193748.2681-1-cedric.hombourger%40siemens.com.