[RFC v2 00/20] add support to build isar unprivileged

["'Felix Moessbauer' via isar-users" <isar-users@googlegroups.com>]

Dear isar-users,

currently isar requires password-less sudo and an environment
where mounting file systems is possible. This has proven problematic
for security reasons, both when running in a privileged container or
locally.

To solve this, we implement fully rootless builds that rely on the
unshare syscall which allows us to avoid sudo and instead operate in
temporary kernel namespaces as a user that is just privileged within
that namespace. This comes with some challenges regarding the handling
of mounts (they are cleared when leaving the namespace), as well as
cross namespace deployments (the outer user might not be able to access
the inner data). For that, we rework the handling of mounts and artifact
passing to make it compatible with both chroot modes (schroot and
unshare).

The patches 1-10 align the file permissions of deployments and artifacts
to avoid the use of chown (which will not work anymore across uid
boundaries). In addition, helpers are introduced to perform privileged
operations, which simplifies the migration of existing layers.

The patches 11 and 12 introduce the unshare mode, which can be executed
as a normal user and does not require root. To enable this mode, set
ISAR_ROOTLESS = "1".

While the series is by far not complete yet, it already passes the DevTest
CI. Know issues are currently:

- no support for VM and container images
- unprivileged cleanup of the build/tmp dir is non trivial
- sporadic issues on partial rebuilds on rootfs_install_sstate_finalize
- interfaces between kas and isar need to be defined

Note, that this series can be tested on a custom kas-container build
provided in [1]. Hints how to migrate downstream layers are provided
in the API changelog.

Changes since RFC 1:

- switch build_type to isar-rootless in isar.yaml (Note: switch back
  if testing locally in a unprepared kas container)
- complete overhaul of the mounting in unshared namespaces
  - fixes the systemd presetting
  - fixes hangs when pulling from snapshot mirrors
- rename the run_privileged_here to run_privileged_heredoc to clarify its intention
- add support for
  - dpkg-source with do_fetch_common_source
  - vm images
  - container images
  - discoverable disk images
- add helper script to clean build dir in unprivileged mode
- reduce clutter we leave after finishing a build
- fix issues when running in a privileged environment without sub user ids
- bugfixes

Still missing is the support for the devshell. Further, the rootless build dir
must not reside in a git worktree (a normal git dir is fine). This is probably a
bug in combination with kas-container.

[1] https://groups.google.com/g/kas-devel/c/NWQFCU2aUHg

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (19):
  refactor bootstrap: store rootfs tar with user permissions
  deb-dl-dir: export without root privileges
  download debs without locking
  introduce wrappers for privileged execution
  bootstrap: move cleanup trap to function
  rootfs: rework sstate caching of rootfs artifact
  rootfs_generate_initramfs: rework deployment to avoid chowning
  wic: rework image deploy logic to deploy under correct user
  use bitbake function to generate mounting scripts
  apt-fetcher: prepare for chroot specific fetching
  add support for fully rootless builds
  add helper script to clean artifacts in build dir
  apt-fetcher: implement support for unshare backend
  vm images: make compatible with rootless build
  ddi image: convert to two stage deploy
  container images: make compatible with rootless build
  dpkg-source: implement multiarch support for unshare backend
  rootfs: remove temporary sstate deploy directory after task execution
  use copy of sbom-chroot for sbom creation

 Kconfig                                       |   2 +-
 RECIPE-API-CHANGELOG.md                       |  58 +++++
 doc/user_manual.md                            |   2 +
 kas/isar.yaml                                 |   2 +-
 meta/classes-global/base.bbclass              | 132 +++++++++++
 meta/classes-recipe/deb-dl-dir.bbclass        |  20 +-
 meta/classes-recipe/dpkg-base.bbclass         |  20 +-
 meta/classes-recipe/dpkg-source.bbclass       |  42 +++-
 meta/classes-recipe/dpkg.bbclass              |  16 +-
 .../image-account-extension.bbclass           |   4 +-
 .../image-locales-extension.bbclass           |  13 +-
 .../image-postproc-extension.bbclass          |  30 +--
 .../image-tools-extension.bbclass             |  96 +++++++-
 meta/classes-recipe/image.bbclass             |  24 +-
 meta/classes-recipe/imagetypes.bbclass        |  47 ++--
 .../imagetypes_container.bbclass              |  37 ++--
 meta/classes-recipe/imagetypes_ddi.bbclass    |   8 +-
 meta/classes-recipe/imagetypes_vm.bbclass     |  29 ++-
 meta/classes-recipe/imagetypes_wic.bbclass    |  12 +-
 meta/classes-recipe/rootfs.bbclass            | 205 +++++++++---------
 meta/classes-recipe/sbuild.bbclass            |  36 ++-
 meta/classes-recipe/sdk.bbclass               |  22 +-
 meta/classes-recipe/squashfs.bbclass          |   2 +-
 meta/classes/sbom.bbclass                     |  29 ++-
 meta/conf/bitbake.conf                        |   7 +-
 meta/lib/aptsrc_fetcher.py                    |  90 +++++++-
 .../isar-mmdebstrap/isar-mmdebstrap.inc       |  47 ++--
 .../sbom-chroot/sbom-chroot.bb                |  11 +-
 .../sbuild-chroot/sbuild-chroot.inc           |  24 +-
 scripts/isar-clean-builddir                   |  73 +++++++
 .../unittests/test_image_account_extension.py |   9 +-
 31 files changed, 886 insertions(+), 263 deletions(-)
 create mode 100755 scripts/isar-clean-builddir

-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260220171601.3845113-1-felix.moessbauer%40siemens.com.