Re: [PATCH v12 00/21] Sbuild/Schroot migration

[Anton Mikanovich <amikan@ilbers.de>]

28.06.2022 09:48, Anton Mikanovich wrote:
> This patchset integrates sbuild/schroot tools into Isar build system.
>
> Base schroot images are created with sbuild-chroot-target or, in case
> of cross-build, sbuild-chroot-host recipes. These images are based on
> isar-bootstrap ones, but include some build-related stuff preinstalled.
>
> To use these changes you need to have sbuild and schroot installed and
> configured. User who runs build should be added to sbuild group.
>
> Currently, sbuild-related stuff is supported in latest kas.
>
> If Gilab CI is used, the following configuration changes are required:
> - external dir for schroot overlay should be set in the config
> /etc/gitlab-runner/config.toml:
>
> volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"]
>
> The patchset was tested with 'meta-iot2050' and 'xenomai-images'
> downstreams in combination with kas image mentioned above.
>
> Current limitations:
> - parallel building of several packages requires more free space in
> comparison with buildchroot-based architecture. This happens due
> the sbuild architecture which uses some basic layer (common for all
> packages) and per-package separate layer (where builddeps are downloaded
> and installed).
>
> Changes sinse v11:
> - Rebase on current next.
>
> Changes sinse v10:
> - Rebase on current next.
>
> Changes sinse v9:
> - Add caching improvement patch;
> - Remove per-package deb artifacts dir after usage to reduce disk space;
> - Improve schroot cleanup on failures.
>
> Changes since v8:
> - Added patch with support of package versions with epochs
>
> Changes since v7:
> - fixed work with proxy by adding some environment filters in sbuild;
> - DEB_BUILD_PROFILES support simplified and improved
> - Fixed gbp-based recipes rebuilding (get back --git-ignore-new option
> previously removed)
> - fixed base-apt caching for packages and source packages temporary
> installed in sbuild isolated environment
> - fixed rare build error related to ccache setup (added locking)
>
> Changes since v6:
> - added patch that switches sbuild from using source directory to using
> .dsc file. The file is generated before sbuild call by `dpkg-source`.
>
> Changes since v5:
> - warning about shell exports in dpkg_runbuild_prepend. This should
> avoided be reworked to templates.
>
> Changes since v4:
> - consider shell exports done in dpkg_runbuild_prepend and pass them
> to sbuild environment;
> - fixed ccache work.
>
> Changes since v3:
> - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for downstreams
> compatibility;
> - dpkg_build_export is used for adjusting sbuild environment;
> - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build.
>
> Changes since v2:
> - patches reworked/squashed for easier reading and understanding;
> - fixed building foreigh architectures with kas-docker;
> - implemented support of ccache;
> - fixed devshell and devshell_nodeps.
>
> Changes since v1:
> - parallel builds with different BUILD_DIR are supported;
> - parallel multiconfig targets in one build are supported;
> - per-task schroot configuration in /etc/schroot/ is now used;
> - patchset now passes Jenkins CI (so patches changes RFC => PATCH).
>
> Anton Mikanovich (3):
>    dpkg: Build packages with sbuild
>    dpkg-base: Cleanup on schroot fail
>    sbuild: Add changelog entry
>
> Felix Moessbauer (4):
>    fix: support build of packages with epoch version
>    always create apt-cache dirs in deb_dl_dir_import
>    avoid absolute SCHROOT_* paths to improve caching
>    locate generated binary package in WORKDIR
>
> Uladzimir Bely (14):
>    dpkg-gbp: Use separate command to export tarball
>    dpkg-gbp: Use host tools for dsc preparation
>    sbuild: Add recipes for host and target rootfs to run sbuild
>    sbuild: Introduce a class for another build method
>    sbuild: Support of DEB_BUILD_PROFILES
>    sbuild: Support of shell exports from dpkg_runbuild_prepend
>    dpkg: Remove builddeps install task
>    sbuild: Add ccache support
>    dpkg-base: Switch devshell to use schroot
>    dpkg-base: Switch apt_fetch and apt_unpack to use schroot
>    doc: Add sbuild-related documentation
>    sbuild: Use .dsc file instead of source directory
>    sbuild: Fixed proxy support
>    sbuild: Fix debsrc_download for packages dependencies
>
>   RECIPE-API-CHANGELOG.md                       |  13 ++
>   doc/user_manual.md                            |  22 ++-
>   meta/classes/deb-dl-dir.bbclass               |   6 +-
>   meta/classes/dpkg-base.bbclass                |  90 ++++++----
>   meta/classes/dpkg-gbp.bbclass                 |  26 +--
>   meta/classes/dpkg.bbclass                     | 109 +++++++++---
>   meta/classes/sbuild.bbclass                   | 159 ++++++++++++++++++
>   meta/conf/bitbake.conf                        |   2 +
>   .../sbuild-chroot/sbuild-chroot-host.bb       |  13 ++
>   .../sbuild-chroot/sbuild-chroot-target.bb     |  10 ++
>   .../sbuild-chroot/sbuild-chroot.inc           |  39 +++++
>   11 files changed, 408 insertions(+), 81 deletions(-)
>   create mode 100644 meta/classes/sbuild.bbclass
>   create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
>   create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
>   create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
>
This is also just a rebase on the current next.
We are planning to merge it in the next few days.

Bitbake update will be the first priority after that.