[PATCH v3 00/20] Sbuild/Schroot migration

[Uladzimir Bely <ubely@ilbers.de>]

This is a patchset showing how sbuild/schroot tools can be integrated
into Isar build system.

Base schroot image is created with sbuild-chroot-target (or
sbuild-chroot-host in case of cross-build) recipe. These images are based
on isar-bootstrap ones, but include some build-related stuff preinstalled.

To use this changes you need to have sbuild and schroot installed and
configured. Current user should be added to sbuild group.

If 'kas-container' is used, it should be modified to support sbuild:
- /var/lib/schroot/union/overlay should be externally mounted (-v option)
- export KAS_IMAGE_VERSION=2.6.2-sbuild
- export KAS_CONTAINER_IMAGE_PATH=ghcr.io/wiselord
Here I uploaded a docker image that has preinstalled software (sbuild,
schroot, etc) required to run sbuild. Later, the changes might be
included in official kas docker images.

If 'gitlab' is used, the similar changes are required:
- 'image: ghcr.io/wiselord/kas-isar:2.6.2-sbuild' in .gitlab-ci.yml
- external dir for schroot overlay should also be specified in
/etc/gitlab-runner/config.toml:
volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"]

The patchset was tested with 'meta-iot2050' and 'xenomai-images'
downstreams. Some patches are required for these downstreams,
appropriate pull requests will be created soon.

Changes since v2:
- patches reworked/squashed for easier reading and understanding
- fixed building foreigh architectures with kas-docker
- implemented support of ccache
- fixed devshell and devshell_nodeps
Changes since v1:
- parallel builds with different BUILD_DIR are supported
- parallel multiconfig targets in one build are supported
- per-task schroot configuration in /etc/schroot/ is now used
- patchset now passes Jenkins CI (so patches changes RFC => PATCH)

Current limitations:
- parallel building of several packages requires more free space in
comparison with buildchroot-based architecture. This happens due
the sbuild architecture which uses some basic layer (common for all
packages) and per-package separate layer (where builddeps are downloaded
and installed).

For experiments, the code can be taken from Isar 'ubely/sbuild' branch.
This is a dev branch, so sometimes it can be force-pushed when some
fixes are applied of rebased to 'next' branch.

First 9 patches (1..9) - are just preparation patches. With them, Isar will
continue using buildchroot-based architecture. So, potentially,
they can be merged as a subseries.

Remaining 11 patches (10..20) represent a sbuild-related changes. 

Anton Mikanovich (10):
  template: Make templates passthrough
  linux-module: Do not use shell environment
  u-boot: Do not use shell environment
  trusted-firmware: Do not use shell environment
  optee-os: Do not use shell environment
  kselftest: Do not use shell environment
  linux-mainline: Move cfg fragment test to debian/rules
  dpkg: Build packages with sbuild
  sbuild: Introduce environment variables export API
  linux-custom: Prepare kernel config inside sbuild

Uladzimir Bely (10):
  dpkg-gbp: Use separate command to export tarball
  dpkg-gbp: Use host tools for dsc preparation
  sbuild: Add recipes for host and target rootfs to run sbuild
  sbuild: Introduce a class for another build method
  dpkg: Remove builddeps install task.
  dpkg-base: Switch devshell to use schroot
  dpkg-base: Switch apt_fetch and apt_unpack to use schroot
  dpkg-base: Cleanup from buildchroot parts.
  doc: Add sbuild-related documentation
  sbuild: add ccache support

 doc/user_manual.md                            |  22 ++-
 .../optee-os/optee-os-stm32mp15x_3.11.0.bb    |   2 +-
 .../linux/linux-mainline_5.4.70.bb            |  16 +-
 meta/classes/dpkg-base.bbclass                |  98 ++++------
 meta/classes/dpkg-gbp.bbclass                 |  26 +--
 meta/classes/dpkg.bbclass                     |  53 ++---
 meta/classes/sbuild.bbclass                   | 184 ++++++++++++++++++
 meta/classes/template.bbclass                 |   7 +-
 meta/conf/bitbake.conf                        |   2 +
 .../files/debian/{rules => rules.tmpl}        |   4 +-
 meta/recipes-bsp/optee-os/optee-os-custom.inc |   9 +-
 .../files/debian/{rules => rules.tmpl}        |   4 +-
 .../trusted-firmware-a-custom.inc             |   9 +-
 .../u-boot/files/debian/{rules => rules.tmpl} |   2 +-
 meta/recipes-bsp/u-boot/u-boot-custom.inc     |   9 +-
 .../sbuild-chroot/sbuild-chroot-host.bb       |  13 ++
 .../sbuild-chroot/sbuild-chroot-target.bb     |  10 +
 .../sbuild-chroot/sbuild-chroot.inc           |  39 ++++
 .../kselftest/files/{rules => rules.tmpl}     |   0
 meta/recipes-kernel/kselftest/kselftest.inc   |  21 +-
 .../linux-module/files/debian/rules           |  37 ----
 .../linux-module/files/debian/rules.tmpl      |  52 +++++
 meta/recipes-kernel/linux-module/module.inc   |  19 +-
 meta/recipes-kernel/linux/linux-custom.inc    |   7 +-
 24 files changed, 437 insertions(+), 208 deletions(-)
 create mode 100644 meta/classes/sbuild.bbclass
 rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl} (75%)
 rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules => rules.tmpl} (77%)
 rename meta/recipes-bsp/u-boot/files/debian/{rules => rules.tmpl} (94%)
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
 rename meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%)
 delete mode 100755 meta/recipes-kernel/linux-module/files/debian/rules
 create mode 100755 meta/recipes-kernel/linux-module/files/debian/rules.tmpl

-- 
2.20.1