Re: [PATCH v3 00/20] Sbuild/Schroot migration

[Jan Kiszka <jan.kiszka@siemens.com>]

On 08.12.21 14:40, Uladzimir Bely wrote:
> In mail from среда, 8 декабря 2021 г. 16:05:24 +03 user Uladzimir Bely wrote:
>> This is a patchset showing how sbuild/schroot tools can be integrated
>> into Isar build system.
>>
>> Base schroot image is created with sbuild-chroot-target (or
>> sbuild-chroot-host in case of cross-build) recipe. These images are based
>> on isar-bootstrap ones, but include some build-related stuff preinstalled.
>>
>> To use this changes you need to have sbuild and schroot installed and
>> configured. Current user should be added to sbuild group.
>>
>> If 'kas-container' is used, it should be modified to support sbuild:
>> - /var/lib/schroot/union/overlay should be externally mounted (-v option)
>> - export KAS_IMAGE_VERSION=2.6.2-sbuild
>> - export KAS_CONTAINER_IMAGE_PATH=ghcr.io/wiselord
>> Here I uploaded a docker image that has preinstalled software (sbuild,
>> schroot, etc) required to run sbuild. Later, the changes might be
>> included in official kas docker images.
>>
>> If 'gitlab' is used, the similar changes are required:
>> - 'image: ghcr.io/wiselord/kas-isar:2.6.2-sbuild' in .gitlab-ci.yml
>> - external dir for schroot overlay should also be specified in
>> /etc/gitlab-runner/config.toml:
>> volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"]
>>
>> The patchset was tested with 'meta-iot2050' and 'xenomai-images'
>> downstreams. Some patches are required for these downstreams,
>> appropriate pull requests will be created soon.
>>
>> Changes since v2:
>> - patches reworked/squashed for easier reading and understanding
>> - fixed building foreigh architectures with kas-docker
>> - implemented support of ccache
>> - fixed devshell and devshell_nodeps
>> Changes since v1:
>> - parallel builds with different BUILD_DIR are supported
>> - parallel multiconfig targets in one build are supported
>> - per-task schroot configuration in /etc/schroot/ is now used
>> - patchset now passes Jenkins CI (so patches changes RFC => PATCH)
>>
>> Current limitations:
>> - parallel building of several packages requires more free space in
>> comparison with buildchroot-based architecture. This happens due
>> the sbuild architecture which uses some basic layer (common for all
>> packages) and per-package separate layer (where builddeps are downloaded
>> and installed).
>>
>> For experiments, the code can be taken from Isar 'ubely/sbuild' branch.
>> This is a dev branch, so sometimes it can be force-pushed when some
>> fixes are applied of rebased to 'next' branch.
>>
>> First 9 patches (1..9) - are just preparation patches. With them, Isar will
>> continue using buildchroot-based architecture. So, potentially,
>> they can be merged as a subseries.
>>
>> Remaining 11 patches (10..20) represent a sbuild-related changes.
>>
>> Anton Mikanovich (10):
>>   template: Make templates passthrough
>>   linux-module: Do not use shell environment
>>   u-boot: Do not use shell environment
>>   trusted-firmware: Do not use shell environment
>>   optee-os: Do not use shell environment
>>   kselftest: Do not use shell environment
>>   linux-mainline: Move cfg fragment test to debian/rules
>>   dpkg: Build packages with sbuild
>>   sbuild: Introduce environment variables export API
>>   linux-custom: Prepare kernel config inside sbuild
>>
>> Uladzimir Bely (10):
>>   dpkg-gbp: Use separate command to export tarball
>>   dpkg-gbp: Use host tools for dsc preparation
>>   sbuild: Add recipes for host and target rootfs to run sbuild
>>   sbuild: Introduce a class for another build method
>>   dpkg: Remove builddeps install task.
>>   dpkg-base: Switch devshell to use schroot
>>   dpkg-base: Switch apt_fetch and apt_unpack to use schroot
>>   dpkg-base: Cleanup from buildchroot parts.
>>   doc: Add sbuild-related documentation
>>   sbuild: add ccache support
>>
>>  doc/user_manual.md                            |  22 ++-
>>  .../optee-os/optee-os-stm32mp15x_3.11.0.bb    |   2 +-
>>  .../linux/linux-mainline_5.4.70.bb            |  16 +-
>>  meta/classes/dpkg-base.bbclass                |  98 ++++------
>>  meta/classes/dpkg-gbp.bbclass                 |  26 +--
>>  meta/classes/dpkg.bbclass                     |  53 ++---
>>  meta/classes/sbuild.bbclass                   | 184 ++++++++++++++++++
>>  meta/classes/template.bbclass                 |   7 +-
>>  meta/conf/bitbake.conf                        |   2 +
>>  .../files/debian/{rules => rules.tmpl}        |   4 +-
>>  meta/recipes-bsp/optee-os/optee-os-custom.inc |   9 +-
>>  .../files/debian/{rules => rules.tmpl}        |   4 +-
>>  .../trusted-firmware-a-custom.inc             |   9 +-
>>  .../u-boot/files/debian/{rules => rules.tmpl} |   2 +-
>>  meta/recipes-bsp/u-boot/u-boot-custom.inc     |   9 +-
>>  .../sbuild-chroot/sbuild-chroot-host.bb       |  13 ++
>>  .../sbuild-chroot/sbuild-chroot-target.bb     |  10 +
>>  .../sbuild-chroot/sbuild-chroot.inc           |  39 ++++
>>  .../kselftest/files/{rules => rules.tmpl}     |   0
>>  meta/recipes-kernel/kselftest/kselftest.inc   |  21 +-
>>  .../linux-module/files/debian/rules           |  37 ----
>>  .../linux-module/files/debian/rules.tmpl      |  52 +++++
>>  meta/recipes-kernel/linux-module/module.inc   |  19 +-
>>  meta/recipes-kernel/linux/linux-custom.inc    |   7 +-
>>  24 files changed, 437 insertions(+), 208 deletions(-)
>>  create mode 100644 meta/classes/sbuild.bbclass
>>  rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl} (75%)
>>  rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules =>
>> rules.tmpl} (77%) rename meta/recipes-bsp/u-boot/files/debian/{rules =>
>> rules.tmpl} (94%) create mode 100644
>> meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode
>> 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create
>> mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc rename
>> meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%) delete
>> mode 100755 meta/recipes-kernel/linux-module/files/debian/rules create mode
>> 100755 meta/recipes-kernel/linux-module/files/debian/rules.tmpl
> 
> Just a question: Does 'xenomai-images' and 'meta-iot2050' have their own 
> maillists where I could send the changes to work with isar-based sbuild? I 
> just found a maillist for 'kas', but not for these projects.
> 

For xenomai-images, we handle changes via xenomai@xenomai.org (please
additional tag related patches with "[xenomai-images]").

meta-iot2050 takes pull requests on github.

For both projects, changes that will work with current Isar can already
be submitted. Changes that depend on your pending series could be
proposed along with an Isar revision bump once 'next' picked up your
queue. Or send them earlier with a clear remark that those are for
reference and future use. But it is probably better to wait for the
'next' merge.

Thanks,
Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux