RE: [PATCH v12 00/21] Sbuild/Schroot migration

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

> -----Original Message-----
> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> Behalf Of Anton Mikanovich
> Sent: Tuesday, June 28, 2022 8:57 AM
> To: isar-users <isar-users@googlegroups.com>
> Subject: Re: [PATCH v12 00/21] Sbuild/Schroot migration
> 
> 28.06.2022 09:48, Anton Mikanovich wrote:
> > This patchset integrates sbuild/schroot tools into Isar build system.
> >
> > Base schroot images are created with sbuild-chroot-target or, in case
> > of cross-build, sbuild-chroot-host recipes. These images are based on
> > isar-bootstrap ones, but include some build-related stuff preinstalled.
> >
> > To use these changes you need to have sbuild and schroot installed and
> > configured. User who runs build should be added to sbuild group.
> >
> > Currently, sbuild-related stuff is supported in latest kas.
> >
> > If Gilab CI is used, the following configuration changes are required:
> > - external dir for schroot overlay should be set in the config
> > /etc/gitlab-runner/config.toml:
> >
> > volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"]
> >
> > The patchset was tested with 'meta-iot2050' and 'xenomai-images'
> > downstreams in combination with kas image mentioned above.
> >
> > Current limitations:
> > - parallel building of several packages requires more free space in
> > comparison with buildchroot-based architecture. This happens due the
> > sbuild architecture which uses some basic layer (common for all
> > packages) and per-package separate layer (where builddeps are
> > downloaded and installed).
> >
> > Changes sinse v11:
> > - Rebase on current next.
> >
> > Changes sinse v10:
> > - Rebase on current next.
> >
> > Changes sinse v9:
> > - Add caching improvement patch;
> > - Remove per-package deb artifacts dir after usage to reduce disk
> > space;
> > - Improve schroot cleanup on failures.
> >
> > Changes since v8:
> > - Added patch with support of package versions with epochs
> >
> > Changes since v7:
> > - fixed work with proxy by adding some environment filters in sbuild;
> > - DEB_BUILD_PROFILES support simplified and improved
> > - Fixed gbp-based recipes rebuilding (get back --git-ignore-new option
> > previously removed)
> > - fixed base-apt caching for packages and source packages temporary
> > installed in sbuild isolated environment
> > - fixed rare build error related to ccache setup (added locking)
> >
> > Changes since v6:
> > - added patch that switches sbuild from using source directory to
> > using .dsc file. The file is generated before sbuild call by `dpkg-source`.
> >
> > Changes since v5:
> > - warning about shell exports in dpkg_runbuild_prepend. This should
> > avoided be reworked to templates.
> >
> > Changes since v4:
> > - consider shell exports done in dpkg_runbuild_prepend and pass them
> > to sbuild environment;
> > - fixed ccache work.
> >
> > Changes since v3:
> > - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for
> > downstreams compatibility;
> > - dpkg_build_export is used for adjusting sbuild environment;
> > - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build.
> >
> > Changes since v2:
> > - patches reworked/squashed for easier reading and understanding;
> > - fixed building foreigh architectures with kas-docker;
> > - implemented support of ccache;
> > - fixed devshell and devshell_nodeps.
> >
> > Changes since v1:
> > - parallel builds with different BUILD_DIR are supported;
> > - parallel multiconfig targets in one build are supported;
> > - per-task schroot configuration in /etc/schroot/ is now used;
> > - patchset now passes Jenkins CI (so patches changes RFC => PATCH).
> >
> > Anton Mikanovich (3):
> >    dpkg: Build packages with sbuild
> >    dpkg-base: Cleanup on schroot fail
> >    sbuild: Add changelog entry
> >
> > Felix Moessbauer (4):
> >    fix: support build of packages with epoch version
> >    always create apt-cache dirs in deb_dl_dir_import
> >    avoid absolute SCHROOT_* paths to improve caching
> >    locate generated binary package in WORKDIR
> >
> > Uladzimir Bely (14):
> >    dpkg-gbp: Use separate command to export tarball
> >    dpkg-gbp: Use host tools for dsc preparation
> >    sbuild: Add recipes for host and target rootfs to run sbuild
> >    sbuild: Introduce a class for another build method
> >    sbuild: Support of DEB_BUILD_PROFILES
> >    sbuild: Support of shell exports from dpkg_runbuild_prepend
> >    dpkg: Remove builddeps install task
> >    sbuild: Add ccache support
> >    dpkg-base: Switch devshell to use schroot
> >    dpkg-base: Switch apt_fetch and apt_unpack to use schroot
> >    doc: Add sbuild-related documentation
> >    sbuild: Use .dsc file instead of source directory
> >    sbuild: Fixed proxy support
> >    sbuild: Fix debsrc_download for packages dependencies
> >
> >   RECIPE-API-CHANGELOG.md                       |  13 ++
> >   doc/user_manual.md                            |  22 ++-
> >   meta/classes/deb-dl-dir.bbclass               |   6 +-
> >   meta/classes/dpkg-base.bbclass                |  90 ++++++----
> >   meta/classes/dpkg-gbp.bbclass                 |  26 +--
> >   meta/classes/dpkg.bbclass                     | 109 +++++++++---
> >   meta/classes/sbuild.bbclass                   | 159 ++++++++++++++++++
> >   meta/conf/bitbake.conf                        |   2 +
> >   .../sbuild-chroot/sbuild-chroot-host.bb       |  13 ++
> >   .../sbuild-chroot/sbuild-chroot-target.bb     |  10 ++
> >   .../sbuild-chroot/sbuild-chroot.inc           |  39 +++++
> >   11 files changed, 408 insertions(+), 81 deletions(-)
> >   create mode 100644 meta/classes/sbuild.bbclass
> >   create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-
> host.bb
> >   create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-
> target.bb
> >   create mode 100644
> > meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
> >
> This is also just a rebase on the current next.
> We are planning to merge it in the next few days.

I'm happy to see the sbuild integrated into ISAR.
We run some of our layers against this series for a while now.
No issues so far (apart from changes required for custom tasks, etc...).

Felix

> 
> Bitbake update will be the first priority after that.
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-users%2F7a97d256-669e-f8a7-699b-
> e6418a270225%2540ilbers.de&amp;data=05%7C01%7Cfelix.moessbauer%40sie
> mens.com%7C17cda068ab944c38019808da58d37a4a%7C38ae3bcd95794fd4ad
> dab42e1495d55a%7C1%7C0%7C637919963283287304%7CUnknown%7CTWFpb
> GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> n0%3D%7C3000%7C%7C%7C&amp;sdata=VwmNjDEboXR4ZlDoHh71WtOjAvk1T
> jWTNMLfWqWllco%3D&amp;reserved=0.