[PATCH v2 00/24] Sbuild/Schroot migration

[PATCH v2 00/24] Sbuild/Schroot migration

[Uladzimir Bely]

This is a patchset showing how sbuild/schroot tools can be integrated
into Isar build system.

Base schroot image is created with sbuild-chroot-target (or
sbuild-chroot-host in case of cross-build) recipe. These images are based
on isar-bootstrap ones, but include some build-related stuff preinstalled.

To use this changes you need to have sbuild and schroot installed and
configured. Current user should be added to sbuild group.

Changes since v1:
- parallel builds with different BUILD_DIR are supported
- parallel multiconfig targets in one build are supported
- per-task schroot configuration in /etc/schroot/ is now used
- patchset now passes Jenkins CI (so patches changes RFC => PATCH)

Current limitations:
- there is an unsolved problem with building foreigh architectures
using kas-docker.

- qemuamd64 images are buildable in kas-docker, but some changes
in kas are reqired (like installing additional packages and adding
'builder' user to 'sbuild group). I used the following changes to
build ghcr.io/siemens/kas/kas-isar:2.6.2-sbuild image based on
current ghcr.io/siemens/kas/kas-isar:2.6.2:
https://github.com/WiseLord/kas/commit/5b7b02aa33

- parallel building of several packages requires more free space
than current buildchroot-based architecture uses. This happens due
the sbuild architecture which uses some basic layer (common for all
packages) and per-package separate layer (where builddeps are downloaded
and installed).

- recently merged to 'next' ccache support is not yet implemented
in case of sbuild used.

For experiments, the code can be taken from Isar 'ubely/sbuild' branch.
This is an dev branch, so sometimes it can be force-pushed when some
fixes are applied of rebased to 'next' branch.

Anton Mikanovich (12):
  dpkg: Install raw package files to source root
  isar-bootstrap: Export bootstrap to schroot config
  linux-module: Do not use shell environment
  u-boot: Do not use shell environment
  trusted-firmware: Do not use shell environment
  optee-os: Do not use shell environment
  kselftest: Do not use shell environment
  dpkg: Build packages with sbuild
  sbuild: Introduce environment variables export API
  dpkg-gbp: Migrate to schroot
  linux-mainline: Move cfg fragment test to debian/rules
  linux-custom: Prepare kernel config inside sbuild

Uladzimir Bely (12):
  dpkg-gbp: Use separate command to export tarball
  sbuild: Add recipes for host and target rootfs to run sbuild
  sbuild: Mount base-apt in schroot
  sbuild: Add sbuildshell task
  dpkg-gbp: Preinstall gbp utils in schroot
  dpkg: Remove builddeps install task.
  dpkg-base: Switch devshell to use schroot
  dpkg-base: Switch apt_fetch and apt_unpack to use schroot
  dpkg-base: Cleanup from buildchroot parts.
  dpkg-gbp: Use host tools for dsc preparation
  doc: Add sbuild-related documentation
  sbuild: Replace isar-apt mounting with copying

 doc/user_manual.md                            |  22 ++-
 .../optee-os/optee-os-stm32mp15x_3.11.0.bb    |   2 +-
 .../linux/linux-mainline_5.4.70.bb            |  16 +-
 meta/classes/dpkg-base.bbclass                |  80 ++------
 meta/classes/dpkg-gbp.bbclass                 |  25 +--
 meta/classes/dpkg-raw.bbclass                 |   8 +-
 meta/classes/dpkg.bbclass                     |  56 +++---
 meta/classes/sbuild.bbclass                   | 177 ++++++++++++++++++
 meta/conf/bitbake.conf                        |   2 +
 .../files/debian/{rules => rules.tmpl}        |   4 +-
 meta/recipes-bsp/optee-os/optee-os-custom.inc |  10 +-
 .../files/debian/{rules => rules.tmpl}        |   4 +-
 .../trusted-firmware-a-custom.inc             |  10 +-
 .../u-boot/files/debian/{rules => rules.tmpl} |   2 +-
 meta/recipes-bsp/u-boot/u-boot-custom.inc     |  10 +-
 .../sbuild-chroot/sbuild-chroot-host.bb       |  13 ++
 .../sbuild-chroot/sbuild-chroot-target.bb     |  10 +
 .../sbuild-chroot/sbuild-chroot.inc           |  36 ++++
 .../kselftest/files/{rules => rules.tmpl}     |   0
 meta/recipes-kernel/kselftest/kselftest.inc   |  21 +--
 .../files/debian/{rules => rules.tmpl}        |  14 +-
 meta/recipes-kernel/linux-module/module.inc   |  23 +--
 meta/recipes-kernel/linux/linux-custom.inc    |   7 +-
 23 files changed, 382 insertions(+), 170 deletions(-)
 create mode 100644 meta/classes/sbuild.bbclass
 rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl} (75%)
 rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules => rules.tmpl} (77%)
 rename meta/recipes-bsp/u-boot/files/debian/{rules => rules.tmpl} (94%)
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
 rename meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%)
 rename meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%)

-- 
2.20.1

[PATCH v2 01/24] dpkg: Install raw package files to source root

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To get rid of using absolute paths in debian/rules move D to source
root. This is default location for dh_install, so now there is no need
in mention it in debian/rules.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/dpkg-raw.bbclass | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index dd7b761f..73356fa0 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -5,7 +5,7 @@
 
 inherit dpkg
 
-D = "${WORKDIR}/image"
+D = "${S}"
 
 # Populate folder that will be picked up as package
 do_install() {
@@ -22,10 +22,4 @@ do_prepare_build() {
 		sed 's:^./::' > ${S}/debian/${PN}.install
 
 	deb_debianize
-
-	cat <<EOF >> ${S}/debian/rules
-
-override_dh_install:
-	dh_install --sourcedir=${PP}/image
-EOF
 }
-- 
2.20.1

[PATCH v2 02/24] dpkg-gbp: Use separate command to export tarball

[Uladzimir Bely]

We don't actually build the package with gbp, but only prepare it for
the building with external dpkg-buildpackage command. In case there is
no need in real build we can perform only export which will produce
upstream tarball to pass for later building.
This allows to use any build tool for later processing.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-gbp.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index d956e8c3..ddf23ca5 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -30,5 +30,9 @@ do_install_builddeps_append() {
 }
 
 dpkg_runbuild_prepend() {
-    export GBP_PREFIX="gbp buildpackage --git-ignore-new ${GBP_EXTRA_OPTIONS} --git-builder="
+    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+        sh -c "cd ${PP}/${PPS} && gbp buildpackage --git-builder=/bin/true ${GBP_EXTRA_OPTIONS}"
+    # NOTE: `buildpackage --git-builder=/bin/true --git-pristine-tar` is used
+    # for compatibility with gbp version froms debian-stretch. In newer distros
+    # it's possible to use a subcommand `export-orig --pristine-tar`
 }
-- 
2.20.1

[PATCH v2 03/24] isar-bootstrap: Export bootstrap to schroot config

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To use isar-bootstrap with sbuild and schroot tools we need to export
the path to schroot config.

We also need to put isar-apt and downloads into fstab for sbuild
profile to have it mounted inside schroot.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/sbuild.bbclass                   | 87 +++++++++++++++++++
 .../isar-bootstrap/isar-bootstrap.inc         | 10 ++-
 2 files changed, 96 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes/sbuild.bbclass

diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
new file mode 100644
index 00000000..d6e5b926
--- /dev/null
+++ b/meta/classes/sbuild.bbclass
@@ -0,0 +1,87 @@
+# This software is a part of ISAR.
+# Copyright (C) 2021 ilbers GmbH
+
+SCHROOT_CONF ?= "/etc/schroot"
+
+python __anonymous() {
+    import pwd
+    d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
+    d.setVar('SCHROOT_USER_HOME', pwd.getpwuid(os.geteuid()).pw_dir)
+
+    mode = d.getVar('ISAR_CROSS_COMPILE', True)
+    distro_arch = d.getVar('DISTRO_ARCH')
+    if mode == "0" or d.getVar('HOST_ARCH') ==  distro_arch or \
+       (d.getVar('HOST_DISTRO') == "debian-stretch" and distro_arch == "i386"):
+        d.setVar('SBUILD_HOST_ARCH', distro_arch)
+    else:
+        d.setVar('SBUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
+}
+
+SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${DISTRO}-${SBUILD_HOST_ARCH}"
+SBUILD_CHROOT_RW ?= "${SBUILD_CHROOT}-rw"
+
+SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}"
+SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}"
+
+SCHROOT_DIR ?= "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${SBUILD_HOST_ARCH}"
+
+schroot_create_configs() {
+    sudo -s <<'EOSUDO'
+        set -e
+
+        cat << EOF > "${SCHROOT_CONF_FILE}"
+[${SBUILD_CHROOT}]
+type=directory
+directory=${SCHROOT_DIR}
+profile=${SBUILD_CHROOT}
+users=${SCHROOT_USER}
+groups=root,sbuild
+root-users=${SCHROOT_USER}
+root-groups=root,sbuild
+source-root-users=${SCHROOT_USER}
+source-root-groups=root,sbuild
+union-type=overlay
+preserve-environment=true
+
+[${SBUILD_CHROOT_RW}]
+type=directory
+directory=${SCHROOT_DIR}
+profile=${SBUILD_CHROOT}
+users=${SCHROOT_USER}
+groups=root,sbuild
+root-users=${SCHROOT_USER}
+root-groups=root,sbuild
+preserve-environment=true
+EOF
+
+        mkdir -p "${SCHROOT_DIR}/etc/apt/preferences.d"
+        cat << EOF > "${SCHROOT_DIR}/etc/apt/preferences.d/isar-apt"
+Package: *
+Pin: release n=${DEBDISTRONAME}
+Pin-Priority: 1000
+EOF
+
+        # Prepare mount points
+        cp -rf "${SCHROOT_CONF}/sbuild" "${SBUILD_CONF_DIR}"
+        sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
+
+        fstab_isarapt="${DEPLOY_DIR}/isar-apt /isar-apt none rw,bind 0 0"
+        grep -qxF "${fstab_isarapt}" ${sbuild_fstab} || echo "${fstab_isarapt}" >> ${sbuild_fstab}
+
+        if [ -d ${DL_DIR} ]; then
+            fstab_downloads="${DL_DIR} /downloads none rw,bind 0 0"
+            grep -qxF "${fstab_downloads}" ${sbuild_fstab} || echo "${fstab_downloads}" >> ${sbuild_fstab}
+        fi
+EOSUDO
+}
+
+schroot_delete_configs() {
+    sudo -s <<'EOSUDO'
+        set -e
+        if [ -d "${SBUILD_CONF_DIR}" ]; then
+            rm -rf "${SBUILD_CONF_DIR}"
+        fi
+        rm -f "${SCHROOT_DIR}/etc/apt/preferences.d/isar-apt"
+        rm -f "${SCHROOT_CONF_FILE}"
+EOSUDO
+}
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index e9f92916..39e9c007 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -25,13 +25,14 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init"
 DISTRO_BOOTSTRAP_KEYFILES = ""
 THIRD_PARTY_APT_KEYFILES = ""
 DEPLOY_ISAR_BOOTSTRAP ?= ""
-DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
+DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,fakeroot,build-essential,debhelper"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_gnupg = ",gnupg"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = "${@https_support(d)}"
 DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}"
 BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}"
 
 inherit deb-dl-dir
+inherit sbuild
 
 python () {
     distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split()
@@ -398,6 +399,13 @@ EOSUDO
 
 addtask bootstrap before do_build after do_generate_keyrings
 
+do_schroot_config() {
+    schroot_create_configs
+}
+
+addtask schroot_config before do_bootstrap after do_generate_keyrings
+# TODO: remove configs from /etc/schroot
+
 CLEANFUNCS = "clean_deploy"
 clean_deploy() {
     rm -f "${DEPLOY_ISAR_BOOTSTRAP}"
-- 
2.20.1

[PATCH v2 04/24] linux-module: Do not use shell environment

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To make package build process independent of the shell environment we
should remove KDIR and PN passing through export call.
KDIR can be prepared during package build. This also will allow not to
rely on previous builddeps install task.
To pass PN variable we can just migrate to template-based debian/rules
file.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 .../files/debian/{rules => rules.tmpl}        | 14 ++++++++++-
 meta/recipes-kernel/linux-module/module.inc   | 23 +++++++------------
 2 files changed, 21 insertions(+), 16 deletions(-)
 rename meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%)

diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules.tmpl
similarity index 50%
rename from meta/recipes-kernel/linux-module/files/debian/rules
rename to meta/recipes-kernel/linux-module/files/debian/rules.tmpl
index 59720b37..d8f1d48a 100755
--- a/meta/recipes-kernel/linux-module/files/debian/rules
+++ b/meta/recipes-kernel/linux-module/files/debian/rules.tmpl
@@ -24,14 +24,26 @@ ifneq (,$(findstring 86,$(DEB_HOST_GNU_CPU)))
 export ARCH=x86
 endif
 
+# Custom kernels contain the build folder directly.
+KDIR := $(shell dpkg -L linux-headers-${KERNEL_NAME} | grep "/lib/modules/.*/build")
+ifeq ($(KDIR),)
+# Debian kernels install that folder indirectly via a dependency.
+KERNEL_DEP := $(shell dpkg-query -W -f '$${Depends}' linux-headers-${KERNEL_NAME} | sed 's/.*\(linux-headers-[[:graph:]]*\).*/\1/')
+KDIR := $(shell dpkg -L $(KERNEL_DEP) | grep "/lib/modules/.*/build")
+endif
+
+# With sbuild `dh clean` is called before chroot's apt database updated. So,
+# KDIR is empty at that moment and the override causes error. Will skip it.
+ifneq ($(KDIR),)
 override_dh_auto_clean:
 	$(MAKE) -C $(KDIR) M=$(PWD) clean
+endif
 
 override_dh_auto_build:
 	$(MAKE) -C $(KDIR) M=$(PWD) modules
 
 override_dh_auto_install:
-	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(PWD)/debian/$(PN) modules_install
+	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(PWD)/debian/${PN} modules_install
 
 %:
 	CFLAGS= LDFLAGS= dh $@ --parallel
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
index 0515621a..5b086233 100644
--- a/meta/recipes-kernel/linux-module/module.inc
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -22,8 +22,9 @@ AUTOLOAD ?= ""
 inherit dpkg
 
 TEMPLATE_FILES = "debian/control.tmpl \
-                  debian/changelog.tmpl"
-TEMPLATE_VARS += "KERNEL_NAME"
+                  debian/changelog.tmpl \
+                  debian/rules.tmpl"
+TEMPLATE_VARS += "KERNEL_NAME PN"
 
 do_prepare_build() {
     cp -r ${WORKDIR}/debian ${S}/
@@ -31,18 +32,10 @@ do_prepare_build() {
     for module in "${AUTOLOAD}"; do
         echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
     done
-}
 
-dpkg_runbuild_prepend() {
-    # Custom kernels contain the build folder directly.
-    export KDIR=$(dpkg -L --root=${BUILDCHROOT_DIR} linux-headers-${KERNEL_NAME} | \
-                  grep "/lib/modules/.*/build")
-    if [ -z "$KDIR" ]; then
-        # Debian kernels install that folder indirectly via a dependency.
-        KERNEL_DEP=$(dpkg-query -W -f '${Depends}' --admindir=${BUILDCHROOT_DIR}/var/lib/dpkg \
-                     linux-headers-${KERNEL_NAME} | sed 's/.*\(linux-headers-[[:graph:]]*\).*/\1/')
-        export KDIR=$(dpkg -L --root=${BUILDCHROOT_DIR} ${KERNEL_DEP} | \
-                      grep "/lib/modules/.*/build")
-    fi
-    export PN=${PN}
+    # remove templates from the source tree
+    find ${S}/debian -name *.tmpl | xargs rm -f
+
+    # restore execute permissions
+    chmod a+x ${S}/debian/rules
 }
-- 
2.20.1

[PATCH v2 05/24] u-boot: Do not use shell environment

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To make package build process independent of the shell environment we
should remove U_BOOT_CONFIG and U_BOOT_BIN passing through export call.
So we migrate to template-based debian/rules file.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .../u-boot/files/debian/{rules => rules.tmpl}          |  2 +-
 meta/recipes-bsp/u-boot/u-boot-custom.inc              | 10 ++++------
 2 files changed, 5 insertions(+), 7 deletions(-)
 rename meta/recipes-bsp/u-boot/files/debian/{rules => rules.tmpl} (94%)

diff --git a/meta/recipes-bsp/u-boot/files/debian/rules b/meta/recipes-bsp/u-boot/files/debian/rules.tmpl
similarity index 94%
rename from meta/recipes-bsp/u-boot/files/debian/rules
rename to meta/recipes-bsp/u-boot/files/debian/rules.tmpl
index 3d667620..806b01fe 100755
--- a/meta/recipes-bsp/u-boot/files/debian/rules
+++ b/meta/recipes-bsp/u-boot/files/debian/rules.tmpl
@@ -13,7 +13,7 @@ SET_CROSS_BUILD_TOOLS=CROSS_BUILD_TOOLS=y
 endif
 
 override_dh_auto_build:
-	$(MAKE) $(PARALLEL_MAKE) $(U_BOOT_CONFIG)
+	$(MAKE) $(PARALLEL_MAKE) ${U_BOOT_CONFIG}
 	$(MAKE) $(PARALLEL_MAKE) ${U_BOOT_BIN}
 	$(MAKE) -n u-boot-initial-env >/dev/null 2>&1; if [ $$? -ne 2 ]; then \
 		$(MAKE) $(PARALLEL_MAKE) u-boot-initial-env; \
diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc
index 9984d8cc..819206af 100644
--- a/meta/recipes-bsp/u-boot/u-boot-custom.inc
+++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc
@@ -26,8 +26,8 @@ python() {
 
 DEBIAN_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler, git"
 
-TEMPLATE_FILES = "debian/control.tmpl"
-TEMPLATE_VARS += "MACHINE DEBIAN_BUILD_DEPENDS"
+TEMPLATE_FILES = "debian/control.tmpl debian/rules.tmpl"
+TEMPLATE_VARS += "MACHINE DEBIAN_BUILD_DEPENDS U_BOOT_CONFIG U_BOOT_BIN"
 
 U_BOOT_TOOLS_PACKAGE ?= "0"
 U_BOOT_CONFIG_PACKAGE ?= "0"
@@ -70,9 +70,7 @@ u-boot-initial-env /etc
 fw_env.config      /etc
 EOF
     fi
-}
 
-dpkg_runbuild_prepend() {
-    export U_BOOT_CONFIG="${U_BOOT_CONFIG}"
-    export U_BOOT_BIN="${U_BOOT_BIN}"
+    # restore execute permissions
+    chmod a+x ${S}/debian/rules
 }
-- 
2.20.1

[PATCH v2 06/24] trusted-firmware: Do not use shell environment

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To make package build process independent of the shell environment we
should remove TF_A_PLATFORM and TF_A_EXTRA_BUILDARGS passing through
export call. So we migrate to template-based debian/rules file.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .../files/debian/{rules => rules.tmpl}                 |  4 ++--
 .../trusted-firmware-a/trusted-firmware-a-custom.inc   | 10 ++++------
 2 files changed, 6 insertions(+), 8 deletions(-)
 rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules => rules.tmpl} (77%)

diff --git a/meta/recipes-bsp/trusted-firmware-a/files/debian/rules b/meta/recipes-bsp/trusted-firmware-a/files/debian/rules.tmpl
similarity index 77%
rename from meta/recipes-bsp/trusted-firmware-a/files/debian/rules
rename to meta/recipes-bsp/trusted-firmware-a/files/debian/rules.tmpl
index 70e1dd66..6dbf44db 100755
--- a/meta/recipes-bsp/trusted-firmware-a/files/debian/rules
+++ b/meta/recipes-bsp/trusted-firmware-a/files/debian/rules.tmpl
@@ -12,8 +12,8 @@ export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)-
 endif
 
 override_dh_auto_build:
-	CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_MAKE) PLAT=$(TF_A_PLATFORM) \
-		$(TF_A_EXTRA_BUILDARGS)
+	CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_MAKE) PLAT=${TF_A_PLATFORM} \
+		${TF_A_EXTRA_BUILDARGS}
 
 %:
 	dh $@
diff --git a/meta/recipes-bsp/trusted-firmware-a/trusted-firmware-a-custom.inc b/meta/recipes-bsp/trusted-firmware-a/trusted-firmware-a-custom.inc
index 1142bfce..3fe4c415 100644
--- a/meta/recipes-bsp/trusted-firmware-a/trusted-firmware-a-custom.inc
+++ b/meta/recipes-bsp/trusted-firmware-a/trusted-firmware-a-custom.inc
@@ -22,8 +22,8 @@ DEBIAN_BUILD_DEPENDS ?= ""
 
 PROVIDES += "trusted-firmware-a-${TF_A_NAME}"
 
-TEMPLATE_FILES = "debian/control.tmpl"
-TEMPLATE_VARS += "TF_A_NAME DEBIAN_BUILD_DEPENDS"
+TEMPLATE_FILES = "debian/control.tmpl debian/rules.tmpl"
+TEMPLATE_VARS += "TF_A_NAME DEBIAN_BUILD_DEPENDS TF_A_PLATFORM TF_A_EXTRA_BUILDARGS"
 
 do_prepare_build() {
     cp -r ${WORKDIR}/debian ${S}/
@@ -35,9 +35,7 @@ do_prepare_build() {
         echo "build/${TF_A_PLATFORM}/$binary /usr/lib/trusted-firmware-a/${TF_A_NAME}/" >> \
             ${S}/debian/trusted-firmware-a-${TF_A_NAME}.install
     done
-}
 
-dpkg_runbuild_prepend() {
-    export TF_A_PLATFORM="${TF_A_PLATFORM}"
-    export TF_A_EXTRA_BUILDARGS="${TF_A_EXTRA_BUILDARGS}"
+    # restore execute permissions
+    chmod a+x ${S}/debian/rules
 }
-- 
2.20.1

[PATCH v2 07/24] optee-os: Do not use shell environment

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To make package build process independent of the shell environment we
should remove OPTEE_PLATFORM and OPTEE_EXTRA_BUILDARGS passing through
export call. So we migrate to template-based debian/rules file.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .../optee-os/files/debian/{rules => rules.tmpl}        |  4 ++--
 meta/recipes-bsp/optee-os/optee-os-custom.inc          | 10 ++++------
 2 files changed, 6 insertions(+), 8 deletions(-)
 rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl} (75%)

diff --git a/meta/recipes-bsp/optee-os/files/debian/rules b/meta/recipes-bsp/optee-os/files/debian/rules.tmpl
similarity index 75%
rename from meta/recipes-bsp/optee-os/files/debian/rules
rename to meta/recipes-bsp/optee-os/files/debian/rules.tmpl
index d2e9900f..9ab80dfe 100755
--- a/meta/recipes-bsp/optee-os/files/debian/rules
+++ b/meta/recipes-bsp/optee-os/files/debian/rules.tmpl
@@ -12,8 +12,8 @@ export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)-
 endif
 
 override_dh_auto_build:
-	CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_MAKE) PLATFORM=$(OPTEE_PLATFORM) \
-		$(OPTEE_EXTRA_BUILDARGS)
+	CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_MAKE) PLATFORM=${OPTEE_PLATFORM} \
+		${OPTEE_EXTRA_BUILDARGS}
 
 %:
 	dh $@
diff --git a/meta/recipes-bsp/optee-os/optee-os-custom.inc b/meta/recipes-bsp/optee-os/optee-os-custom.inc
index 1bd51969..e9eed54d 100644
--- a/meta/recipes-bsp/optee-os/optee-os-custom.inc
+++ b/meta/recipes-bsp/optee-os/optee-os-custom.inc
@@ -22,8 +22,8 @@ DEBIAN_BUILD_DEPENDS ?= "python3-pycryptodome:native, python3-pyelftools"
 
 PROVIDES += "optee-os-${OPTEE_NAME}"
 
-TEMPLATE_FILES = "debian/control.tmpl"
-TEMPLATE_VARS += "OPTEE_NAME DEBIAN_BUILD_DEPENDS"
+TEMPLATE_FILES = "debian/control.tmpl debian/rules.tmpl"
+TEMPLATE_VARS += "OPTEE_NAME DEBIAN_BUILD_DEPENDS OPTEE_PLATFORM OPTEE_EXTRA_BUILDARGS"
 
 # split strip platform flavor, if any, from the specified platform string
 OPTEE_PLATFORM_BASE = "${@d.getVar('OPTEE_PLATFORM').split('-')[0]}"
@@ -38,9 +38,7 @@ do_prepare_build() {
         echo "out/arm-plat-${OPTEE_PLATFORM_BASE}/core/$binary /usr/lib/optee-os/${OPTEE_NAME}/" >> \
             ${S}/debian/optee-os-${OPTEE_NAME}.install
     done
-}
 
-dpkg_runbuild_prepend() {
-    export OPTEE_PLATFORM="${OPTEE_PLATFORM}"
-    export OPTEE_EXTRA_BUILDARGS="${OPTEE_EXTRA_BUILDARGS}"
+    # restore execute permissions
+    chmod a+x ${S}/debian/rules
 }
-- 
2.20.1

[PATCH v2 08/24] kselftest: Do not use shell environment

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

To make package build process independent of the shell environment we
should remove KSELFTEST_ARGS passing through export call. The same
logic can be done by internal recipe parser and then pass variable to
template-based debian/rules file.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .../kselftest/files/{rules => rules.tmpl}     |  0
 meta/recipes-kernel/kselftest/kselftest.inc   | 21 +++++++------------
 2 files changed, 8 insertions(+), 13 deletions(-)
 rename meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%)

diff --git a/meta/recipes-kernel/kselftest/files/rules b/meta/recipes-kernel/kselftest/files/rules.tmpl
similarity index 100%
rename from meta/recipes-kernel/kselftest/files/rules
rename to meta/recipes-kernel/kselftest/files/rules.tmpl
diff --git a/meta/recipes-kernel/kselftest/kselftest.inc b/meta/recipes-kernel/kselftest/kselftest.inc
index 2a76028b..6187d8e4 100644
--- a/meta/recipes-kernel/kselftest/kselftest.inc
+++ b/meta/recipes-kernel/kselftest/kselftest.inc
@@ -33,26 +33,21 @@ DEBIAN_BUILD_DEPENDS ?= " \
     llvm:native, \
 "
 
-SRC_URI += "file://rules"
+SRC_URI += "file://rules.tmpl"
 S = "${WORKDIR}/linux-${PV}"
 
+TEMPLATE_FILES = "rules.tmpl"
+TEMPLATE_VARS += "KSELFTEST_ARGS"
+
 KSELFTEST_TARGETS ?= ""
 KSELFTEST_SKIP_TARGETS ?= ""
 KSELFTEST_FORCE_TARGETS ?= "0"
 
+KSELFTEST_ARGS = "${@ "TARGETS=\"${KSELFTEST_TARGETS}\"" if d.getVar('KSELFTEST_TARGETS', True) else ''}"
+KSELFTEST_ARGS_append = "${@ " FORCE_TARGETS=1" if d.getVar('KSELFTEST_FORCE_TARGETS', True) == '1' else ''}"
+KSELFTEST_ARGS_append .= "${@ " SKIP_TARGETS=\"${KSELFTEST_SKIP_TARGETS}\"" if d.getVar('KSELFTEST_SKIP_TARGETS', True) else ''}"
+
 do_prepare_build[cleandirs] += "${S}/debian"
 do_prepare_build() {
     deb_debianize
 }
-
-dpkg_runbuild_prepend() {
-    if [ -n "${KSELFTEST_TARGETS}" ];then
-        export KSELFTEST_ARGS="TARGETS=\"${KSELFTEST_TARGETS}\""
-    fi
-    if [ ${KSELFTEST_FORCE_TARGETS} -eq 1 ];then
-        export KSELFTEST_ARGS="${KSELFTEST_ARGS} FORCE_TARGETS=1"
-    fi
-    if [ -n "${KSELFTEST_SKIP_TARGETS}" ];then
-        export KSELFTEST_ARGS="${KSELFTEST_ARGS} SKIP_TARGETS=\"${KSELFTEST_SKIP_TARGETS}\""
-    fi
-}
-- 
2.20.1

[PATCH v2 09/24] dpkg: Build packages with sbuild

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

Use previously exported to schroot rootfs to build the package with
sbuild tool. Provide isar-apt as extra repository to be automatically
add to apt sources.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/dpkg.bbclass | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 86b47c28..e021e546 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -2,6 +2,7 @@
 # Copyright (C) 2015-2018 ilbers GmbH
 
 inherit dpkg-base
+inherit sbuild
 
 PACKAGE_ARCH ?= "${DISTRO_ARCH}"
 
@@ -28,10 +29,16 @@ do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
 
 addtask devshell after do_install_builddeps
 
+ISAR_APT_REPO ?= "deb [trusted=yes] file:///isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} ${DEBDISTRONAME} main"
+
 # Build package from sources using build script
 dpkg_runbuild() {
     E="${@ isar_export_proxies(d)}"
     export PARALLEL_MAKE="${PARALLEL_MAKE}"
-    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
-         /isar/build.sh ${PP}/${PPS} ${PACKAGE_ARCH} ${USE_CCACHE}
+
+    sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
+        --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} \
+        --starting-build-commands="runuser -u ${SCHROOT_USER} -- sh -c \"${SBUILD_PREBUILD:-:}\"" \
+        --no-run-lintian --no-run-piuparts --no-run-autopkgtest \
+        --build-dir=${WORKDIR} ${WORKDIR}/${PPS}
 }
-- 
2.20.1

[PATCH v2 10/24] sbuild: Introduce environment variables export API

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

In case there is no ability to pass variables through template
mechanism we need to keep fallback API to migrate from bare export.
Using it with optee-os-stm32mp15x recipe as example.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .../optee-os/optee-os-stm32mp15x_3.11.0.bb          |  2 +-
 meta/classes/sbuild.bbclass                         | 13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb b/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb
index 0fa2005d..19aa763a 100644
--- a/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb
+++ b/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb
@@ -21,5 +21,5 @@ OPTEE_BINARIES = "tee-header_v2.stm32 tee-pageable_v2.stm32 tee-pager_v2.stm32"
 # Set version manually to PV, the tarball does not contain any hint.
 # Alternative: pull from git and add git as build dependency.
 dpkg_runbuild_prepend() {
-    export TEE_IMPL_VERSION=${PV}
+    sbuild_export TEE_IMPL_VERSION=${PV}
 }
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index d6e5b926..5f9f7691 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -85,3 +85,16 @@ schroot_delete_configs() {
         rm -f "${SCHROOT_CONF_FILE}"
 EOSUDO
 }
+
+sbuild_export() {
+    SBUILD_CONFIG="${WORKDIR}/sbuild.conf"
+    VAR_LINE="'${1%%=*}' => '${1#*=}',"
+    if [ -s "${SBUILD_CONFIG}" ]; then
+        sed -i -e "\$i\\" -e "${VAR_LINE}" ${SBUILD_CONFIG}
+    else
+        echo "\$build_environment = {" > ${SBUILD_CONFIG}
+        echo "${VAR_LINE}" >> ${SBUILD_CONFIG}
+        echo "};" >> ${SBUILD_CONFIG}
+    fi
+    export SBUILD_CONFIG="${SBUILD_CONFIG}"
+}
-- 
2.20.1

[PATCH v2 11/24] dpkg-gbp: Migrate to schroot

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

Use schroot instead of chroot for git-buildpackage.

Also added /home/.git-downloads mount while it's used
as git alternates location.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-gbp.bbclass | 21 ++++------------
 meta/classes/sbuild.bbclass   | 45 +++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 16 deletions(-)

diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index ddf23ca5..7ebbf2e0 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -13,25 +13,14 @@ GBP_DEPENDS ?= "git-buildpackage pristine-tar"
 GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
 
 do_install_builddeps_append() {
-    dpkg_do_mounts
-    distro="${DISTRO}"
-    if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
-       distro="${HOST_DISTRO}"
-    fi
-    deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
-    sudo -E chroot ${BUILDCHROOT_DIR} \
-        apt-get install -y -o Debug::pkgProblemResolver=yes \
-                        --no-install-recommends --download-only ${GBP_DEPENDS}
-    deb_dl_dir_export "${BUILDCHROOT_DIR}" "${distro}"
-    sudo -E chroot ${BUILDCHROOT_DIR} \
-        apt-get install -y -o Debug::pkgProblemResolver=yes \
-                        --no-install-recommends ${GBP_DEPENDS}
-    dpkg_undo_mounts
+    schroot_install "${GBP_DEPENDS}"
 }
 
+SCHROOT_MOUNTS = "${WORKDIR}:${PP} ${GITDIR}:/home/.git-downloads"
+
 dpkg_runbuild_prepend() {
-    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
-        sh -c "cd ${PP}/${PPS} && gbp buildpackage --git-builder=/bin/true ${GBP_EXTRA_OPTIONS}"
+    schroot_run -d ${PP}/${PPS} -c ${SBUILD_CHROOT} -- \
+        gbp buildpackage --git-builder=/bin/true ${GBP_EXTRA_OPTIONS}
     # NOTE: `buildpackage --git-builder=/bin/true --git-pristine-tar` is used
     # for compatibility with gbp version froms debian-stretch. In newer distros
     # it's possible to use a subcommand `export-orig --pristine-tar`
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index 5f9f7691..b7a349e6 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -3,6 +3,8 @@
 
 SCHROOT_CONF ?= "/etc/schroot"
 
+SCHROOT_MOUNTS ?= ""
+
 python __anonymous() {
     import pwd
     d.setVar('SCHROOT_USER', pwd.getpwuid(os.geteuid()).pw_name)
@@ -98,3 +100,46 @@ sbuild_export() {
     fi
     export SBUILD_CONFIG="${SBUILD_CONFIG}"
 }
+
+schroot_install() {
+    APTS="$1"
+    #TODO deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
+    schroot -d / -c ${SBUILD_CHROOT_RW} -u root -- \
+        apt install -y -o Debug::pkgProblemResolver=yes \
+                    --no-install-recommends --download-only ${APTS}
+    #TODO deb_dl_dir_export "${BUILDCHROOT_DIR}" "${distro}"
+    schroot -d / -c ${SBUILD_CHROOT_RW} -u root -- \
+        apt install -y -o Debug::pkgProblemResolver=yes \
+                    --no-install-recommends ${APTS}
+}
+
+insert_mounts() {
+    sudo -s <<'EOSUDO'
+        ( flock 9
+        set -e
+        for mp in ${SCHROOT_MOUNTS}; do
+            FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind 0 0"
+            grep -qxF "${FSTAB_LINE}" ${SBUILD_CONF_DIR}/fstab || \
+                echo "${FSTAB_LINE}" >> ${SBUILD_CONF_DIR}/fstab
+        done
+        ) 9>'${SCHROOT_LOCKFILE}'
+EOSUDO
+}
+
+remove_mounts() {
+    sudo -s <<'EOSUDO'
+        ( flock 9
+        set -e
+        for mp in ${SCHROOT_MOUNTS}; do
+            FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind 0 0"
+            sed -i "\|${FSTAB_LINE}|d" ${SBUILD_CONF_DIR}/fstab
+        done
+        ) 9>'${SCHROOT_LOCKFILE}'
+EOSUDO
+}
+
+schroot_run() {
+    insert_mounts
+    schroot $@
+    remove_mounts
+}
-- 
2.20.1

[PATCH v2 12/24] linux-mainline: Move cfg fragment test to debian/rules

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

Perform all config fragments checking in Debian way.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 .../linux/linux-mainline_5.4.70.bb               | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb b/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb
index 980e137b..28e51c0d 100644
--- a/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb
+++ b/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb
@@ -25,11 +25,13 @@ LINUX_VERSION_EXTENSION = "-isar"
 
 # For testing purposes only
 dpkg_configure_kernel_append() {
-    if ! grep "# CONFIG_MTD is not set" ${S}/${KERNEL_BUILD_DIR}/.config && \
-       ! grep "# CONFIG_MTD_UBI is not set" ${S}/${KERNEL_BUILD_DIR}/.config; then
-        grep "# CONFIG_UBIFS_FS is not set" ${S}/${KERNEL_BUILD_DIR}/.config || \
-            bbfatal "Self-check failed: CONFIG_UBIFS_FS still enabled"
-    fi
-    grep "CONFIG_ROOT_NFS=y" ${S}/${KERNEL_BUILD_DIR}/.config || \
-        bbfatal "Self-check failed: CONFIG_ROOT_NFS not enabled"
+cat << EOF | sed -i '/^override_dh_auto_build/ r /dev/stdin' ${S}/debian/rules
+	if ! grep "# CONFIG_MTD is not set" \$(O)/.config && \\
+	   ! grep "# CONFIG_MTD_UBI is not set" \$(O)/.config; then \\
+	    grep "# CONFIG_UBIFS_FS is not set" \$(O)/.config || \\
+	        (echo "Self-check failed: CONFIG_UBIFS_FS still enabled" && exit 1); \\
+	fi
+	grep "CONFIG_ROOT_NFS=y" \$(O)/.config || \\
+	    (echo "Self-check failed: CONFIG_ROOT_NFS not enabled" && exit 1)
+EOF
 }
-- 
2.20.1

[PATCH v2 13/24] linux-custom: Prepare kernel config inside sbuild

[Uladzimir Bely]

From: Anton Mikanovich <amikan@ilbers.de>

Move prepare code inside sbuild hook to perform all the operations
inside schroot.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes/dpkg.bbclass                           |  4 ++++
 meta/classes/sbuild.bbclass                         | 10 +++++-----
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc |  8 --------
 meta/recipes-kernel/linux/linux-custom.inc          |  7 +++++--
 4 files changed, 14 insertions(+), 15 deletions(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index e021e546..6da61185 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -36,9 +36,13 @@ dpkg_runbuild() {
     E="${@ isar_export_proxies(d)}"
     export PARALLEL_MAKE="${PARALLEL_MAKE}"
 
+    schroot_create_configs
+
     sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
         --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} \
         --starting-build-commands="runuser -u ${SCHROOT_USER} -- sh -c \"${SBUILD_PREBUILD:-:}\"" \
         --no-run-lintian --no-run-piuparts --no-run-autopkgtest \
         --build-dir=${WORKDIR} ${WORKDIR}/${PPS}
+
+    schroot_delete_configs
 }
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index b7a349e6..a03883f0 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -19,7 +19,7 @@ python __anonymous() {
         d.setVar('SBUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
 }
 
-SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${DISTRO}-${SBUILD_HOST_ARCH}"
+SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${@os.getpid()}"
 SBUILD_CHROOT_RW ?= "${SBUILD_CHROOT}-rw"
 
 SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}"
@@ -102,6 +102,7 @@ sbuild_export() {
 }
 
 schroot_install() {
+    schroot_create_configs
     APTS="$1"
     #TODO deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
     schroot -d / -c ${SBUILD_CHROOT_RW} -u root -- \
@@ -111,35 +112,34 @@ schroot_install() {
     schroot -d / -c ${SBUILD_CHROOT_RW} -u root -- \
         apt install -y -o Debug::pkgProblemResolver=yes \
                     --no-install-recommends ${APTS}
+    schroot_delete_configs
 }
 
 insert_mounts() {
     sudo -s <<'EOSUDO'
-        ( flock 9
         set -e
         for mp in ${SCHROOT_MOUNTS}; do
             FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind 0 0"
             grep -qxF "${FSTAB_LINE}" ${SBUILD_CONF_DIR}/fstab || \
                 echo "${FSTAB_LINE}" >> ${SBUILD_CONF_DIR}/fstab
         done
-        ) 9>'${SCHROOT_LOCKFILE}'
 EOSUDO
 }
 
 remove_mounts() {
     sudo -s <<'EOSUDO'
-        ( flock 9
         set -e
         for mp in ${SCHROOT_MOUNTS}; do
             FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind 0 0"
             sed -i "\|${FSTAB_LINE}|d" ${SBUILD_CONF_DIR}/fstab
         done
-        ) 9>'${SCHROOT_LOCKFILE}'
 EOSUDO
 }
 
 schroot_run() {
+    schroot_create_configs
     insert_mounts
     schroot $@
     remove_mounts
+    schroot_delete_configs
 }
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 39e9c007..1b227028 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -32,7 +32,6 @@ DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else
 BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}"
 
 inherit deb-dl-dir
-inherit sbuild
 
 python () {
     distro_bootstrap_keys = (d.getVar("DISTRO_BOOTSTRAP_KEYS") or "").split()
@@ -399,13 +398,6 @@ EOSUDO
 
 addtask bootstrap before do_build after do_generate_keyrings
 
-do_schroot_config() {
-    schroot_create_configs
-}
-
-addtask schroot_config before do_bootstrap after do_generate_keyrings
-# TODO: remove configs from /etc/schroot
-
 CLEANFUNCS = "clean_deploy"
 clean_deploy() {
     rm -f "${DEPLOY_ISAR_BOOTSTRAP}"
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index ed89aa09..21115840 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -125,6 +125,9 @@ do_prepare_build_prepend() {
 	# remove templates from the source tree
 	find ${S}/debian -name *.tmpl | xargs rm -f
 
+	# restore execute permissions
+	chmod a+x ${S}/debian/rules
+
 	# rename install/remove hooks to match user-specified name for our linux-image package
 	mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst
 	mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm
@@ -164,9 +167,9 @@ dpkg_configure_kernel() {
 		(cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/)
 	fi
 
-	sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \
+	SBUILD_PREBUILD="                                                      \
 		export ARCH=${KERNEL_ARCH} &&                                  \
-		cd ${PP}/${PPS} &&                                             \
+		cd %p &&                                                       \
 		make O=${KERNEL_BUILD_DIR} ${config_target} &&                 \
 		./scripts/kconfig/merge_config.sh                              \
 			-O ${KERNEL_BUILD_DIR}/                                \
-- 
2.20.1

[PATCH v2 14/24] sbuild: Add recipes for host and target rootfs to run sbuild

[Uladzimir Bely]

Similar to buildchroot, we need a separate rootfs to be used
for schroot.

It's based on bootstrapped rootfs, but includes several
common build-related packages.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-base.bbclass                |  3 ++
 meta/classes/dpkg.bbclass                     |  1 -
 meta/classes/sbuild.bbclass                   |  7 ++--
 meta/conf/bitbake.conf                        |  2 ++
 .../isar-bootstrap/isar-bootstrap.inc         |  2 +-
 .../sbuild-chroot/sbuild-chroot-host.bb       | 13 +++++++
 .../sbuild-chroot/sbuild-chroot-target.bb     | 10 ++++++
 .../sbuild-chroot/sbuild-chroot.inc           | 36 +++++++++++++++++++
 8 files changed, 70 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
 create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 8bbc4165..acf967ac 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -4,6 +4,7 @@
 #
 # SPDX-License-Identifier: MIT
 
+inherit sbuild
 inherit buildchroot
 inherit debianize
 inherit terminal
@@ -214,6 +215,8 @@ python do_dpkg_build() {
 
 addtask dpkg_build before do_build
 
+do_dpkg_build[depends] = "${SCHROOT_DEP}"
+
 KEEP_INSTALLED_ON_CLEAN ?= "0"
 
 CLEANFUNCS += "deb_clean"
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 6da61185..d49648be 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -2,7 +2,6 @@
 # Copyright (C) 2015-2018 ilbers GmbH
 
 inherit dpkg-base
-inherit sbuild
 
 PACKAGE_ARCH ?= "${DISTRO_ARCH}"
 
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index a03883f0..80f9ef25 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -15,8 +15,13 @@ python __anonymous() {
     if mode == "0" or d.getVar('HOST_ARCH') ==  distro_arch or \
        (d.getVar('HOST_DISTRO') == "debian-stretch" and distro_arch == "i386"):
         d.setVar('SBUILD_HOST_ARCH', distro_arch)
+        d.setVar('SCHROOT_DIR', d.getVar('SCHROOT_TARGET_DIR'))
+        dep = "sbuild-chroot-target:do_build"
     else:
         d.setVar('SBUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
+        d.setVar('SCHROOT_DIR', d.getVar('SCHROOT_HOST_DIR'))
+        dep = "sbuild-chroot-host:do_build"
+    d.setVar('SCHROOT_DEP', dep)
 }
 
 SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${@os.getpid()}"
@@ -25,8 +30,6 @@ SBUILD_CHROOT_RW ?= "${SBUILD_CHROOT}-rw"
 SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}"
 SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}"
 
-SCHROOT_DIR ?= "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${SBUILD_HOST_ARCH}"
-
 schroot_create_configs() {
     sudo -s <<'EOSUDO'
         set -e
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index dce467b2..83508ccc 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -57,6 +57,8 @@ DL_DIR ?= "${TOPDIR}/downloads"
 SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
 BUILDCHROOT_HOST_DIR = "${DEPLOY_DIR_BUILDCHROOT}-host/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}"
 BUILDCHROOT_TARGET_DIR = "${DEPLOY_DIR_BUILDCHROOT}-target/${DISTRO}-${DISTRO_ARCH}"
+SCHROOT_HOST_DIR = "${DEPLOY_DIR}/schroot-host/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}"
+SCHROOT_TARGET_DIR = "${DEPLOY_DIR}/schroot-target/${DISTRO}-${DISTRO_ARCH}"
 SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${DISTRO}-${DISTRO_ARCH}"
 CACHE = "${TMPDIR}/cache"
 KERNEL_FILE ?= "vmlinuz"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 1b227028..e9f92916 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -25,7 +25,7 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init"
 DISTRO_BOOTSTRAP_KEYFILES = ""
 THIRD_PARTY_APT_KEYFILES = ""
 DEPLOY_ISAR_BOOTSTRAP ?= ""
-DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,fakeroot,build-essential,debhelper"
+DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_gnupg = ",gnupg"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = "${@https_support(d)}"
 DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}"
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
new file mode 100644
index 00000000..aa82846b
--- /dev/null
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
@@ -0,0 +1,13 @@
+# Root filesystem for packages building
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2021 ilbers GmbH
+
+DESCRIPTION = "Isar sbuild/schroot filesystem for host"
+
+SBUILD_VARIANT = "host"
+
+require sbuild-chroot.inc
+
+ROOTFS_ARCH = "${HOST_ARCH}"
+ROOTFS_DISTRO = "${HOST_DISTRO}"
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
new file mode 100644
index 00000000..d75d783b
--- /dev/null
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
@@ -0,0 +1,10 @@
+# Root filesystem for packages building
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2021 ilbers GmbH
+
+DESCRIPTION = "Isar sbuild/schroot filesystem for target"
+
+SBUILD_VARIANT = "target"
+
+require sbuild-chroot.inc
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
new file mode 100644
index 00000000..177a8a6d
--- /dev/null
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -0,0 +1,36 @@
+# Common part for build chroot filesystem.
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2021 ilbers GmbH
+
+LICENSE = "gpl-2.0"
+LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
+
+PV = "1.0"
+
+inherit rootfs
+
+SBUILD_CHROOT_PREINSTALL_COMMON = " \
+    fakeroot \
+    build-essential \
+    debhelper \
+"
+
+SBUILD_CHROOT_PREINSTALL ?= " \
+    ${SBUILD_CHROOT_PREINSTALL_COMMON} \
+"
+
+SBUILD_CHROOT_DIR = "${WORKDIR}/rootfs"
+ROOTFSDIR = "${SBUILD_CHROOT_DIR}"
+ROOTFS_PACKAGES = "${SBUILD_CHROOT_PREINSTALL}"
+
+# We don't need /etc/apt/sources.list.d/isar-apt.list' while it's handled by sbuild
+ROOTFS_CONFIGURE_COMMAND_remove = "rootfs_configure_isar_apt"
+
+DEPLOY_SCHROOT = "${@d.getVar('SCHROOT_' + d.getVar('SBUILD_VARIANT').upper() + '_DIR')}"
+
+do_sbuildchroot_deploy[dirs] = "${DEPLOY_DIR}/schroot-${SBUILD_VARIANT}"
+do_sbuildchroot_deploy() {
+    ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_SCHROOT}"
+}
+addtask sbuildchroot_deploy before do_build after do_rootfs
-- 
2.20.1

[PATCH v2 15/24] sbuild: Mount base-apt in schroot

[Uladzimir Bely]

This adds mounts for base-apt inside schroot and
adds import/export deb files to/from schroot.

So that it becomes possible to run second `cached` build
from local base-apt repo.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/sbuild.bbclass | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index 80f9ef25..474333e7 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -72,6 +72,8 @@ EOF
 
         fstab_isarapt="${DEPLOY_DIR}/isar-apt /isar-apt none rw,bind 0 0"
         grep -qxF "${fstab_isarapt}" ${sbuild_fstab} || echo "${fstab_isarapt}" >> ${sbuild_fstab}
+        fstab_baseapt="${REPO_BASE_DIR} /base-apt none rw,bind 0 0"
+        grep -qxF "${fstab_baseapt}" ${sbuild_fstab} || echo "${fstab_baseapt}" >> ${sbuild_fstab}
 
         if [ -d ${DL_DIR} ]; then
             fstab_downloads="${DL_DIR} /downloads none rw,bind 0 0"
@@ -107,11 +109,18 @@ sbuild_export() {
 schroot_install() {
     schroot_create_configs
     APTS="$1"
-    #TODO deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
+
+    distro="${DISTRO}"
+    if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+       distro="${HOST_DISTRO}"
+    fi
+
+    deb_dl_dir_import ${SCHROOT_DIR} ${distro}
     schroot -d / -c ${SBUILD_CHROOT_RW} -u root -- \
         apt install -y -o Debug::pkgProblemResolver=yes \
                     --no-install-recommends --download-only ${APTS}
-    #TODO deb_dl_dir_export "${BUILDCHROOT_DIR}" "${distro}"
+    deb_dl_dir_export ${SCHROOT_DIR} ${distro}
+
     schroot -d / -c ${SBUILD_CHROOT_RW} -u root -- \
         apt install -y -o Debug::pkgProblemResolver=yes \
                     --no-install-recommends ${APTS}
-- 
2.20.1

[PATCH v2 16/24] sbuild: Add sbuildshell task

[Uladzimir Bely]

Similar to devshell for buildchroot, sbuildshell allows
to enter schroot environment with interactive terminal

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/sbuild.bbclass | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index 474333e7..dcdd638b 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -155,3 +155,22 @@ schroot_run() {
     remove_mounts
     schroot_delete_configs
 }
+
+python do_sbuildshell() {
+    import sys
+
+    oe_lib_path = os.path.join(d.getVar('LAYERDIR_core'), 'lib')
+    sys.path.insert(0, oe_lib_path)
+
+    bb.build.exec_func('schroot_create_configs', d)
+
+    buildchroot = d.getVar('SBUILD_CHROOT')
+    rundir = "/"
+    termcmd = "schroot -d / -c {0} -u root -- sh -c 'cd {1}; $SHELL -i'"
+    oe_terminal(termcmd.format(buildchroot, rundir), "Isar devshell", d)
+
+    bb.build.exec_func('schroot_delete_configs', d)
+}
+
+addtask sbuildshell after do_prepare_build
+do_sbuildshell[nostamp] = "1"
-- 
2.20.1

[PATCH v2 17/24] dpkg-gbp: Preinstall gbp utils in schroot

[Uladzimir Bely]

This preinstalls gbp depends like git-buildpackage and pristine-tar
in basic schroot environment.

So, there is no need to do such kind of installation
with every gpb recipe like cowsay.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-gbp.bbclass                         | 4 ----
 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc | 2 ++
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index 7ebbf2e0..436fad0f 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -12,10 +12,6 @@ PATCHTOOL ?= "git"
 GBP_DEPENDS ?= "git-buildpackage pristine-tar"
 GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
 
-do_install_builddeps_append() {
-    schroot_install "${GBP_DEPENDS}"
-}
-
 SCHROOT_MOUNTS = "${WORKDIR}:${PP} ${GITDIR}:/home/.git-downloads"
 
 dpkg_runbuild_prepend() {
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
index 177a8a6d..b6b73550 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -12,6 +12,8 @@ inherit rootfs
 
 SBUILD_CHROOT_PREINSTALL_COMMON = " \
     fakeroot \
+    git-buildpackage \
+    pristine-tar \
     build-essential \
     debhelper \
 "
-- 
2.20.1

[PATCH v2 18/24] dpkg: Remove builddeps install task.

[Uladzimir Bely]

While builddeps are now handled interlnally by sbuild, we don't need
a task to preinstall them in buildchroot.

But for the second local (base-apt) build we need a way to keep
all dependencies in form of .deb files in DL_DIR. This is done
by executing additional commands in sbuild that copy them to/from
an externally mounted folder.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg.bbclass | 43 ++++++++++++++++++---------------------
 1 file changed, 20 insertions(+), 23 deletions(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index d49648be..d80deeab 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -5,29 +5,6 @@ inherit dpkg-base
 
 PACKAGE_ARCH ?= "${DISTRO_ARCH}"
 
-# Install build dependencies for package
-do_install_builddeps() {
-    dpkg_do_mounts
-    E="${@ isar_export_proxies(d)}"
-    distro="${DISTRO}"
-    if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
-       distro="${HOST_DISTRO}"
-    fi
-    deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
-    sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
-        ${PP}/${PPS} ${PACKAGE_ARCH} --download-only
-    deb_dl_dir_export "${BUILDCHROOT_DIR}" "${distro}"
-    sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
-        ${PP}/${PPS} ${PACKAGE_ARCH}
-    dpkg_undo_mounts
-}
-
-addtask install_builddeps after do_prepare_build before do_dpkg_build
-# apt and reprepro may not run in parallel, acquire the Isar lock
-do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
-
-addtask devshell after do_install_builddeps
-
 ISAR_APT_REPO ?= "deb [trusted=yes] file:///isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} ${DEBDISTRONAME} main"
 
 # Build package from sources using build script
@@ -37,11 +14,31 @@ dpkg_runbuild() {
 
     schroot_create_configs
 
+    distro="${DISTRO}"
+    if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+        distro="${HOST_DISTRO}"
+    fi
+
+    deb_dl_dir_import "${WORKDIR}/rootfs" "${distro}"
+
+    deb_dir="/var/cache/apt/archives/"
+    ext_deb_dir="/home/builder/${PN}/rootfs/${deb_dir}"
+
+    ( flock 9
+        grep -qxF '$apt_keep_downloaded_packages = 1;' ${SCHROOT_USER_HOME}/.sbuildrc ||
+            echo '$apt_keep_downloaded_packages = 1;' >> ${SCHROOT_USER_HOME}/.sbuildrc
+    ) 9>"${TMPDIR}/sbuildrc.lock"
+
     sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
         --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} \
         --starting-build-commands="runuser -u ${SCHROOT_USER} -- sh -c \"${SBUILD_PREBUILD:-:}\"" \
         --no-run-lintian --no-run-piuparts --no-run-autopkgtest \
+        --chroot-setup-commands="cp -n --no-preserve=owner ${ext_deb_dir}/*.deb -t ${deb_dir}/" \
+        --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \
+        --finished-build-commands="cp -n --no-preserve=owner ${deb_dir}/*.deb -t ${ext_deb_dir}/" \
         --build-dir=${WORKDIR} ${WORKDIR}/${PPS}
 
+    deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
+
     schroot_delete_configs
 }
-- 
2.20.1

[PATCH v2 19/24] dpkg-base: Switch devshell to use schroot

[Uladzimir Bely]

While packages are now build in schroot with sbuild, devshell
task should make terminal open inside sbuild environment.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-base.bbclass | 10 +++++-----
 meta/classes/sbuild.bbclass    |  3 +++
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index acf967ac..861b2cb7 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -256,16 +256,16 @@ python do_devshell() {
     oe_lib_path = os.path.join(d.getVar('LAYERDIR_core'), 'lib')
     sys.path.insert(0, oe_lib_path)
 
-    bb.build.exec_func('dpkg_do_mounts', d)
+    bb.build.exec_func('schroot_create_configs', d)
 
     isar_export_proxies(d)
 
-    buildchroot = d.getVar('BUILDCHROOT_DIR')
+    schroot = d.getVar('SBUILD_CHROOT')
     pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS'))
-    termcmd = "sudo -E chroot {0} sh -c 'cd {1}; $SHELL -i'"
-    oe_terminal(termcmd.format(buildchroot, pp_pps), "Isar devshell", d)
+    termcmd = "schroot -d / -c {0} -u root -- sh -c 'cd {1}; $SHELL -i'"
+    oe_terminal(termcmd.format(schroot, pp_pps), "Isar devshell", d)
 
-    bb.build.exec_func('dpkg_undo_mounts', d)
+    bb.build.exec_func('schroot_delete_configs', d)
 }
 
 addtask devshell after do_prepare_build
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index dcdd638b..799f210e 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -75,6 +75,9 @@ EOF
         fstab_baseapt="${REPO_BASE_DIR} /base-apt none rw,bind 0 0"
         grep -qxF "${fstab_baseapt}" ${sbuild_fstab} || echo "${fstab_baseapt}" >> ${sbuild_fstab}
 
+        fstab_pkgdir="${WORKDIR} /home/builder/${PN} none rw,bind 0 0"
+        grep -qxF "${fstab_pkgdir}" ${sbuild_fstab} || echo "${fstab_pkgdir}" >> ${sbuild_fstab}
+
         if [ -d ${DL_DIR} ]; then
             fstab_downloads="${DL_DIR} /downloads none rw,bind 0 0"
             grep -qxF "${fstab_downloads}" ${sbuild_fstab} || echo "${fstab_downloads}" >> ${sbuild_fstab}
-- 
2.20.1

[PATCH v2 20/24] dpkg-base: Switch apt_fetch and apt_unpack to use schroot

[Uladzimir Bely]

This moves downloading and unpacking deb-src to schroot environment.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-base.bbclass | 26 ++++++++------------------
 1 file changed, 8 insertions(+), 18 deletions(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 861b2cb7..f530df58 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -97,34 +97,25 @@ python() {
 }
 
 do_apt_fetch() {
-    dpkg_do_mounts
-    E="${@ isar_export_proxies(d)}"
-    sudo -E chroot ${BUILDCHROOT_DIR} /usr/bin/apt-get update \
-        -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
-        -o Dir::Etc::SourceParts="-" \
-        -o APT::Get::List-Cleanup="0"
-
+    schroot_create_configs
     for uri in "${SRC_APT}"; do
-        sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+        schroot -d / -c ${SBUILD_CHROOT} -- \
             sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}"
     done
-
-    dpkg_undo_mounts
+    schroot_delete_configs
 }
 
 addtask apt_fetch after do_unpack before do_apt_unpack
 do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
 
-# Add dependency from the correct buildchroot: host or target
-do_apt_fetch[depends] = "${BUILDCHROOT_DEP}"
+# Add dependency from the correct schroot: host or target
+do_apt_fetch[depends] = "${SCHROOT_DEP}"
 
 do_apt_unpack() {
     rm -rf ${S}
-    dpkg_do_mounts
-    E="${@ isar_export_proxies(d)}"
-
+    schroot_create_configs
     for uri in "${SRC_APT}"; do
-        sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+        schroot -d / -c ${SBUILD_CHROOT} -- \
             sh -c ' \
                 set -e
                 dscfile="$(apt-get -y -qq --print-uris --only-source source "${2}" | cut -d " " -f2 | grep -E "*.dsc")"
@@ -133,8 +124,7 @@ do_apt_unpack() {
                 dpkg-source -x "${dscfile}" "${PPS}"' \
                     my_script "${DISTRO}" "${uri}"
     done
-
-    dpkg_undo_mounts
+    schroot_delete_configs
 }
 
 addtask apt_unpack after do_apt_fetch before do_patch
-- 
2.20.1

[PATCH v2 21/24] dpkg-base: Cleanup from buildchroot parts.

[Uladzimir Bely]

This removes remaining buildchroot-related tasks and variables
from dpkg-base class while there are not used anymore.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-base.bbclass | 43 +++-------------------------------
 meta/classes/dpkg.bbclass      |  4 ----
 2 files changed, 3 insertions(+), 44 deletions(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index f530df58..d01ef5e8 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -5,7 +5,6 @@
 # SPDX-License-Identifier: MIT
 
 inherit sbuild
-inherit buildchroot
 inherit debianize
 inherit terminal
 inherit repository
@@ -163,29 +162,7 @@ addtask prepare_build after do_patch do_transform_template before do_dpkg_build
 # deployed to isar-apt
 do_prepare_build[deptask] = "do_deploy_deb"
 
-BUILDROOT = "${BUILDCHROOT_DIR}/${PP}"
-
-dpkg_do_mounts() {
-    mkdir -p ${BUILDROOT}
-    sudo mount --bind ${WORKDIR} ${BUILDROOT}
-
-    buildchroot_do_mounts
-}
-
-dpkg_undo_mounts() {
-    i=0
-    while ! sudo umount ${BUILDROOT}; do
-        sleep 0.1
-        if [ `expr $i % 100` -eq 0 ] ; then
-            bbwarn "${BUILDROOT}: Couldn't unmount ($i), retrying..."
-        fi
-        if [ $i -ge 10000 ]; then
-            bbfatal "${BUILDROOT}: Couldn't unmount after timeout"
-        fi
-        i=`expr $i + 1`
-    done
-    sudo rmdir ${BUILDROOT}
-}
+do_prepare_build[depends] = "${SCHROOT_DEP}"
 
 # Placeholder for actual dpkg_runbuild() implementation
 dpkg_runbuild() {
@@ -193,22 +170,17 @@ dpkg_runbuild() {
 }
 
 python do_dpkg_build() {
-    lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
-                             shared=True)
-    bb.build.exec_func("dpkg_do_mounts", d)
+    bb.build.exec_func('schroot_create_configs', d)
     try:
         bb.build.exec_func("dpkg_runbuild", d)
     finally:
-        bb.build.exec_func("dpkg_undo_mounts", d)
-        bb.utils.unlockfile(lock)
+        bb.build.exec_func('schroot_delete_configs', d)
 }
 
 addtask dpkg_build before do_build
 
 do_dpkg_build[depends] = "${SCHROOT_DEP}"
 
-KEEP_INSTALLED_ON_CLEAN ?= "0"
-
 CLEANFUNCS += "deb_clean"
 
 deb_clean() {
@@ -217,15 +189,6 @@ deb_clean() {
         for d in ${DEBS}; do
             repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \
                 "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${d}"
-            if [ "${KEEP_INSTALLED_ON_CLEAN}" = "1" ]; then
-                continue;
-            fi
-            package=$(basename "${d}")
-            package_remove="/usr/bin/apt-get remove -y ${package%%_*}"
-            sudo -E chroot ${BUILDCHROOT_DIR} ${package_remove} || true
-            if [ "${BUILDCHROOT_DIR}" != "${BUILDCHROOT_TARGET_DIR}" ]; then
-                    sudo -E chroot ${BUILDCHROOT_TARGET_DIR} ${package_remove} || true
-            fi
         done
     fi
 }
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index d80deeab..355da541 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -12,8 +12,6 @@ dpkg_runbuild() {
     E="${@ isar_export_proxies(d)}"
     export PARALLEL_MAKE="${PARALLEL_MAKE}"
 
-    schroot_create_configs
-
     distro="${DISTRO}"
     if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
         distro="${HOST_DISTRO}"
@@ -39,6 +37,4 @@ dpkg_runbuild() {
         --build-dir=${WORKDIR} ${WORKDIR}/${PPS}
 
     deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
-
-    schroot_delete_configs
 }
-- 
2.20.1

[PATCH v2 22/24] dpkg-gbp: Use host tools for dsc preparation

[Uladzimir Bely]

Instead of preinstalling gbp-related packages in sbuild chroot
we will use their host versions in order to keep sbuild environment
as clean as possible.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 doc/user_manual.md                                    | 4 +++-
 meta/classes/dpkg-gbp.bbclass                         | 4 +++-
 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc | 2 --
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 535d456a..84c99604 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -78,7 +78,9 @@ apt install \
   qemu \
   qemu-user-static \
   reprepro \
-  sudo
+  sudo \
+  git-buildpackage \
+  pristine-tar
 ```
 
 If your host is >= buster, also install the following package.
diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index 436fad0f..c9cc487e 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -15,8 +15,10 @@ GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
 SCHROOT_MOUNTS = "${WORKDIR}:${PP} ${GITDIR}:/home/.git-downloads"
 
 dpkg_runbuild_prepend() {
-    schroot_run -d ${PP}/${PPS} -c ${SBUILD_CHROOT} -- \
+    sh -c "
+        cd ${WORKDIR}/${PPS}
         gbp buildpackage --git-builder=/bin/true ${GBP_EXTRA_OPTIONS}
+    "
     # NOTE: `buildpackage --git-builder=/bin/true --git-pristine-tar` is used
     # for compatibility with gbp version froms debian-stretch. In newer distros
     # it's possible to use a subcommand `export-orig --pristine-tar`
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
index b6b73550..177a8a6d 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -12,8 +12,6 @@ inherit rootfs
 
 SBUILD_CHROOT_PREINSTALL_COMMON = " \
     fakeroot \
-    git-buildpackage \
-    pristine-tar \
     build-essential \
     debhelper \
 "
-- 
2.20.1

[PATCH v2 23/24] doc: Add sbuild-related documentation

[Uladzimir Bely]

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 doc/user_manual.md | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 84c99604..01a7c5be 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -83,6 +83,24 @@ apt install \
   pristine-tar
 ```
 
+Additional setup is required since `sbuild` is now used for package build.
+Install the following packages:
+```
+apt install \
+  sbuild \
+  schroot
+```
+Also, user who runs isar should be added to `sbuild` group.
+
+**NOTE:** sbuild version (<=0.78.1) packaged in Debian Buster doesn't support
+`$apt_keep_downloaded_packages` option which is required in Isar for
+populating `${DL_DIR}/deb`. So, host `sbuild` in this case should be manually
+upgraded to >=0.81.2 version from Debian Bullseye.
+
+```
+sudo gpasswd -a <username> sbuild
+```
+
 If your host is >= buster, also install the following package.
 ```
 apt install python3-distutils
-- 
2.20.1

[PATCH v2 24/24] sbuild: Replace isar-apt mounting with copying

[Uladzimir Bely]

sbuild internally doesn apt-get update and apt-get install <builddeps>.
If isar-apt repo is changed at that time by deploy_deb task for another
package going in parallel, apt won't be able to download dependency
from this repo.
We may copletely lock isar-apt completely for dpkg_build, but this
slows (serializes) the whole build. Instead, it's better to copy
current isar-apt into the package workidir under the lock and use
copied isar-apt repo for building.
While "-l' option for `cp` is used, it won't occupy much additional
space on disk.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg.bbclass   | 13 ++++++++++++-
 meta/classes/sbuild.bbclass |  2 --
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 355da541..a9a2b3e2 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -5,7 +5,18 @@ inherit dpkg-base
 
 PACKAGE_ARCH ?= "${DISTRO_ARCH}"
 
-ISAR_APT_REPO ?= "deb [trusted=yes] file:///isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} ${DEBDISTRONAME} main"
+ISAR_APT_REPO ?= "deb [trusted=yes] file:///home/builder/${PN}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} ${DEBDISTRONAME} main"
+
+# Install build dependencies for package
+do_install_isarapt() {
+    # Make local copy of isar-apt not affected by other parallel tasks
+    mkdir -p ${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}
+    rm -rf ${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/*
+    cp -Rl ${REPO_ISAR_DIR} ${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}
+}
+
+addtask install_isarapt after do_prepare_build before do_dpkg_build
+do_install_isarapt[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
 
 # Build package from sources using build script
 dpkg_runbuild() {
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index 799f210e..73a5fd6a 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -70,8 +70,6 @@ EOF
         cp -rf "${SCHROOT_CONF}/sbuild" "${SBUILD_CONF_DIR}"
         sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
 
-        fstab_isarapt="${DEPLOY_DIR}/isar-apt /isar-apt none rw,bind 0 0"
-        grep -qxF "${fstab_isarapt}" ${sbuild_fstab} || echo "${fstab_isarapt}" >> ${sbuild_fstab}
         fstab_baseapt="${REPO_BASE_DIR} /base-apt none rw,bind 0 0"
         grep -qxF "${fstab_baseapt}" ${sbuild_fstab} || echo "${fstab_baseapt}" >> ${sbuild_fstab}
 
-- 
2.20.1

Re: [PATCH v2 04/24] linux-module: Do not use shell environment

[Jan Kiszka]

On 19.11.21 13:13, Uladzimir Bely wrote:
> From: Anton Mikanovich <amikan@ilbers.de>
> 
> To make package build process independent of the shell environment we
> should remove KDIR and PN passing through export call.
> KDIR can be prepared during package build. This also will allow not to
> rely on previous builddeps install task.
> To pass PN variable we can just migrate to template-based debian/rules
> file.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> ---
>  .../files/debian/{rules => rules.tmpl}        | 14 ++++++++++-
>  meta/recipes-kernel/linux-module/module.inc   | 23 +++++++------------
>  2 files changed, 21 insertions(+), 16 deletions(-)
>  rename meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%)
> 
> diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules.tmpl
> similarity index 50%
> rename from meta/recipes-kernel/linux-module/files/debian/rules
> rename to meta/recipes-kernel/linux-module/files/debian/rules.tmpl
> index 59720b37..d8f1d48a 100755
> --- a/meta/recipes-kernel/linux-module/files/debian/rules
> +++ b/meta/recipes-kernel/linux-module/files/debian/rules.tmpl
> @@ -24,14 +24,26 @@ ifneq (,$(findstring 86,$(DEB_HOST_GNU_CPU)))
>  export ARCH=x86
>  endif
>  
> +# Custom kernels contain the build folder directly.
> +KDIR := $(shell dpkg -L linux-headers-${KERNEL_NAME} | grep "/lib/modules/.*/build")
> +ifeq ($(KDIR),)
> +# Debian kernels install that folder indirectly via a dependency.
> +KERNEL_DEP := $(shell dpkg-query -W -f '$${Depends}' linux-headers-${KERNEL_NAME} | sed 's/.*\(linux-headers-[[:graph:]]*\).*/\1/')

Could we wrap this long line?

> +KDIR := $(shell dpkg -L $(KERNEL_DEP) | grep "/lib/modules/.*/build")
> +endif
> +
> +# With sbuild `dh clean` is called before chroot's apt database updated. So,
> +# KDIR is empty at that moment and the override causes error. Will skip it.
> +ifneq ($(KDIR),)
>  override_dh_auto_clean:
>  	$(MAKE) -C $(KDIR) M=$(PWD) clean
> +endif

This looks wrong, or at least fragile. Why can we live without a proper
clean on the kernel tree?

>  
>  override_dh_auto_build:
>  	$(MAKE) -C $(KDIR) M=$(PWD) modules
>  
>  override_dh_auto_install:
> -	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(PWD)/debian/$(PN) modules_install
> +	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(PWD)/debian/${PN} modules_install
>  
>  %:
>  	CFLAGS= LDFLAGS= dh $@ --parallel
> diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
> index 0515621a..5b086233 100644
> --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -22,8 +22,9 @@ AUTOLOAD ?= ""
>  inherit dpkg
>  
>  TEMPLATE_FILES = "debian/control.tmpl \
> -                  debian/changelog.tmpl"
> -TEMPLATE_VARS += "KERNEL_NAME"
> +                  debian/changelog.tmpl \
> +                  debian/rules.tmpl"
> +TEMPLATE_VARS += "KERNEL_NAME PN"
>  
>  do_prepare_build() {
>      cp -r ${WORKDIR}/debian ${S}/
> @@ -31,18 +32,10 @@ do_prepare_build() {
>      for module in "${AUTOLOAD}"; do
>          echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
>      done
> -}
>  
> -dpkg_runbuild_prepend() {
> -    # Custom kernels contain the build folder directly.
> -    export KDIR=$(dpkg -L --root=${BUILDCHROOT_DIR} linux-headers-${KERNEL_NAME} | \
> -                  grep "/lib/modules/.*/build")
> -    if [ -z "$KDIR" ]; then
> -        # Debian kernels install that folder indirectly via a dependency.
> -        KERNEL_DEP=$(dpkg-query -W -f '${Depends}' --admindir=${BUILDCHROOT_DIR}/var/lib/dpkg \
> -                     linux-headers-${KERNEL_NAME} | sed 's/.*\(linux-headers-[[:graph:]]*\).*/\1/')
> -        export KDIR=$(dpkg -L --root=${BUILDCHROOT_DIR} ${KERNEL_DEP} | \
> -                      grep "/lib/modules/.*/build")
> -    fi
> -    export PN=${PN}
> +    # remove templates from the source tree
> +    find ${S}/debian -name *.tmpl | xargs rm -f

Hmm...

> +
> +    # restore execute permissions
> +    chmod a+x ${S}/debian/rules

The mode should be fixed (aligned to the template) in template.bbclass
to avoid bothering every recipe with such boilerplate code. I'm seeing
this in other changes as well.

With all those "do not use shell env" - how will a downstream user
detect that a recipe is not properly converted? That looks like a nice
trap during migration to sbuild (because of silent / non-obvious errors).

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 04/24] linux-module: Do not use shell environment

[Jan Kiszka]

On 19.11.21 13:44, Jan Kiszka wrote:
> On 19.11.21 13:13, Uladzimir Bely wrote:
>> From: Anton Mikanovich <amikan@ilbers.de>
>>
>> To make package build process independent of the shell environment we
>> should remove KDIR and PN passing through export call.
>> KDIR can be prepared during package build. This also will allow not to
>> rely on previous builddeps install task.
>> To pass PN variable we can just migrate to template-based debian/rules
>> file.
>>
>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
>> ---
>>  .../files/debian/{rules => rules.tmpl}        | 14 ++++++++++-
>>  meta/recipes-kernel/linux-module/module.inc   | 23 +++++++------------
>>  2 files changed, 21 insertions(+), 16 deletions(-)
>>  rename meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%)
>>
>> diff --git a/meta/recipes-kernel/linux-module/files/debian/rules b/meta/recipes-kernel/linux-module/files/debian/rules.tmpl
>> similarity index 50%
>> rename from meta/recipes-kernel/linux-module/files/debian/rules
>> rename to meta/recipes-kernel/linux-module/files/debian/rules.tmpl
>> index 59720b37..d8f1d48a 100755
>> --- a/meta/recipes-kernel/linux-module/files/debian/rules
>> +++ b/meta/recipes-kernel/linux-module/files/debian/rules.tmpl
>> @@ -24,14 +24,26 @@ ifneq (,$(findstring 86,$(DEB_HOST_GNU_CPU)))
>>  export ARCH=x86
>>  endif
>>  
>> +# Custom kernels contain the build folder directly.
>> +KDIR := $(shell dpkg -L linux-headers-${KERNEL_NAME} | grep "/lib/modules/.*/build")
>> +ifeq ($(KDIR),)
>> +# Debian kernels install that folder indirectly via a dependency.
>> +KERNEL_DEP := $(shell dpkg-query -W -f '$${Depends}' linux-headers-${KERNEL_NAME} | sed 's/.*\(linux-headers-[[:graph:]]*\).*/\1/')
> 
> Could we wrap this long line?
> 
>> +KDIR := $(shell dpkg -L $(KERNEL_DEP) | grep "/lib/modules/.*/build")
>> +endif
>> +
>> +# With sbuild `dh clean` is called before chroot's apt database updated. So,
>> +# KDIR is empty at that moment and the override causes error. Will skip it.
>> +ifneq ($(KDIR),)
>>  override_dh_auto_clean:
>>  	$(MAKE) -C $(KDIR) M=$(PWD) clean
>> +endif
> 
> This looks wrong, or at least fragile. Why can we live without a proper
> clean on the kernel tree?
> 
>>  
>>  override_dh_auto_build:
>>  	$(MAKE) -C $(KDIR) M=$(PWD) modules
>>  
>>  override_dh_auto_install:
>> -	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(PWD)/debian/$(PN) modules_install
>> +	$(MAKE) -C $(KDIR) M=$(PWD) INSTALL_MOD_PATH=$(PWD)/debian/${PN} modules_install
>>  
>>  %:
>>  	CFLAGS= LDFLAGS= dh $@ --parallel
>> diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
>> index 0515621a..5b086233 100644
>> --- a/meta/recipes-kernel/linux-module/module.inc
>> +++ b/meta/recipes-kernel/linux-module/module.inc
>> @@ -22,8 +22,9 @@ AUTOLOAD ?= ""
>>  inherit dpkg
>>  
>>  TEMPLATE_FILES = "debian/control.tmpl \
>> -                  debian/changelog.tmpl"
>> -TEMPLATE_VARS += "KERNEL_NAME"
>> +                  debian/changelog.tmpl \
>> +                  debian/rules.tmpl"
>> +TEMPLATE_VARS += "KERNEL_NAME PN"
>>  
>>  do_prepare_build() {
>>      cp -r ${WORKDIR}/debian ${S}/
>> @@ -31,18 +32,10 @@ do_prepare_build() {
>>      for module in "${AUTOLOAD}"; do
>>          echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
>>      done
>> -}
>>  
>> -dpkg_runbuild_prepend() {
>> -    # Custom kernels contain the build folder directly.
>> -    export KDIR=$(dpkg -L --root=${BUILDCHROOT_DIR} linux-headers-${KERNEL_NAME} | \
>> -                  grep "/lib/modules/.*/build")
>> -    if [ -z "$KDIR" ]; then
>> -        # Debian kernels install that folder indirectly via a dependency.
>> -        KERNEL_DEP=$(dpkg-query -W -f '${Depends}' --admindir=${BUILDCHROOT_DIR}/var/lib/dpkg \
>> -                     linux-headers-${KERNEL_NAME} | sed 's/.*\(linux-headers-[[:graph:]]*\).*/\1/')
>> -        export KDIR=$(dpkg -L --root=${BUILDCHROOT_DIR} ${KERNEL_DEP} | \
>> -                      grep "/lib/modules/.*/build")
>> -    fi
>> -    export PN=${PN}
>> +    # remove templates from the source tree
>> +    find ${S}/debian -name *.tmpl | xargs rm -f
> 
> Hmm...
> 
>> +
>> +    # restore execute permissions
>> +    chmod a+x ${S}/debian/rules
> 
> The mode should be fixed (aligned to the template) in template.bbclass
> to avoid bothering every recipe with such boilerplate code. I'm seeing
> this in other changes as well.
> 
> With all those "do not use shell env" - how will a downstream user
> detect that a recipe is not properly converted? That looks like a nice
> trap during migration to sbuild (because of silent / non-obvious errors).
> 

Oh, and then you still also add sbuild_export - why not use that for
cases that are ugly to convert (like this one)?

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Henning Schild]

Did not look into any of the patches so far ... but the number 24 makes
it close to impossible to review. Can this q be split up to ease review?

Henning

Am Fri, 19 Nov 2021 13:13:09 +0100
schrieb Uladzimir Bely <ubely@ilbers.de>:

> This is a patchset showing how sbuild/schroot tools can be integrated
> into Isar build system.
> 
> Base schroot image is created with sbuild-chroot-target (or
> sbuild-chroot-host in case of cross-build) recipe. These images are
> based on isar-bootstrap ones, but include some build-related stuff
> preinstalled.
> 
> To use this changes you need to have sbuild and schroot installed and
> configured. Current user should be added to sbuild group.
> 
> Changes since v1:
> - parallel builds with different BUILD_DIR are supported
> - parallel multiconfig targets in one build are supported
> - per-task schroot configuration in /etc/schroot/ is now used
> - patchset now passes Jenkins CI (so patches changes RFC => PATCH)
> 
> Current limitations:
> - there is an unsolved problem with building foreigh architectures
> using kas-docker.
> 
> - qemuamd64 images are buildable in kas-docker, but some changes
> in kas are reqired (like installing additional packages and adding
> 'builder' user to 'sbuild group). I used the following changes to
> build ghcr.io/siemens/kas/kas-isar:2.6.2-sbuild image based on
> current ghcr.io/siemens/kas/kas-isar:2.6.2:
> https://github.com/WiseLord/kas/commit/5b7b02aa33
> 
> - parallel building of several packages requires more free space
> than current buildchroot-based architecture uses. This happens due
> the sbuild architecture which uses some basic layer (common for all
> packages) and per-package separate layer (where builddeps are
> downloaded and installed).
> 
> - recently merged to 'next' ccache support is not yet implemented
> in case of sbuild used.
> 
> For experiments, the code can be taken from Isar 'ubely/sbuild'
> branch. This is an dev branch, so sometimes it can be force-pushed
> when some fixes are applied of rebased to 'next' branch.
> 
> Anton Mikanovich (12):
>   dpkg: Install raw package files to source root
>   isar-bootstrap: Export bootstrap to schroot config
>   linux-module: Do not use shell environment
>   u-boot: Do not use shell environment
>   trusted-firmware: Do not use shell environment
>   optee-os: Do not use shell environment
>   kselftest: Do not use shell environment
>   dpkg: Build packages with sbuild
>   sbuild: Introduce environment variables export API
>   dpkg-gbp: Migrate to schroot
>   linux-mainline: Move cfg fragment test to debian/rules
>   linux-custom: Prepare kernel config inside sbuild
> 
> Uladzimir Bely (12):
>   dpkg-gbp: Use separate command to export tarball
>   sbuild: Add recipes for host and target rootfs to run sbuild
>   sbuild: Mount base-apt in schroot
>   sbuild: Add sbuildshell task
>   dpkg-gbp: Preinstall gbp utils in schroot
>   dpkg: Remove builddeps install task.
>   dpkg-base: Switch devshell to use schroot
>   dpkg-base: Switch apt_fetch and apt_unpack to use schroot
>   dpkg-base: Cleanup from buildchroot parts.
>   dpkg-gbp: Use host tools for dsc preparation
>   doc: Add sbuild-related documentation
>   sbuild: Replace isar-apt mounting with copying
> 
>  doc/user_manual.md                            |  22 ++-
>  .../optee-os/optee-os-stm32mp15x_3.11.0.bb    |   2 +-
>  .../linux/linux-mainline_5.4.70.bb            |  16 +-
>  meta/classes/dpkg-base.bbclass                |  80 ++------
>  meta/classes/dpkg-gbp.bbclass                 |  25 +--
>  meta/classes/dpkg-raw.bbclass                 |   8 +-
>  meta/classes/dpkg.bbclass                     |  56 +++---
>  meta/classes/sbuild.bbclass                   | 177
> ++++++++++++++++++ meta/conf/bitbake.conf                        |
> 2 + .../files/debian/{rules => rules.tmpl}        |   4 +-
>  meta/recipes-bsp/optee-os/optee-os-custom.inc |  10 +-
>  .../files/debian/{rules => rules.tmpl}        |   4 +-
>  .../trusted-firmware-a-custom.inc             |  10 +-
>  .../u-boot/files/debian/{rules => rules.tmpl} |   2 +-
>  meta/recipes-bsp/u-boot/u-boot-custom.inc     |  10 +-
>  .../sbuild-chroot/sbuild-chroot-host.bb       |  13 ++
>  .../sbuild-chroot/sbuild-chroot-target.bb     |  10 +
>  .../sbuild-chroot/sbuild-chroot.inc           |  36 ++++
>  .../kselftest/files/{rules => rules.tmpl}     |   0
>  meta/recipes-kernel/kselftest/kselftest.inc   |  21 +--
>  .../files/debian/{rules => rules.tmpl}        |  14 +-
>  meta/recipes-kernel/linux-module/module.inc   |  23 +--
>  meta/recipes-kernel/linux/linux-custom.inc    |   7 +-
>  23 files changed, 382 insertions(+), 170 deletions(-)
>  create mode 100644 meta/classes/sbuild.bbclass
>  rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl}
> (75%) rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules
> => rules.tmpl} (77%) rename
> meta/recipes-bsp/u-boot/files/debian/{rules => rules.tmpl} (94%)
> create mode 100644
> meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode
> 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb
> create mode 100644
> meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc rename
> meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%)
> rename meta/recipes-kernel/linux-module/files/debian/{rules =>
> rules.tmpl} (50%)
> 

Re: [PATCH v2 10/24] sbuild: Introduce environment variables export API

[Jan Kiszka]

On 19.11.21 13:13, Uladzimir Bely wrote:
> From: Anton Mikanovich <amikan@ilbers.de>
> 
> In case there is no ability to pass variables through template
> mechanism we need to keep fallback API to migrate from bare export.
> Using it with optee-os-stm32mp15x recipe as example.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  .../optee-os/optee-os-stm32mp15x_3.11.0.bb          |  2 +-
>  meta/classes/sbuild.bbclass                         | 13 +++++++++++++
>  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb b/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb
> index 0fa2005d..19aa763a 100644
> --- a/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb
> +++ b/meta-isar/recipes-bsp/optee-os/optee-os-stm32mp15x_3.11.0.bb
> @@ -21,5 +21,5 @@ OPTEE_BINARIES = "tee-header_v2.stm32 tee-pageable_v2.stm32 tee-pager_v2.stm32"
>  # Set version manually to PV, the tarball does not contain any hint.
>  # Alternative: pull from git and add git as build dependency.
>  dpkg_runbuild_prepend() {
> -    export TEE_IMPL_VERSION=${PV}
> +    sbuild_export TEE_IMPL_VERSION=${PV}

build_export - avoids having to rename this once again when we should
switch to something != sbuild.

Again, I think this is the nicer way to make the recipes sbuild
compatible (less invasive, less boilerplate than templating).

Jan

>  }
> diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
> index d6e5b926..5f9f7691 100644
> --- a/meta/classes/sbuild.bbclass
> +++ b/meta/classes/sbuild.bbclass
> @@ -85,3 +85,16 @@ schroot_delete_configs() {
>          rm -f "${SCHROOT_CONF_FILE}"
>  EOSUDO
>  }
> +
> +sbuild_export() {
> +    SBUILD_CONFIG="${WORKDIR}/sbuild.conf"
> +    VAR_LINE="'${1%%=*}' => '${1#*=}',"
> +    if [ -s "${SBUILD_CONFIG}" ]; then
> +        sed -i -e "\$i\\" -e "${VAR_LINE}" ${SBUILD_CONFIG}
> +    else
> +        echo "\$build_environment = {" > ${SBUILD_CONFIG}
> +        echo "${VAR_LINE}" >> ${SBUILD_CONFIG}
> +        echo "};" >> ${SBUILD_CONFIG}
> +    fi
> +    export SBUILD_CONFIG="${SBUILD_CONFIG}"
> +}
> 

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Jan Kiszka]

On 19.11.21 22:48, Henning Schild wrote:
> Did not look into any of the patches so far ... but the number 24 makes
> it close to impossible to review. Can this q be split up to ease review?

I don't think 24 is a show-stopper per se. The preparatory changes to
recipes are split along the recipe types, and that adds a certain
length. In that sense, the series is already split, but I also do not
see that those preparatory patches need to come separately.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 04/24] linux-module: Do not use shell environment

[Uladzimir Bely]

In the email from Friday, 19 Nov 2021. 15:44:12 +03 user Jan Kiszka wrote:
> On 19.11.21 13:13, Uladzimir Bely wrote:
> 
> > +KDIR := $(shell dpkg -L $(KERNEL_DEP) | grep "/lib/modules/.*/build")
> > +endif
> > +
> > +# With sbuild `dh clean` is called before chroot's apt database updated.
> > So, +# KDIR is empty at that moment and the override causes error. Will
> > skip it. +ifneq ($(KDIR),)
> > 
> >  override_dh_auto_clean:
> >  	$(MAKE) -C $(KDIR) M=$(PWD) clean
> > 
> > +endif
> 
> This looks wrong, or at least fragile. Why can we live without a proper
> clean on the kernel tree?
> 
> Jan

Actually, this workaround with `ifneq ($(KDIR),)` I added to the patch after 
the commit https://github.com/ilbers/isar/commit/248436880eb2 was applied to 
`next`.

This change doesn't cancel a proper clean on the kernel. Here are some example 
logs:

`next`: https://gist.github.com/WiseLord/0fb98bc036532ec03e4f763b2ba06a41
`sbuild`: https://gist.github.com/WiseLord/3c35d477efc025bcc9cd3c5cbd3dbc5d

On current `next` code dh_clean (and override-related `make ... clean` on 
kernel code) is executed by dpkg-buildpackage. At this point KDIR is set to a 
proper value, so it works as expected.

With sbuild dh_clean happens to be executed twice:
- 1: immediately after sbuild is run, executed by sbuild itself
- 2: after few stages (like downloading deps and so on), executed by dpkg-
buildpackage.

The issue is that at 1st execution KDIR is not set, so `make ... clean` 
(coming with override) produces wrong command.

That's why this workaround (skip `make ... clean` in case of empty KDIR) was 
added. Proper cleaning is still performed, but on the second call of dh_clean.

Probably, there is a way to say sbuild not to run this 'generic' dh_clean and 
use just one from dpkg-buildpackage later...

-- 
Uladzimir Bely
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Uladzimir Bely]

In the email from Friday, 19 Nov 2021 г. 15:13:09 +03 user Uladzimir Bely 
wrote:
> This is a patchset showing how sbuild/schroot tools can be integrated
> into Isar build system.
>
> ...
> 
> Current limitations:
> - there is an unsolved problem with building foreigh architectures
> using kas-docker.
> 
> - qemuamd64 images are buildable in kas-docker, but some changes
> in kas are reqired (like installing additional packages and adding
> 'builder' user to 'sbuild group). I used the following changes to
> build ghcr.io/siemens/kas/kas-isar:2.6.2-sbuild image based on
> current ghcr.io/siemens/kas/kas-isar:2.6.2:
> https://github.com/WiseLord/kas/commit/5b7b02aa33
> 
> ...
> meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%)

Here I would like to add some info regarding running the patchset with `kas-
docker` and/or using Gitlab CI.

Initially, the patchset was checked in Jenkins CI (that doesn't use docker) 
and sent to the maillist then. But there are some docker-related issues solved 
a bit later. So here I'll try to explain them (and this is to be added to user 
docs in the following patchset version), so someone could play with sbuild 
patchset.

1. Additional packages required.

The patchset requires 'schroot' and 'sbuild' packages to be installed on the 
host. Also, they both should have at least "bullseye" version, otherwise some 
things won't work properly (like caching .deb via 'sbuild' or foreign 
architectures support in 'schroot').

While current (ghcr.io/siemens/kas/kas-isar:2.6.2) image doesn't include the 
required packages, I temporary prepared and upuloaded one based on it 
(ghcr.io/wiselord/kas-isar:2.6.2-sbuild). Look at Dockerfile.isar.sbuild chunk 
in the patch https://github.com/WiseLord/kas/commit/38f4f11f11 for details.

Gitlab

Gitlab uses .gitlab-ci.yml for setup. So, to use modified image in Gitlab CI 
user should simply change the first line:

> - image: ghcr.io/siemens/kas/kas-isar:latest
> + image: ghcr.io/wiselord/kas-isar:2.6.2-sbuild

This is required until official kas-isar image have everything included.

2. User should be added to 'sbuild' group.

In case of 'kas-docker' it happend to be a bit tricky, because 'builder' user 
is created 'on the fly', when container is run. I'm not an expert in 'kas' so 
I simply pathced container-entrypoint to add user to sbuild group. Look at
`container.entrypoint` chunk in the patch for details.

3. Overlayfs restrictions.

Docker uses overlayfs to mount dockerimage rootfs. Schroot uses /var/lib/
schroot/union/{overlay,underlay} directories to keep 'basic' image and 
temporary layers on top of it.

So we happen to have 'overlayfs over overlayfs' situation that is not 
supported by overlayfs kernel driver.

The solutionis to use an external volume for the /var/lib/schroot/union/. Look 
at 'kas-container' chunk in the patch for details.

Gitlab

It seems there is no way to say gitlab using external volume via the 
configuration file `.gitlab-ci.yml`. But it can be done by the following 
changes in /etc/gitlab/runner/config.toml:

> -volumes = ["/cache"]
> +volumes = ["/m/ws-10/schroot-10a/union:/var/lib/schroot/union", "/cache"]

So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on host. 
Also, there should be empty "overlay" and "underlay" directories created in 
it.

-- 
Uladzimir Bely
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 04/24] linux-module: Do not use shell environment

[Jan Kiszka]

On 23.11.21 13:24, Uladzimir Bely wrote:
> In the email from Friday, 19 Nov 2021. 15:44:12 +03 user Jan Kiszka wrote:
>> On 19.11.21 13:13, Uladzimir Bely wrote:
>>
>>> +KDIR := $(shell dpkg -L $(KERNEL_DEP) | grep "/lib/modules/.*/build")
>>> +endif
>>> +
>>> +# With sbuild `dh clean` is called before chroot's apt database updated.
>>> So, +# KDIR is empty at that moment and the override causes error. Will
>>> skip it. +ifneq ($(KDIR),)
>>>
>>>  override_dh_auto_clean:
>>>  	$(MAKE) -C $(KDIR) M=$(PWD) clean
>>>
>>> +endif
>>
>> This looks wrong, or at least fragile. Why can we live without a proper
>> clean on the kernel tree?
>>
>> Jan
> 
> Actually, this workaround with `ifneq ($(KDIR),)` I added to the patch after 
> the commit https://github.com/ilbers/isar/commit/248436880eb2 was applied to 
> `next`.
> 
> This change doesn't cancel a proper clean on the kernel. Here are some example 
> logs:
> 
> `next`: https://gist.github.com/WiseLord/0fb98bc036532ec03e4f763b2ba06a41
> `sbuild`: https://gist.github.com/WiseLord/3c35d477efc025bcc9cd3c5cbd3dbc5d
> 
> On current `next` code dh_clean (and override-related `make ... clean` on 
> kernel code) is executed by dpkg-buildpackage. At this point KDIR is set to a 
> proper value, so it works as expected.
> 
> With sbuild dh_clean happens to be executed twice:
> - 1: immediately after sbuild is run, executed by sbuild itself
> - 2: after few stages (like downloading deps and so on), executed by dpkg-
> buildpackage.
> 
> The issue is that at 1st execution KDIR is not set, so `make ... clean` 
> (coming with override) produces wrong command.
> 
> That's why this workaround (skip `make ... clean` in case of empty KDIR) was 
> added. Proper cleaning is still performed, but on the second call of dh_clean.
> 
> Probably, there is a way to say sbuild not to run this 'generic' dh_clean and 
> use just one from dpkg-buildpackage later...
> 

Thanks for the explanation. How about clarifying that no clean is lost
in the commit message?

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 04/24] linux-module: Do not use shell environment

[Uladzimir Bely]

In mail from Friday, 19 Nov 2021 г. 15:44:12 +03 user Jan Kiszka wrote:
> On 19.11.21 13:13, Uladzimir Bely wrote:
> > +
> > +    # restore execute permissions
> > +    chmod a+x ${S}/debian/rules
> 
> The mode should be fixed (aligned to the template) in template.bbclass
> to avoid bothering every recipe with such boilerplate code. I'm seeing
> this in other changes as well.
> 
> With all those "do not use shell env" - how will a downstream user
> detect that a recipe is not properly converted? That looks like a nice
> trap during migration to sbuild (because of silent / non-obvious errors).
> 
> Jan

Regarding restoring permissions: this case can be covered by `[PATCH] 
template: Make templates passthrough` (https://groups.google.com/u/0/g/isar-users/c/Pmr2jUjviz8) which automatically keeps debian/<file>.tmpl permissions 
when transforming it to debian/<file>. Internally, I've included this patch 
and removed all these parts with `chmod` from the recipes and it looks working 
(anyway, CI tests are still needed).

And you are right, there might be some way to warn users about non-converted 
downstream recipes. It is worth thinking.

-- 
Uladzimir Bely
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Jan Kiszka]

On 23.11.21 14:05, Uladzimir Bely wrote:
> In the email from Friday, 19 Nov 2021 г. 15:13:09 +03 user Uladzimir Bely 
> wrote:
>> This is a patchset showing how sbuild/schroot tools can be integrated
>> into Isar build system.
>>
>> ...
>>
>> Current limitations:
>> - there is an unsolved problem with building foreigh architectures
>> using kas-docker.

What exactly is the problem here?

>>
>> - qemuamd64 images are buildable in kas-docker, but some changes
>> in kas are reqired (like installing additional packages and adding
>> 'builder' user to 'sbuild group). I used the following changes to
>> build ghcr.io/siemens/kas/kas-isar:2.6.2-sbuild image based on
>> current ghcr.io/siemens/kas/kas-isar:2.6.2:
>> https://github.com/WiseLord/kas/commit/5b7b02aa33
>>
>> ...
>> meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%)
> 
> Here I would like to add some info regarding running the patchset with `kas-
> docker` and/or using Gitlab CI.
> 
> Initially, the patchset was checked in Jenkins CI (that doesn't use docker) 
> and sent to the maillist then. But there are some docker-related issues solved 
> a bit later. So here I'll try to explain them (and this is to be added to user 
> docs in the following patchset version), so someone could play with sbuild 
> patchset.
> 
> 1. Additional packages required.
> 
> The patchset requires 'schroot' and 'sbuild' packages to be installed on the 
> host. Also, they both should have at least "bullseye" version, otherwise some 
> things won't work properly (like caching .deb via 'sbuild' or foreign 
> architectures support in 'schroot').
> 
> While current (ghcr.io/siemens/kas/kas-isar:2.6.2) image doesn't include the 
> required packages, I temporary prepared and upuloaded one based on it 
> (ghcr.io/wiselord/kas-isar:2.6.2-sbuild). Look at Dockerfile.isar.sbuild chunk 
> in the patch https://github.com/WiseLord/kas/commit/38f4f11f11 for details.
> 
> Gitlab
> 
> Gitlab uses .gitlab-ci.yml for setup. So, to use modified image in Gitlab CI 
> user should simply change the first line:
> 
>> - image: ghcr.io/siemens/kas/kas-isar:latest
>> + image: ghcr.io/wiselord/kas-isar:2.6.2-sbuild
> 
> This is required until official kas-isar image have everything included.
> 
> 2. User should be added to 'sbuild' group.
> 
> In case of 'kas-docker' it happend to be a bit tricky, because 'builder' user 
> is created 'on the fly', when container is run. I'm not an expert in 'kas' so 
> I simply pathced container-entrypoint to add user to sbuild group. Look at
> `container.entrypoint` chunk in the patch for details.
> 
> 3. Overlayfs restrictions.
> 
> Docker uses overlayfs to mount dockerimage rootfs. Schroot uses /var/lib/
> schroot/union/{overlay,underlay} directories to keep 'basic' image and 
> temporary layers on top of it.
> 
> So we happen to have 'overlayfs over overlayfs' situation that is not 
> supported by overlayfs kernel driver.
> 
> The solutionis to use an external volume for the /var/lib/schroot/union/. Look 
> at 'kas-container' chunk in the patch for details.
> 

I've commented on some aspects of you kas patch. It generally looks
feasible, just some smaller details should be improved.

> Gitlab
> 
> It seems there is no way to say gitlab using external volume via the 
> configuration file `.gitlab-ci.yml`. But it can be done by the following 
> changes in /etc/gitlab/runner/config.toml:
> 
>> -volumes = ["/cache"]
>> +volumes = ["/m/ws-10/schroot-10a/union:/var/lib/schroot/union", "/cache"]
> 
> So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on host. 
> Also, there should be empty "overlay" and "underlay" directories created in 
> it.

That's also not unrealistic, given that we already need to provide
special runners for the purpose of granting privileges and allowing
binfmt_misc. One target would be
https://gitlab.com/cip-project/cip-testing/gitlab-cloud-ci, the backend
for many of our gitlab CI runners that are Isar-compatible.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Uladzimir Bely]

In mail from Friday, 26 Nov 2021. 09:43:58 +03 user Jan Kiszka wrote:
> On 23.11.21 14:05, Uladzimir Bely wrote:
> >> Current limitations:
> >> - there is an unsolved problem with building foreigh architectures
> >> using kas-docker.
> 
> What exactly is the problem here?
> 

Actually, the problem was related to UsrMerge. Our Jenkins root is also based 
on buster, but /bin, /lib are symlinks to /usr/bin, /usr/lib and so on. While 
in buster-slim used in kas-docker /bin, /lib are separate directories. And 
schroot version used in buster couldn't work in this case (qemu-*-static 
packages couldn't find some libraries).

But when I schroot version from bullseye seems to support both kinds of 
rootfs. So, no changes were needed in sbuild patches, I just preinstalled 
newer schroot (from bullseye) in kas-docker image (in addition to a newer 
sbuild), and it works.

> >> ...
> 
> I've commented on some aspects of you kas patch. It generally looks
> feasible, just some smaller details should be improved.
> 

Yes, I saw your comments on github. I'm not yet experienced in kas 
development, so my implementation is not ideal.

> > ...
> > So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on
> > host. Also, there should be empty "overlay" and "underlay" directories
> > created in it.
> 
> That's also not unrealistic, given that we already need to provide
> special runners for the purpose of granting privileges and allowing
> binfmt_misc. One target would be
> https://gitlab.com/cip-project/cip-testing/gitlab-cloud-ci, the backend
> for many of our gitlab CI runners that are Isar-compatible.
> 
> Jan

Ok, I'll take a look at it.

Currently I'm trying to find/fix possible issues in downstreams. I took meta-
iot2050 for experiments. For now it seems to use quite old docker image and 
isar codebase. At first, I made sure it can build with no errors current Isar 
'next' branch with using my modified docker image. But for now it is not yet 
able to build even partial (preparation patches only) sbuild patchset and I'm 
trying to understand and fix the issues.

-- 
Uladzimir Bely
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Jan Kiszka]

On 26.11.21 09:03, Uladzimir Bely wrote:
> In mail from Friday, 26 Nov 2021. 09:43:58 +03 user Jan Kiszka wrote:
>> On 23.11.21 14:05, Uladzimir Bely wrote:
>>>> Current limitations:
>>>> - there is an unsolved problem with building foreigh architectures
>>>> using kas-docker.
>>
>> What exactly is the problem here?
>>
> 
> Actually, the problem was related to UsrMerge. Our Jenkins root is also based 
> on buster, but /bin, /lib are symlinks to /usr/bin, /usr/lib and so on. While 
> in buster-slim used in kas-docker /bin, /lib are separate directories. And 
> schroot version used in buster couldn't work in this case (qemu-*-static 
> packages couldn't find some libraries).
> 
> But when I schroot version from bullseye seems to support both kinds of 
> rootfs. So, no changes were needed in sbuild patches, I just preinstalled 
> newer schroot (from bullseye) in kas-docker image (in addition to a newer 
> sbuild), and it works.

Ah, ok. Then we will be automatically fine when moving the complete kas
image to bullseye.

> 
>>>> ...
>>
>> I've commented on some aspects of you kas patch. It generally looks
>> feasible, just some smaller details should be improved.
>>
> 
> Yes, I saw your comments on github. I'm not yet experienced in kas 
> development, so my implementation is not ideal.
> 

No problem, that can be sorted out. If you have questions, just drop
them on kas mailing list.

>>> ...
>>> So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on
>>> host. Also, there should be empty "overlay" and "underlay" directories
>>> created in it.
>>
>> That's also not unrealistic, given that we already need to provide
>> special runners for the purpose of granting privileges and allowing
>> binfmt_misc. One target would be
>> https://gitlab.com/cip-project/cip-testing/gitlab-cloud-ci, the backend
>> for many of our gitlab CI runners that are Isar-compatible.
>>
>> Jan
> 
> Ok, I'll take a look at it.
> 
> Currently I'm trying to find/fix possible issues in downstreams. I took meta-
> iot2050 for experiments. For now it seems to use quite old docker image and 
> isar codebase. At first, I made sure it can build with no errors current Isar 
> 'next' branch with using my modified docker image. But for now it is not yet 
> able to build even partial (preparation patches only) sbuild patchset and I'm 
> trying to understand and fix the issues.
> 

Yeah, meta-io2050 dependencies were frozen for an upcoming release. Try
bumping them first and building those with unpatched Isar and unpatched
kas 2.6.2 (that should work, see
https://github.com/siemens/meta-iot2050/pull/205), then modify kas and
finally patch isar. If there are issues already in the first step, let
me know.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Michael Adler]

Hi Uladzimir,

> It seems there is no way to say gitlab using external volume via the configuration file `.gitlab-ci.yml`

yes, because it is not universally applicable: there are various runner implementations [1] and the "volumes" feature is
not supported by every runner (e.g. SSH-based).

For CIP, we use custom Kubernetes runners. Although it is possible to setup custom volumes [2], that's only half of the
story: the other half is to ensure that these directories actually exist on the host, which in the case of
gitlab-cloud-ci [3] (used by CIP and Siemens) are ephemeral, i.e. they are dynamically created and destroyed based on
on the CI workload. It should be possible to support such volumes but it's definitely extra engineering/maintenance
effort.

> So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on host.

What happens if I choose not to provide such a volume? Is it mandatory for the build? Do I also need this for local
containerized builds?

Kind regards,
Michael

[1] https://docs.gitlab.com/runner/executors/
[2] https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes
[3] https://gitlab.com/cip-project/cip-testing/gitlab-cloud-ci

-- 
Michael Adler

Siemens AG
T RDA IOT SES-DE
Otto-Hahn-Ring 6
81739 M�nchen, Deutschland

Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Jim Hagemann Snabe; Vorstand: Roland Busch, Vorsitzender; Klaus Helmrich, Cedrik Neike, Matthias Rebellius, Ralf P. Thomas, Judith Wiese; Sitz der Gesellschaft: Berlin und M�nchen, Deutschland; Registergericht: Berlin-Charlottenburg, HRB 12300, M�nchen, HRB 6684; WEEE-Reg.-Nr. DE 23691322

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Uladzimir Bely]

In mail from Friday, 26 Nov 2021. 15:09:21 +03 user Michael Adler wrote:
> Hi Uladzimir,
> 
> > It seems there is no way to say gitlab using external volume via the
> > configuration file `.gitlab-ci.yml`
> yes, because it is not universally applicable: there are various runner
> implementations [1] and the "volumes" feature is not supported by every
> runner (e.g. SSH-based).
> 
> For CIP, we use custom Kubernetes runners. Although it is possible to setup
> custom volumes [2], that's only half of the story: the other half is to
> ensure that these directories actually exist on the host, which in the case
> of gitlab-cloud-ci [3] (used by CIP and Siemens) are ephemeral, i.e. they
> are dynamically created and destroyed based on on the CI workload. It
> should be possible to support such volumes but it's definitely extra
> engineering/maintenance effort.
> 
> > So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on
> > host.
> What happens if I choose not to provide such a volume? Is it mandatory for
> the build? Do I also need this for local containerized builds?
>

As I wrote, the problem that docker uses overlayfs to produce a working 
chroot. So, /var/lib/schroot/union/ happens to be on overlayfs. Sbuild also 
uses overlayfs - it mounts rootfs to /var/lib/schroot/union/underlay (as a 
lower, read-only layer) and uses /var/lib/schroot/union/overlay/ handle rw 
layer.
But, it is not supported by overlayfs (owerdir=/lower,upperdir=/
upper,workdir=/work) when upperdir is on overlayfs (from docker).

> 
> Kind regards,
> Michael
> 
> [1] https://docs.gitlab.com/runner/executors/
> [2] https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes
> [3] https://gitlab.com/cip-project/cip-testing/gitlab-cloud-ci


-- 
Uladzimir Bely
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Jan Kiszka]

On 26.11.21 13:09, Michael Adler wrote:
> Hi Uladzimir,
> 
>> It seems there is no way to say gitlab using external volume via the configuration file `.gitlab-ci.yml`
> 
> yes, because it is not universally applicable: there are various runner implementations [1] and the "volumes" feature is
> not supported by every runner (e.g. SSH-based).
> 
> For CIP, we use custom Kubernetes runners. Although it is possible to setup custom volumes [2], that's only half of the
> story: the other half is to ensure that these directories actually exist on the host, which in the case of
> gitlab-cloud-ci [3] (used by CIP and Siemens) are ephemeral, i.e. they are dynamically created and destroyed based on
> on the CI workload. It should be possible to support such volumes but it's definitely extra engineering/maintenance
> effort.

We would not need those union mounts to be persist across jobs - just
like the rest the container generated during job execution and that is
not explicitly exported otherwise (artifacts, local cache,
self-uploads). We just need that union to be on something else than an
overlayfs.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [PATCH v2 00/24] Sbuild/Schroot migration

[Jan Kiszka]

On 26.11.21 09:50, Jan Kiszka wrote:
> On 26.11.21 09:03, Uladzimir Bely wrote:
>> In mail from Friday, 26 Nov 2021. 09:43:58 +03 user Jan Kiszka wrote:
>>> On 23.11.21 14:05, Uladzimir Bely wrote:
>>>>> Current limitations:
>>>>> - there is an unsolved problem with building foreigh architectures
>>>>> using kas-docker.
>>>
>>> What exactly is the problem here?
>>>
>>
>> Actually, the problem was related to UsrMerge. Our Jenkins root is also based 
>> on buster, but /bin, /lib are symlinks to /usr/bin, /usr/lib and so on. While 
>> in buster-slim used in kas-docker /bin, /lib are separate directories. And 
>> schroot version used in buster couldn't work in this case (qemu-*-static 
>> packages couldn't find some libraries).
>>
>> But when I schroot version from bullseye seems to support both kinds of 
>> rootfs. So, no changes were needed in sbuild patches, I just preinstalled 
>> newer schroot (from bullseye) in kas-docker image (in addition to a newer 
>> sbuild), and it works.
> 
> Ah, ok. Then we will be automatically fine when moving the complete kas
> image to bullseye.
> 

FYI: kas 3.x development has started, 'next' is now on bullseye.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux