From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057122428766388224 X-Received: by 2002:a05:6000:1a50:: with SMTP id t16mr21986595wry.571.1643734844842; Tue, 01 Feb 2022 09:00:44 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:694d:: with SMTP id r13ls221835wrw.3.gmail; Tue, 01 Feb 2022 09:00:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJzUoBWRryV9rF/ZDtQu2yNFAH0b6x4qSDjYtwL6xUHO2P4YPcM8DvJerCSdHMswO3lYjICj X-Received: by 2002:a05:6000:168f:: with SMTP id y15mr23176303wrd.365.1643734843868; Tue, 01 Feb 2022 09:00:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643734843; cv=none; d=google.com; s=arc-20160816; b=Ug5ZjtX5RuaCiqkn3hHvfGSkNJeyU2Lg065KYgGJhSqNuSdx9LvIr2MQcTmsbycwS0 vkKFor/KRao5CeDm2QfcvLkuGnZcvSzsN2VZ4iNpbVwR+rycwIyDY37r5GPxbkm0Gjbs HQjLVZaVLvuFWt81hmOOmkZqYLGYsOE9oo+w0JOFR1DteyW0gAS1eA5Vh/Eq5oRyLav7 71WALOO881kVOJhn8Jm8mgTeATOZkq/m22/byjz9zrbH/X3X+rAb1kLH9067MpnPxw9T Qy2Y8I2D2tpAuOZxJ8LSgasiYgV8vCd9MAscYglgIzentLYddjs9Tar6PjmtsK7krw5c v6ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=cNbLFYTTZrBlWrSw6IqKH28Ae0epv76RSv/ebi6rWN4=; b=yOfX4xEzg26oJzeWp94N2Wc9SsK0+jkiZU5tH5OBniuh5DTn7bZ/An1xKQlaFhM+oi vgd2E1JnhKHJ3Rsh6FTqgLeFL+wE/I5+pP4EZFT6CQ9M2msp2DkzZvh9dFIE86Cd2cpx P+KOH8kaVvzYvasF31YBessLEkEHRso6U/xkOV5X1NDC0BLNjNydTUkQwDNQmS/eT5Bz fdOa9MrNF6GUZXuIN3QwA5MEnTC/8bvWv6PQS2/MzONGEtTsdMrvY++NDCvrLFDP7GYL rVJ15yIOb8PPRj4y6GyWr3rLHUyHJheQXdSRMa4yxKUJwrJRcoBJP+Ep1G/rS35vghHs Gpcg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id h2si1394102wrp.7.2022.02.01.09.00.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 01 Feb 2022 09:00:43 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 211H0cA4019722 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 1 Feb 2022 18:00:39 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v5 00/12] Sbuild/Schroot migration Date: Tue, 1 Feb 2022 18:00:26 +0100 Message-Id: <20220201170038.5723-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: /zhWJBfheqrW This is a patchset showing how sbuild/schroot tools can be integrated into Isar build system. This patchset should be applied on top of "preparation" patchset named "Avoid using shell environment during the build". Base schroot image is created with sbuild-chroot-target (or sbuild-chroot-host in case of cross-build) recipe. These images are based on isar-bootstrap ones, but include some build-related stuff preinstalled. To use this changes you need to have sbuild and schroot installed and configured. Current user should be added to sbuild group. If 'kas-container' is used, it should be modified to support sbuild: - /var/lib/schroot/union/overlay should be externally mounted (-v option) Currently, sbuild-related stuff is supported in kas-isar:next image To use the patchset with KAS: - export KAS_IMAGE_VERSION=next If 'gitlab' is used, the similar changes are required: - `next` tag for kas image should be set - external dir for schroot overlay should also be specified in /etc/gitlab-runner/config.toml: volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"] The patchset was tested with 'meta-iot2050' and 'xenomai-images' downstreams in combination with kas image mentioned above. Current limitations: - parallel building of several packages requires more free space in comparison with buildchroot-based architecture. This happens due the sbuild architecture which uses some basic layer (common for all packages) and per-package separate layer (where builddeps are downloaded and installed). Changes since v4: - consider shell exports done in dpkg_runbuild_prepend and pass them to sbuild environment; - fixed ccache work. Changes since v3: - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for downstreams compatibility; - dpkg_build_export is used for adjusting sbuild environment; - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build. Changes since v2: - patches reworked/squashed for easier reading and understanding; - fixed building foreigh architectures with kas-docker; - implemented support of ccache; - fixed devshell and devshell_nodeps, Changes since v1: - parallel builds with different BUILD_DIR are supported; - parallel multiconfig targets in one build are supported; - per-task schroot configuration in /etc/schroot/ is now used; - patchset now passes Jenkins CI (so patches changes RFC => PATCH). Anton Mikanovich (1): dpkg: Build packages with sbuild Uladzimir Bely (11): dpkg-gbp: Use separate command to export tarball dpkg-gbp: Use host tools for dsc preparation sbuild: Add recipes for host and target rootfs to run sbuild sbuild: Introduce a class for another build method sbuild: support of DEB_BUILD_PROFILES sbuild: support of shell exports from dpkg_runbuild_prepend dpkg: Remove builddeps install task. sbuild: add ccache support dpkg-base: Switch devshell to use schroot dpkg-base: Switch apt_fetch and apt_unpack to use schroot doc: Add sbuild-related documentation doc/user_manual.md | 22 ++- meta/classes/dpkg-base.bbclass | 79 ++++---- meta/classes/dpkg-gbp.bbclass | 26 +-- meta/classes/dpkg.bbclass | 80 +++++--- meta/classes/sbuild.bbclass | 184 ++++++++++++++++++ meta/conf/bitbake.conf | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ .../sbuild-chroot/sbuild-chroot-target.bb | 10 + .../sbuild-chroot/sbuild-chroot.inc | 39 ++++ 9 files changed, 375 insertions(+), 80 deletions(-) create mode 100644 meta/classes/sbuild.bbclass create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc -- 2.20.1