From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057122428766388224 X-Received: by 2002:a5d:47c2:: with SMTP id o2mr4089001wrc.460.1643114357691; Tue, 25 Jan 2022 04:39:17 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:f04f:: with SMTP id t15ls60809wro.1.gmail; Tue, 25 Jan 2022 04:39:16 -0800 (PST) X-Google-Smtp-Source: ABdhPJxe01gKEylEqaPi9ZdqhFjbEvjxwUPqG3cHt1sU6cfIj4V7/ZsM4tg0jiycMk8AEjN71RnW X-Received: by 2002:a5d:47c2:: with SMTP id o2mr4088951wrc.460.1643114356754; Tue, 25 Jan 2022 04:39:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643114356; cv=none; d=google.com; s=arc-20160816; b=QhBaZDE1jgYISEFltksR1Vr29xwjeEFpHs14KBSMGLB9GO0nJbz5ImQmuLPvMTQ0qT l3TsC27UQRW3sSFVz0o+87Ul52clqgDulIme6o3jfDRzwtYDcDjDbUVpR+0CWpXAKQFr yfUd9nWH7WA5GkQT18rM7JYuJMaXb6+8SweJf4yrlAGuTNHvNJXFoTuakBucc5qCkPiD 2Ru+xP6njDHf7TVi3Jt3HETQ5W/RTfWHvfNgMMKZ3I6NOS9D4SUB2B8q97+rfVcvSIM3 W1cYCpP4DDE4p6WMZYqivDOAQJ/3eH+fXbKLlJCrXukKoPeWvJHC5oxjBoeMMTTt6YdK Hcvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=0kZfaKmwEBFs81nbRgpLmlCRxTPfkYuCb9RpCwHM8f8=; b=OGqIQE8D2w3gZEo4hdERX71Eoo8qrAc3IRj2KJv8mHEObXiYbfnOKYVF2KjbuXgQ+G MjHsUuxe8GmmltNqWkdxMmwthoK+Y7PsCsxD4P8hEJH1WtByOdRjSpapNQ8T6+d6ZNmG fXpZ24ziGfQ4HG3MFKJd1Uh0yrZAOXLfcVndhi+wPryy23qIQ31G49gmERbA/NaKRcSf cki4kzUTO1GlTFw/IlTwd4xGiW1V+Xm2y5VonhlM5bE2kuWuDPFI1zk4VvmRaQp0/qfq IH3T2Z1tL5ZukKyLhBknAGYem/wAUJAZHEY/5C5yQLG6bSPq1Kav0xCewXgfOgQDJ1Ms scgw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id o82si7022wme.2.2022.01.25.04.39.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 25 Jan 2022 04:39:16 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 20PCdFma019254 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 25 Jan 2022 13:39:16 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v4 00/12] Sbuild/Schroot migration Date: Tue, 25 Jan 2022 13:39:03 +0100 Message-Id: <20220125123915.1053-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: vxzmoztOLrhX This is a patchset showing how sbuild/schroot tools can be integrated into Isar build system. This patchset should be applied on top of "preparation" patchset named "Avoid using shell environment during the build". Base schroot image is created with sbuild-chroot-target (or sbuild-chroot-host in case of cross-build) recipe. These images are based on isar-bootstrap ones, but include some build-related stuff preinstalled. To use this changes you need to have sbuild and schroot installed and configured. Current user should be added to sbuild group. If 'kas-container' is used, it should be modified to support sbuild: - /var/lib/schroot/union/overlay should be externally mounted (-v option) Currently, official `kas` images don't include all required software, so there was an alternative docker image prepared: `docker pull ghcr.io/wiselord/kas/kas-isar:next-sbuild` To use it with KAS: - export KAS_IMAGE_VERSION=3.0.0-sbuild - export KAS_CONTAINER_IMAGE_PATH=ghcr.io/wiselord/kas If 'gitlab' is used, the similar changes are required: - 'image: ghcr.io/wiselord/kas/kas-isar:3.0.0-sbuild' in .gitlab-ci.yml - external dir for schroot overlay should also be specified in /etc/gitlab-runner/config.toml: volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"] The patchset was tested with 'meta-iot2050' and 'xenomai-images' downstreams. Some patches are required for these downstreams, appropriate pull requests will be created soon. Current limitations: - parallel building of several packages requires more free space in comparison with buildchroot-based architecture. This happens due the sbuild architecture which uses some basic layer (common for all packages) and per-package separate layer (where builddeps are downloaded and installed). Changes since v3: - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for downstreams compatibility; - dpkg_build_export is used for adjusting sbuild environment; - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build. Changes since v2: - patches reworked/squashed for easier reading and understanding; - fixed building foreigh architectures with kas-docker; - implemented support of ccache; - fixed devshell and devshell_nodeps, Changes since v1: - parallel builds with different BUILD_DIR are supported; - parallel multiconfig targets in one build are supported; - per-task schroot configuration in /etc/schroot/ is now used; - patchset now passes Jenkins CI (so patches changes RFC => PATCH). Anton Mikanovich (1): dpkg: Build packages with sbuild Uladzimir Bely (11): dpkg-gbp: Use separate command to export tarball dpkg-gbp: Use host tools for dsc preparation sbuild: Add recipes for host and target rootfs to run sbuild sbuild: Introduce a class for another build method sbuild: support of DEB_BUILD_PROFILES sbuild: add ccache support dpkg: Remove builddeps install task. dpkg-base: Switch devshell to use schroot dpkg-base: Switch apt_fetch and apt_unpack to use schroot dpkg-base: Cleanup from buildchroot parts. doc: Add sbuild-related documentation doc/user_manual.md | 22 ++- meta/classes/dpkg-base.bbclass | 81 ++++---- meta/classes/dpkg-gbp.bbclass | 26 +-- meta/classes/dpkg.bbclass | 62 +++--- meta/classes/sbuild.bbclass | 185 ++++++++++++++++++ meta/conf/bitbake.conf | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ .../sbuild-chroot/sbuild-chroot-target.bb | 10 + .../sbuild-chroot/sbuild-chroot.inc | 39 ++++ 9 files changed, 359 insertions(+), 81 deletions(-) create mode 100644 meta/classes/sbuild.bbclass create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc -- 2.20.1