From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057122428766388224 X-Received: by 2002:a05:6512:3184:: with SMTP id i4mr6626681lfe.275.1644738302445; Sat, 12 Feb 2022 23:45:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:43cc:: with SMTP id u12ls291858lfl.2.gmail; Sat, 12 Feb 2022 23:45:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJyIBFZkM5I1eW33Q+x04MgEglzUpSchvDKluysCUiJ5+vWxVfZY+9Po0FPEM622z1VoiVQd X-Received: by 2002:ac2:4c09:: with SMTP id t9mr6540189lfq.406.1644738301443; Sat, 12 Feb 2022 23:45:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1644738301; cv=none; d=google.com; s=arc-20160816; b=uW9lyf7Dg5RxpTe18u5NENAgekskzmaAGkIgk8aXa8jKtkGozMh/MYQ7ldBQpnX0OC R3nV78gT1pCZwCaYCFkRIHtwQx3r9+C5bMH/Xa/8qZkjS4mQ91zwKFIsP9gu4ZfBU+bU OH0VQusuBaqU/Q6n2Lf9MOg4LVFv2JDFN7rX3IBORTbFYvUlpo10u20/L5I/yToBcQx1 nNzJ73oE9VX3Tp2ta77aI6I+eNvjnvQlf9vO/of1x/zIEOUUQjg6DzhaxvO174ue4OGN v8Bgn4OnLk00jxRIx+I8VRrYAFmCwMqq2burIsgvafZ+dX+vMPYkdk1YMxoQ9De2unq+ 9mzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=k3YDCmPBK11ML3N5gAZT/YGFWSM3mdnWvVVdoHvA9+4=; b=w4d2B6dCpoSxKNCw4DFfcngyHIKIL1Lpaii0BGSpLMy4kuT16mrX7T0sjRL4m4OJzP LFPi7wh9/VKSiVBfD68x8Djhoz8bxuX+keGjnPnqhdwH5Fv1pt4sg12CHtUXG8nF3sM4 34zr09SEP6xoEehQS0fRur78lTkz9oi9DFbP8W/kaZ9eWR1jD4JWzQ/1DcoqwVM5Lwed E1KSBeLZqlfii6Kkdo17+K0JcHr/lZHZBfqNC/rOYekl277p2w/uFMTlsxQ39NHhOwYU ek9kq4E44SwO6chB6LUJ7MjB6JYTYVcIJ1PuOvF2n3HBTvCbi6edcZsGp4uX7TubG/Dq 8ILg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id d6si833674lfn.5.2022.02.12.23.45.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 12 Feb 2022 23:45:01 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 21D7j0BS010453 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 13 Feb 2022 08:45:00 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v7 00/13] Sbuild/Schroot migration Date: Sun, 13 Feb 2022 08:44:46 +0100 Message-Id: <20220213074459.24200-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: AcoWACVz2i3j This is a patchset showing how sbuild/schroot tools can be integrated into Isar build system. This patchset should be applied on top of "preparation" patchset named "Avoid using shell environment during the build". Base schroot image is created with sbuild-chroot-target (or sbuild-chroot-host in case of cross-build) recipe. These images are based on isar-bootstrap ones, but include some build-related stuff preinstalled. To use this changes you need to have sbuild and schroot installed and configured. Current user should be added to sbuild group. If 'kas-container' is used, it should be modified to support sbuild: - /var/lib/schroot/union/overlay should be externally mounted (-v option) Currently, sbuild-related stuff is supported in kas-isar:next image To use the patchset with KAS: - export KAS_IMAGE_VERSION=next If 'gitlab' is used, the similar changes are required: - `next` tag for kas image should be set - external dir for schroot overlay should also be specified in /etc/gitlab-runner/config.toml: volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"] The patchset was tested with 'meta-iot2050' and 'xenomai-images' downstreams in combination with kas image mentioned above. Current limitations: - parallel building of several packages requires more free space in comparison with buildchroot-based architecture. This happens due the sbuild architecture which uses some basic layer (common for all packages) and per-package separate layer (where builddeps are downloaded and installed). Changes since v6: - added patch that switches sbuild from using source directory to using .dsc file. The file is generated before sbuild call by `dpkg-source`. Changes since v5: - warning about shell exports in dpkg_runbuild_prepend. This should avoided be reworked to templates. Changes since v4: - consider shell exports done in dpkg_runbuild_prepend and pass them to sbuild environment; - fixed ccache work. Changes since v3: - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for downstreams compatibility; - dpkg_build_export is used for adjusting sbuild environment; - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build. Changes since v2: - patches reworked/squashed for easier reading and understanding; - fixed building foreigh architectures with kas-docker; - implemented support of ccache; - fixed devshell and devshell_nodeps. Changes since v1: - parallel builds with different BUILD_DIR are supported; - parallel multiconfig targets in one build are supported; - per-task schroot configuration in /etc/schroot/ is now used; - patchset now passes Jenkins CI (so patches changes RFC => PATCH). Anton Mikanovich (1): dpkg: Build packages with sbuild Uladzimir Bely (12): dpkg-gbp: Use separate command to export tarball dpkg-gbp: Use host tools for dsc preparation sbuild: Add recipes for host and target rootfs to run sbuild sbuild: Introduce a class for another build method sbuild: support of DEB_BUILD_PROFILES sbuild: support of shell exports from dpkg_runbuild_prepend dpkg: Remove builddeps install task. sbuild: add ccache support dpkg-base: Switch devshell to use schroot dpkg-base: Switch apt_fetch and apt_unpack to use schroot doc: Add sbuild-related documentation sbuild: Use .dsc file instead of source directory. doc/user_manual.md | 22 ++- meta/classes/dpkg-base.bbclass | 79 ++++---- meta/classes/dpkg-gbp.bbclass | 26 +-- meta/classes/dpkg.bbclass | 92 ++++++--- meta/classes/sbuild.bbclass | 184 ++++++++++++++++++ meta/conf/bitbake.conf | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ .../sbuild-chroot/sbuild-chroot-target.bb | 10 + .../sbuild-chroot/sbuild-chroot.inc | 39 ++++ 9 files changed, 386 insertions(+), 81 deletions(-) create mode 100644 meta/classes/sbuild.bbclass create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc -- 2.20.1