From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7032253102499561472 X-Received: by 2002:a05:6402:27d3:: with SMTP id c19mr24272500ede.390.1637324016857; Fri, 19 Nov 2021 04:13:36 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:907:6da0:: with SMTP id sb32ls1275068ejc.8.gmail; Fri, 19 Nov 2021 04:13:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJze/14a4tVfjWON3+XTT1AXxh2d2rn6GVjWeBXo8qBHRf43aJUFENP3zW21Lw7JTFB0uOOQ X-Received: by 2002:a17:906:1613:: with SMTP id m19mr7419630ejd.136.1637324014541; Fri, 19 Nov 2021 04:13:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637324014; cv=none; d=google.com; s=arc-20160816; b=NMto1ZAiz97+7FBpcBjeC+I6g5x/7Pnra35PwkVDPAWFMSVrAeDtS5GheXm6sOhTZU kFENVD4adafvb1UgekWafyoCZceeonqc2OzPFyeOJSrMdCeWZCs3JuT91DOw5zdsSFdn eb70Hxoyozf5NLqs4weS82mVz3RVg0TRRN3d0kvBqwfNlzC30vUCNzKIhComVI/pBh/5 tRQRgtuYsJnlP8xTW60cK/+utxJnS1Il/bm3xttTTInGv65LIDzIlFVSaTxDs8rwVmh3 VznKmSp+rKbmhMsh8k8jZsl/maGpVCykdedYni5LNhlUGgjItIAHlsa85cE8ClQPancs QfYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=lupK+wcM8XFyl1yutpUACWHp+lpt+aIDKUYlRNqYpe8=; b=W8Qoe0r4VnJhb5DmexGogiOlMntw7QdOT903H1+w6vKxrxwNATjeMRRP8G9WbawlSz a5hTAFKlzc+QLQg05KOWGaA3O9djZZZyLMOzxRQuwM5YncVfOIb2JU6qGjXeMbXHZw1Q N1PSopcQ8DVlQYT5vRfE74bZXGvzmBDzWZB12JPg14zuh7ra/BU7SGevhLkU3pxM9jOq C++jSovatfCh9bRCEU1amE0pAyUqfHlNV63tHvWZzyZPmeXK5k0i88UOPujz2f3wWwB1 bDOtR9Jl/x6LywzeBATbZgf5vdLiUG8RXHz3F902whxWNn3Iz4bLLQf383o94ePLLpWy qnLg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id w5si127479ede.3.2021.11.19.04.13.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 19 Nov 2021 04:13:34 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 1AJCDX7A021829 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 19 Nov 2021 13:13:33 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v2 00/24] Sbuild/Schroot migration Date: Fri, 19 Nov 2021 13:13:09 +0100 Message-Id: <20211119121333.13805-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: vRmHmFIqstMQ This is a patchset showing how sbuild/schroot tools can be integrated into Isar build system. Base schroot image is created with sbuild-chroot-target (or sbuild-chroot-host in case of cross-build) recipe. These images are based on isar-bootstrap ones, but include some build-related stuff preinstalled. To use this changes you need to have sbuild and schroot installed and configured. Current user should be added to sbuild group. Changes since v1: - parallel builds with different BUILD_DIR are supported - parallel multiconfig targets in one build are supported - per-task schroot configuration in /etc/schroot/ is now used - patchset now passes Jenkins CI (so patches changes RFC => PATCH) Current limitations: - there is an unsolved problem with building foreigh architectures using kas-docker. - qemuamd64 images are buildable in kas-docker, but some changes in kas are reqired (like installing additional packages and adding 'builder' user to 'sbuild group). I used the following changes to build ghcr.io/siemens/kas/kas-isar:2.6.2-sbuild image based on current ghcr.io/siemens/kas/kas-isar:2.6.2: https://github.com/WiseLord/kas/commit/5b7b02aa33 - parallel building of several packages requires more free space than current buildchroot-based architecture uses. This happens due the sbuild architecture which uses some basic layer (common for all packages) and per-package separate layer (where builddeps are downloaded and installed). - recently merged to 'next' ccache support is not yet implemented in case of sbuild used. For experiments, the code can be taken from Isar 'ubely/sbuild' branch. This is an dev branch, so sometimes it can be force-pushed when some fixes are applied of rebased to 'next' branch. Anton Mikanovich (12): dpkg: Install raw package files to source root isar-bootstrap: Export bootstrap to schroot config linux-module: Do not use shell environment u-boot: Do not use shell environment trusted-firmware: Do not use shell environment optee-os: Do not use shell environment kselftest: Do not use shell environment dpkg: Build packages with sbuild sbuild: Introduce environment variables export API dpkg-gbp: Migrate to schroot linux-mainline: Move cfg fragment test to debian/rules linux-custom: Prepare kernel config inside sbuild Uladzimir Bely (12): dpkg-gbp: Use separate command to export tarball sbuild: Add recipes for host and target rootfs to run sbuild sbuild: Mount base-apt in schroot sbuild: Add sbuildshell task dpkg-gbp: Preinstall gbp utils in schroot dpkg: Remove builddeps install task. dpkg-base: Switch devshell to use schroot dpkg-base: Switch apt_fetch and apt_unpack to use schroot dpkg-base: Cleanup from buildchroot parts. dpkg-gbp: Use host tools for dsc preparation doc: Add sbuild-related documentation sbuild: Replace isar-apt mounting with copying doc/user_manual.md | 22 ++- .../optee-os/optee-os-stm32mp15x_3.11.0.bb | 2 +- .../linux/linux-mainline_5.4.70.bb | 16 +- meta/classes/dpkg-base.bbclass | 80 ++------ meta/classes/dpkg-gbp.bbclass | 25 +-- meta/classes/dpkg-raw.bbclass | 8 +- meta/classes/dpkg.bbclass | 56 +++--- meta/classes/sbuild.bbclass | 177 ++++++++++++++++++ meta/conf/bitbake.conf | 2 + .../files/debian/{rules => rules.tmpl} | 4 +- meta/recipes-bsp/optee-os/optee-os-custom.inc | 10 +- .../files/debian/{rules => rules.tmpl} | 4 +- .../trusted-firmware-a-custom.inc | 10 +- .../u-boot/files/debian/{rules => rules.tmpl} | 2 +- meta/recipes-bsp/u-boot/u-boot-custom.inc | 10 +- .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ .../sbuild-chroot/sbuild-chroot-target.bb | 10 + .../sbuild-chroot/sbuild-chroot.inc | 36 ++++ .../kselftest/files/{rules => rules.tmpl} | 0 meta/recipes-kernel/kselftest/kselftest.inc | 21 +-- .../files/debian/{rules => rules.tmpl} | 14 +- meta/recipes-kernel/linux-module/module.inc | 23 +-- meta/recipes-kernel/linux/linux-custom.inc | 7 +- 23 files changed, 382 insertions(+), 170 deletions(-) create mode 100644 meta/classes/sbuild.bbclass rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl} (75%) rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules => rules.tmpl} (77%) rename meta/recipes-bsp/u-boot/files/debian/{rules => rules.tmpl} (94%) create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc rename meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%) rename meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%) -- 2.20.1