From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057122428766388224 X-Received: by 2002:a05:6512:3f93:: with SMTP id x19mr4209778lfa.440.1644475582919; Wed, 09 Feb 2022 22:46:22 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:bf1a:: with SMTP id c26ls749651ljr.10.gmail; Wed, 09 Feb 2022 22:46:21 -0800 (PST) X-Google-Smtp-Source: ABdhPJw7iyqHpe1MTUV86XDlIrQXT/bv0UihH/PZFnBn3KkBi6mqnBg7YwrGVovFI+8Zq1GaHjrK X-Received: by 2002:a05:651c:1542:: with SMTP id y2mr3934557ljp.334.1644475581711; Wed, 09 Feb 2022 22:46:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1644475581; cv=none; d=google.com; s=arc-20160816; b=Zb9ksvS5o37HHED9ikoSR26LZ5pkbZnLYfz/jWC8E4ynGDmT2SjZbsoajdq9PiCBj/ RZlHiFyr78qRfO5e/3hE7Tv4FWhGM/FXfgnnLXeORNGLAfFXSUhq0YGJXaWljz5X8a7G BvMBk4EjjsKouw9GMkqy1dMAt3Og7Q5t/TazQan4Ia55RuJr6jl0OAtvB6rYF4kMmGzz UusicVTyT05HLHghttFUUFb5/Hr6VKjChpiQOI/GCWfFsTHoEbVTtZWJqklT/T7aX/IQ L0+UuQzbKKHZc0/JkRK12wU2m+aRE0oJgHZJfX/HUk+AGaSjemqOTHr4xJFKY8Kkga8O 1eEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=PKo478NTPDrS8xlwmEDDyyC+MKPuDIdqpB9FeaLrt/Y=; b=MPRlO347UwJ2MFP7HR2A49pRC6Ev9yAIj2vtTsLqLkie+OjIvyUNIX5gJ6C+EEqmZO ie43qNgR6l5DPXDHrIMcjfedM637UWSA1f+imMTqaBkWdSLxXD7Gx/i90SZfho++G2z1 ZQyGHY0P0JrIEzzFdpR5PY3pRi6XBw0TUkWi4kJWzmv2XBETCDuw2+aZBFGCOZoONb2J BZfl6jxCTuUcqWGshmp6fkqmjxFkQLA4dWjtyu1Ito+jJ2JpwvvcwXf/kZzRrSz673D/ ecHjPi3QmHwcC9qQ259tvjon4V5/QBbPWlBgG7Zj0obsF5M4OBfbFIOxVW34HePSBlJr rqKg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id v26si125209lfo.10.2022.02.09.22.46.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 09 Feb 2022 22:46:21 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 21A6kKr2028169 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 10 Feb 2022 07:46:20 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v6 00/12] Sbuild/Schroot migration Date: Thu, 10 Feb 2022 07:46:08 +0100 Message-Id: <20220210064620.4392-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: o8/rNQfhU51f This is a patchset showing how sbuild/schroot tools can be integrated into Isar build system. This patchset should be applied on top of "preparation" patchset named "Avoid using shell environment during the build". Base schroot image is created with sbuild-chroot-target (or sbuild-chroot-host in case of cross-build) recipe. These images are based on isar-bootstrap ones, but include some build-related stuff preinstalled. To use this changes you need to have sbuild and schroot installed and configured. Current user should be added to sbuild group. If 'kas-container' is used, it should be modified to support sbuild: - /var/lib/schroot/union/overlay should be externally mounted (-v option) Currently, sbuild-related stuff is supported in kas-isar:next image To use the patchset with KAS: - export KAS_IMAGE_VERSION=next If 'gitlab' is used, the similar changes are required: - `next` tag for kas image should be set - external dir for schroot overlay should also be specified in /etc/gitlab-runner/config.toml: volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"] The patchset was tested with 'meta-iot2050' and 'xenomai-images' downstreams in combination with kas image mentioned above. Current limitations: - parallel building of several packages requires more free space in comparison with buildchroot-based architecture. This happens due the sbuild architecture which uses some basic layer (common for all packages) and per-package separate layer (where builddeps are downloaded and installed). Changes since v5: - warning about shell exports in dpkg_runbuild_prepend. This should avoided be reworked to templates. Changes since v4: - consider shell exports done in dpkg_runbuild_prepend and pass them to sbuild environment; - fixed ccache work. Changes since v3: - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for downstreams compatibility; - dpkg_build_export is used for adjusting sbuild environment; - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build. Changes since v2: - patches reworked/squashed for easier reading and understanding; - fixed building foreigh architectures with kas-docker; - implemented support of ccache; - fixed devshell and devshell_nodeps, Changes since v1: - parallel builds with different BUILD_DIR are supported; - parallel multiconfig targets in one build are supported; - per-task schroot configuration in /etc/schroot/ is now used; - patchset now passes Jenkins CI (so patches changes RFC => PATCH). Anton Mikanovich (1): dpkg: Build packages with sbuild Uladzimir Bely (11): dpkg-gbp: Use separate command to export tarball dpkg-gbp: Use host tools for dsc preparation sbuild: Add recipes for host and target rootfs to run sbuild sbuild: Introduce a class for another build method sbuild: support of DEB_BUILD_PROFILES sbuild: support of shell exports from dpkg_runbuild_prepend dpkg: Remove builddeps install task. sbuild: add ccache support dpkg-base: Switch devshell to use schroot dpkg-base: Switch apt_fetch and apt_unpack to use schroot doc: Add sbuild-related documentation doc/user_manual.md | 22 ++- meta/classes/dpkg-base.bbclass | 79 ++++---- meta/classes/dpkg-gbp.bbclass | 26 +-- meta/classes/dpkg.bbclass | 88 ++++++--- meta/classes/sbuild.bbclass | 184 ++++++++++++++++++ meta/conf/bitbake.conf | 2 + .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ .../sbuild-chroot/sbuild-chroot-target.bb | 10 + .../sbuild-chroot/sbuild-chroot.inc | 39 ++++ 9 files changed, 382 insertions(+), 81 deletions(-) create mode 100644 meta/classes/sbuild.bbclass create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc -- 2.20.1