From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7114179385042665472 X-Received: by 2002:a5d:5602:0:b0:21b:a332:4d48 with SMTP id l2-20020a5d5602000000b0021ba3324d48mr16003571wrv.555.1656399453414; Mon, 27 Jun 2022 23:57:33 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:6dab:0:b0:212:d9db:a98 with SMTP id u11-20020a5d6dab000000b00212d9db0a98ls30260697wrs.3.gmail; Mon, 27 Jun 2022 23:57:32 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uQGdUHbvZXI8d6ZJNj94IvrzbhYpa3Nn1jCjcrnw07A2d25OT5JgWYF/ghFO17vP2ROcD0 X-Received: by 2002:adf:ffcf:0:b0:21a:3cc0:d624 with SMTP id x15-20020adfffcf000000b0021a3cc0d624mr16123188wrs.164.1656399452408; Mon, 27 Jun 2022 23:57:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656399452; cv=none; d=google.com; s=arc-20160816; b=cnORab+Av9LY5EkoJf0FBkjRNNR+eTcwJJ3tNFEtuDbs6fuwUqtZ1/Gv72NbgB8Cr6 5V3GmdxCT4mtOMiUaPsoC8Gce9msc0DiGrnUaNcXil6I8h1NB4Mfuax8DtL7z0VKaRv3 hMcNLDbomiJeJchhOdeDZWZNKqAepfNb4tVO4uUYyJpxN51GpznNnYRow5HlJvjUCR+V qnS4eExsfEo1FU4qQ6Scy0dQGUjjpi9f/u4P8DxsUmSwQhULOq4RVz0A/8u7iOYG6XxR LkKjS2+042MiqqZ/2bngx6qzQ28sWSiI/7p8t8gNHaCbOARcETcVt295xe67AAz/P1pR 9wyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id; bh=iPzHESsbW0oeN8H+zwFCjpqvrb6A/c/N/+8ljER8Qu0=; b=LU/0v+QrxluGPN+8ojYLOwdP16gcXgkk3o0KAi5RvXRgnLQHijjuuS6kVi2KQsf4FY pF/CJdjZslsslZKIShhPGaS9uq8+ALPdfm3qgs3mTehICrsw0mf/ieR0qEuBjigk9s8m ucVKL7hD323h47iB1gsxdDLmJM6szXAyW1UNlhsiIRc033hDexVCBCnWEY3cDm069laH KdwXlaV5LfHX18djl3sP3nnxW1Kl71ff92DA+V/U2TOeXDUiNtiJKwfu1+RQNNrO0Ygu ocIZK59QnQ+MHIJjnsmtdE/thnnhhD6j2dhjLd6wK9/1/I/1Gzl+H4zYF2x5NH6VWtsS iA9Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id p39-20020a05600c1da700b003a03ade6826si422445wms.0.2022.06.27.23.57.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 27 Jun 2022 23:57:32 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 25S6vTUS031250 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 28 Jun 2022 08:57:30 +0200 Message-ID: <7a97d256-669e-f8a7-699b-e6418a270225@ilbers.de> Date: Tue, 28 Jun 2022 09:57:29 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [PATCH v12 00/21] Sbuild/Schroot migration Content-Language: en-US To: isar-users References: <20220628064913.27938-1-amikan@ilbers.de> From: Anton Mikanovich In-Reply-To: <20220628064913.27938-1-amikan@ilbers.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: VMBL0/v1hEKf 28.06.2022 09:48, Anton Mikanovich wrote: > This patchset integrates sbuild/schroot tools into Isar build system. > > Base schroot images are created with sbuild-chroot-target or, in case > of cross-build, sbuild-chroot-host recipes. These images are based on > isar-bootstrap ones, but include some build-related stuff preinstalled. > > To use these changes you need to have sbuild and schroot installed and > configured. User who runs build should be added to sbuild group. > > Currently, sbuild-related stuff is supported in latest kas. > > If Gilab CI is used, the following configuration changes are required: > - external dir for schroot overlay should be set in the config > /etc/gitlab-runner/config.toml: > > volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"] > > The patchset was tested with 'meta-iot2050' and 'xenomai-images' > downstreams in combination with kas image mentioned above. > > Current limitations: > - parallel building of several packages requires more free space in > comparison with buildchroot-based architecture. This happens due > the sbuild architecture which uses some basic layer (common for all > packages) and per-package separate layer (where builddeps are downloaded > and installed). > > Changes sinse v11: > - Rebase on current next. > > Changes sinse v10: > - Rebase on current next. > > Changes sinse v9: > - Add caching improvement patch; > - Remove per-package deb artifacts dir after usage to reduce disk space; > - Improve schroot cleanup on failures. > > Changes since v8: > - Added patch with support of package versions with epochs > > Changes since v7: > - fixed work with proxy by adding some environment filters in sbuild; > - DEB_BUILD_PROFILES support simplified and improved > - Fixed gbp-based recipes rebuilding (get back --git-ignore-new option > previously removed) > - fixed base-apt caching for packages and source packages temporary > installed in sbuild isolated environment > - fixed rare build error related to ccache setup (added locking) > > Changes since v6: > - added patch that switches sbuild from using source directory to using > .dsc file. The file is generated before sbuild call by `dpkg-source`. > > Changes since v5: > - warning about shell exports in dpkg_runbuild_prepend. This should > avoided be reworked to templates. > > Changes since v4: > - consider shell exports done in dpkg_runbuild_prepend and pass them > to sbuild environment; > - fixed ccache work. > > Changes since v3: > - dpkg_do_mounts() and dpkg_undo_mounts() are not removed for downstreams > compatibility; > - dpkg_build_export is used for adjusting sbuild environment; > - DEB_BUILD_PROFILES support for sbuild is fixed in case of cross-build. > > Changes since v2: > - patches reworked/squashed for easier reading and understanding; > - fixed building foreigh architectures with kas-docker; > - implemented support of ccache; > - fixed devshell and devshell_nodeps. > > Changes since v1: > - parallel builds with different BUILD_DIR are supported; > - parallel multiconfig targets in one build are supported; > - per-task schroot configuration in /etc/schroot/ is now used; > - patchset now passes Jenkins CI (so patches changes RFC => PATCH). > > Anton Mikanovich (3): > dpkg: Build packages with sbuild > dpkg-base: Cleanup on schroot fail > sbuild: Add changelog entry > > Felix Moessbauer (4): > fix: support build of packages with epoch version > always create apt-cache dirs in deb_dl_dir_import > avoid absolute SCHROOT_* paths to improve caching > locate generated binary package in WORKDIR > > Uladzimir Bely (14): > dpkg-gbp: Use separate command to export tarball > dpkg-gbp: Use host tools for dsc preparation > sbuild: Add recipes for host and target rootfs to run sbuild > sbuild: Introduce a class for another build method > sbuild: Support of DEB_BUILD_PROFILES > sbuild: Support of shell exports from dpkg_runbuild_prepend > dpkg: Remove builddeps install task > sbuild: Add ccache support > dpkg-base: Switch devshell to use schroot > dpkg-base: Switch apt_fetch and apt_unpack to use schroot > doc: Add sbuild-related documentation > sbuild: Use .dsc file instead of source directory > sbuild: Fixed proxy support > sbuild: Fix debsrc_download for packages dependencies > > RECIPE-API-CHANGELOG.md | 13 ++ > doc/user_manual.md | 22 ++- > meta/classes/deb-dl-dir.bbclass | 6 +- > meta/classes/dpkg-base.bbclass | 90 ++++++---- > meta/classes/dpkg-gbp.bbclass | 26 +-- > meta/classes/dpkg.bbclass | 109 +++++++++--- > meta/classes/sbuild.bbclass | 159 ++++++++++++++++++ > meta/conf/bitbake.conf | 2 + > .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ > .../sbuild-chroot/sbuild-chroot-target.bb | 10 ++ > .../sbuild-chroot/sbuild-chroot.inc | 39 +++++ > 11 files changed, 408 insertions(+), 81 deletions(-) > create mode 100644 meta/classes/sbuild.bbclass > create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb > create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb > create mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc > This is also just a rebase on the current next. We are planning to merge it in the next few days. Bitbake update will be the first priority after that.