From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7032253102499561472 X-Received: by 2002:a5d:4b82:: with SMTP id b2mr60917967wrt.419.1638984048871; Wed, 08 Dec 2021 09:20:48 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:fe0b:: with SMTP id n11ls955720wrr.0.gmail; Wed, 08 Dec 2021 09:20:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJxofqmVwGs0RoohiK8f43hgXXiCgClS4ikgDu4t9VkZuegDf8lmvSZ64FaXeYsbpmNQtBqk X-Received: by 2002:adf:eb06:: with SMTP id s6mr59126wrn.96.1638984047767; Wed, 08 Dec 2021 09:20:47 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1638984047; cv=pass; d=google.com; s=arc-20160816; b=bz9SYu4y2ExOHP997jaLLgqsUg0YR1Z1YoMht5n6nuwxVa9A2G4IOGvihuh4t3t+9w HOo3Xr5Nq7O3VVzwtcwPItQYWQ+KNwwM0V/Vjl8dfcORsUjhK/YjquCV67Bop7IfXtd+ ppRpmrymsmYMBXc2xmp1s3bWQ9GecrTnUBsHI4bgEXO8OIDvmcPtf7VHyAy/MSN13BI9 J2IGhp6iSqjb1byF2iZYOyBs2cbXZtFMAp3p43AJRcgYU2JW5G+/BTmJSuQ1jaZzVdDJ NvoIA7Q0i1enexenNdcO81vNSxqOZXZmxwWt3+11TqsFdwTVtG2aD1BcJo0PTta35XdC nUQA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :dkim-signature; bh=pUNluoMDjI3Rs3FRW3q965FrcCYrKGSb4M6Xs6xI1eo=; b=NupFGpDn5BxzVrk7gs/ZM3J8HJzlO1ANA204Z7ihPyQRwWAqMWNfdKhP3Z6NW293ud I0khWyrHtOLsEV1T1NEX8ileUbEy5xmHgyQTzH2UNwA2zgIOtF5eiyyD/kYzFRebKMLV g2kCMBK1C6jt8ovNchRxUC4G/RfXv9akyI9y4YEOu50tkL4fnal3myxKUgXnfRB00yjf IkfaX8911rkBTF7jx42jwZv2yOn+m+P1cyiXbG1XdUqAHf78Jtyt6XSfb0mzVPvj9+kT OM0LABPyC5lmIXg47fFg5oxEyaOM2DUN+qczSZ4UxEcfYie0jT6cZ9C6F1JLRyUF5XdT io+g== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=qFBX9juu; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 40.107.20.89 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2089.outbound.protection.outlook.com. [40.107.20.89]) by gmr-mx.google.com with ESMTPS id o19si433735wme.2.2021.12.08.09.20.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 09:20:47 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 40.107.20.89 as permitted sender) client-ip=40.107.20.89; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=qFBX9juu; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 40.107.20.89 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ILsjbftZDuasIt6evWjcrjhdE7223Kb8H57g03Dhqx4lEt6y3nEfWWnapGBD+UrdmZtZcmEfWzHufFmQpXeYl65WajW0eFTFuIRFuKbuajCl8lFmMRJU4sFtaoN11oo19LkbEz70FpMoYyLmlBsjP0ac+BpEhs9tE89wTPGJNyl75eq55XNhzUUoMieqm13hi+J57brM2qpAVXM0CLnDgvPTJAgtBgMeaQl4h5/8na4Y09KuM+hS/gCTLlo2iK+fOVjMHgY3XLjFOQrfL69o6X01I9JgL5aCRdzPpJ7Aknjmke0pyDkrMvImT2/RhlpxdMB2Mr56IgKPd8+DzVo5Vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pUNluoMDjI3Rs3FRW3q965FrcCYrKGSb4M6Xs6xI1eo=; b=EsL+I4S0K0NgME6p6hut0ss0hDQ4sOG8a7SGXk7EOK5FbXC2uLKQ/7ah347Idd3/qSzgUSoPHVSKcDH8VRr36C3bP709cF49tD5juwtqD41VKsMO1M1rXQqZlOlnKS5yHbGTLFEK9lC9VG0bDZ0C+jaHtNc6rcPDWijY2p6/9RveD04xHcfVZua6Hvgz7SvZ6zrwcsOCjNoDKCwlzsV15VTdzV2UGYkeZ2+eVYAuaptkaFCkLeqzElupuRRSNfGI9Qt+cydAy8VQMmXX4topTdJO6HAvkt+tRJOgfPt2/I426ei9ipnSa1pmXl7o5mw2i2Nn4X/RV9pxZX6XvlLhBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.73) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pUNluoMDjI3Rs3FRW3q965FrcCYrKGSb4M6Xs6xI1eo=; b=qFBX9juucl8bfiHWfrxQujnu+5Lslr1zNf/+gs2X7jI6W62AjW0T/zpGypmgRMRyux2bONnhDcI2p6XCN+VXEBMA1IMNr7U00HajwClfGuP31i/TidylD3Z9YNvUiE01tTdUU+QN3p+Bt+sVSmJ6Ol7ozFJF5vwnqwxvMZaLZu50RjQuLNNULvdN8HLBO3RUMz/yMzgupL3j2rPGEMqMVn3k4sNeuEex2qSPxYjoYr//AC1LyinwNylwwD1rjz9QPzguarCYWRxLEFE1ZD2hC79XCHVDum/B26L/an3bMsCRBEwwc7T0BxaPPKvbrm6KgQMVlJJr2UfSejPcuJbm0g== Received: from AS8PR04CA0084.eurprd04.prod.outlook.com (2603:10a6:20b:313::29) by AM7PR10MB3719.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:138::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.12; Wed, 8 Dec 2021 17:20:46 +0000 Received: from VE1EUR01FT059.eop-EUR01.prod.protection.outlook.com (2603:10a6:20b:313:cafe::e1) by AS8PR04CA0084.outlook.office365.com (2603:10a6:20b:313::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.16 via Frontend Transport; Wed, 8 Dec 2021 17:20:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.73) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.73 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.73; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.73) by VE1EUR01FT059.mail.protection.outlook.com (10.152.3.127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4755.13 via Frontend Transport; Wed, 8 Dec 2021 17:20:46 +0000 Received: from DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) by DEMCHDC9SNA.ad011.siemens.net (194.138.21.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 8 Dec 2021 18:20:46 +0100 Received: from [139.25.68.37] (139.25.68.37) by DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 8 Dec 2021 18:20:45 +0100 Subject: Re: [PATCH v3 00/20] Sbuild/Schroot migration To: Uladzimir Bely , References: <20211208130544.15133-1-ubely@ilbers.de> <1978289.YKUYFuaPT4@hp> From: Jan Kiszka Message-ID: <76aefe9a-56ce-91f5-d0a6-0149fad1e507@siemens.com> Date: Wed, 8 Dec 2021 18:20:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <1978289.YKUYFuaPT4@hp> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit Return-Path: jan.kiszka@siemens.com X-Originating-IP: [139.25.68.37] X-ClientProxiedBy: DEMCHDC89XA.ad011.siemens.net (139.25.226.103) To DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4c39133f-9776-47f5-5ad6-08d9ba6f12ac X-MS-TrafficTypeDiagnostic: AM7PR10MB3719:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EhdK/KNimI2un0y/ehcPxXixKckyOAO3JwuT+j+S+RzHu9Yt98I/eiHYeYGBeWf9oapj50YdkMk94Vs6ir0C269tGiVZAol8D3sx6u4f86nEC0F31Fn5ShAr/d1FjszV8P/tEW78gktjAfnaw+ci9pnupSsaszMUcZXNMCGN3MO2/0PdujxycxxqeOvHwgUxAQP/iYbrkLVKjNjFdS+Ps3DhzS1+gjytXslUiwAG+b84+yTHiQREXhM1ekA5irrweYzmAq1fTHcwDhL1DwZORWLSGnl1e2ESJWZfjaSbtaCT8PUQEc8zaejypRkCpFbWFLPAFyyzSmmiBPfwImv4kexU6ToDKCFX+dHfJ8N+eKFtzfNGszA42Izqa3eOx8jACxr5baStGGR1T57+ecgrjR+YqxQhYAejlSn0kk74FLmE4rKUd6I89pMoil/jkxUIoHaTcHmEecUG1modS75I1FskzXI+EXV5ksQ+8hTG8eGzorZVHFjFDZGKQAQ2cwIdl1jcxFcIGMSaUEl7cKaCxPjy+eixaRa+/T2y9ayo5Ltj5T6AqGiiTAYLerA7E0hTbyLqmH31+v/yQmkKa5/QEPb5c7hDk3nqXU3ntc5Z26/orDzBvUno3LpIjKpNHdV4PKsx6JZJ/V0+XWd8GU6SQb3peodE6ToPgKRoMN3A3/Xr+cuXgWDmfUJvjK+9/XWmureK2Mfxfv0BUd03b1hnvX7wLZ/qpLA488PPESzR0VrmHsrZNqcwx9uWr37159061uBF//emj8QEyuPgzMcAvBEdKwohHOJnJ4481tIR8YmlaTDgeKzg1FI3Rghyec8vRa5dZeriD5IRTAc+Gr4nboHRMhv5LHQJyfweFaYFexrXqfnkha/AG0IYF2WDPl45 X-Forefront-Antispam-Report: CIP:194.138.21.73;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(316002)(508600001)(8936002)(16576012)(110136005)(70206006)(5660300002)(70586007)(2906002)(36860700001)(31686004)(47076005)(8676002)(82310400004)(83380400001)(36756003)(26005)(16526019)(84970400001)(44832011)(31696002)(336012)(6706004)(356005)(82960400001)(186003)(956004)(2616005)(53546011)(86362001)(81166007)(3940600001)(43740500002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2021 17:20:46.6432 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4c39133f-9776-47f5-5ad6-08d9ba6f12ac X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.73];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR01FT059.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR10MB3719 X-TUID: NjDYWsBg2Zzv On 08.12.21 14:40, Uladzimir Bely wrote: > In mail from среда, 8 декабря 2021 г. 16:05:24 +03 user Uladzimir Bely wrote: >> This is a patchset showing how sbuild/schroot tools can be integrated >> into Isar build system. >> >> Base schroot image is created with sbuild-chroot-target (or >> sbuild-chroot-host in case of cross-build) recipe. These images are based >> on isar-bootstrap ones, but include some build-related stuff preinstalled. >> >> To use this changes you need to have sbuild and schroot installed and >> configured. Current user should be added to sbuild group. >> >> If 'kas-container' is used, it should be modified to support sbuild: >> - /var/lib/schroot/union/overlay should be externally mounted (-v option) >> - export KAS_IMAGE_VERSION=2.6.2-sbuild >> - export KAS_CONTAINER_IMAGE_PATH=ghcr.io/wiselord >> Here I uploaded a docker image that has preinstalled software (sbuild, >> schroot, etc) required to run sbuild. Later, the changes might be >> included in official kas docker images. >> >> If 'gitlab' is used, the similar changes are required: >> - 'image: ghcr.io/wiselord/kas-isar:2.6.2-sbuild' in .gitlab-ci.yml >> - external dir for schroot overlay should also be specified in >> /etc/gitlab-runner/config.toml: >> volumes = ["/path/to/overlay:/var/lib/schroot/union/overlay"] >> >> The patchset was tested with 'meta-iot2050' and 'xenomai-images' >> downstreams. Some patches are required for these downstreams, >> appropriate pull requests will be created soon. >> >> Changes since v2: >> - patches reworked/squashed for easier reading and understanding >> - fixed building foreigh architectures with kas-docker >> - implemented support of ccache >> - fixed devshell and devshell_nodeps >> Changes since v1: >> - parallel builds with different BUILD_DIR are supported >> - parallel multiconfig targets in one build are supported >> - per-task schroot configuration in /etc/schroot/ is now used >> - patchset now passes Jenkins CI (so patches changes RFC => PATCH) >> >> Current limitations: >> - parallel building of several packages requires more free space in >> comparison with buildchroot-based architecture. This happens due >> the sbuild architecture which uses some basic layer (common for all >> packages) and per-package separate layer (where builddeps are downloaded >> and installed). >> >> For experiments, the code can be taken from Isar 'ubely/sbuild' branch. >> This is a dev branch, so sometimes it can be force-pushed when some >> fixes are applied of rebased to 'next' branch. >> >> First 9 patches (1..9) - are just preparation patches. With them, Isar will >> continue using buildchroot-based architecture. So, potentially, >> they can be merged as a subseries. >> >> Remaining 11 patches (10..20) represent a sbuild-related changes. >> >> Anton Mikanovich (10): >> template: Make templates passthrough >> linux-module: Do not use shell environment >> u-boot: Do not use shell environment >> trusted-firmware: Do not use shell environment >> optee-os: Do not use shell environment >> kselftest: Do not use shell environment >> linux-mainline: Move cfg fragment test to debian/rules >> dpkg: Build packages with sbuild >> sbuild: Introduce environment variables export API >> linux-custom: Prepare kernel config inside sbuild >> >> Uladzimir Bely (10): >> dpkg-gbp: Use separate command to export tarball >> dpkg-gbp: Use host tools for dsc preparation >> sbuild: Add recipes for host and target rootfs to run sbuild >> sbuild: Introduce a class for another build method >> dpkg: Remove builddeps install task. >> dpkg-base: Switch devshell to use schroot >> dpkg-base: Switch apt_fetch and apt_unpack to use schroot >> dpkg-base: Cleanup from buildchroot parts. >> doc: Add sbuild-related documentation >> sbuild: add ccache support >> >> doc/user_manual.md | 22 ++- >> .../optee-os/optee-os-stm32mp15x_3.11.0.bb | 2 +- >> .../linux/linux-mainline_5.4.70.bb | 16 +- >> meta/classes/dpkg-base.bbclass | 98 ++++------ >> meta/classes/dpkg-gbp.bbclass | 26 +-- >> meta/classes/dpkg.bbclass | 53 ++--- >> meta/classes/sbuild.bbclass | 184 ++++++++++++++++++ >> meta/classes/template.bbclass | 7 +- >> meta/conf/bitbake.conf | 2 + >> .../files/debian/{rules => rules.tmpl} | 4 +- >> meta/recipes-bsp/optee-os/optee-os-custom.inc | 9 +- >> .../files/debian/{rules => rules.tmpl} | 4 +- >> .../trusted-firmware-a-custom.inc | 9 +- >> .../u-boot/files/debian/{rules => rules.tmpl} | 2 +- >> meta/recipes-bsp/u-boot/u-boot-custom.inc | 9 +- >> .../sbuild-chroot/sbuild-chroot-host.bb | 13 ++ >> .../sbuild-chroot/sbuild-chroot-target.bb | 10 + >> .../sbuild-chroot/sbuild-chroot.inc | 39 ++++ >> .../kselftest/files/{rules => rules.tmpl} | 0 >> meta/recipes-kernel/kselftest/kselftest.inc | 21 +- >> .../linux-module/files/debian/rules | 37 ---- >> .../linux-module/files/debian/rules.tmpl | 52 +++++ >> meta/recipes-kernel/linux-module/module.inc | 19 +- >> meta/recipes-kernel/linux/linux-custom.inc | 7 +- >> 24 files changed, 437 insertions(+), 208 deletions(-) >> create mode 100644 meta/classes/sbuild.bbclass >> rename meta/recipes-bsp/optee-os/files/debian/{rules => rules.tmpl} (75%) >> rename meta/recipes-bsp/trusted-firmware-a/files/debian/{rules => >> rules.tmpl} (77%) rename meta/recipes-bsp/u-boot/files/debian/{rules => >> rules.tmpl} (94%) create mode 100644 >> meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb create mode >> 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb create >> mode 100644 meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc rename >> meta/recipes-kernel/kselftest/files/{rules => rules.tmpl} (100%) delete >> mode 100755 meta/recipes-kernel/linux-module/files/debian/rules create mode >> 100755 meta/recipes-kernel/linux-module/files/debian/rules.tmpl > > Just a question: Does 'xenomai-images' and 'meta-iot2050' have their own > maillists where I could send the changes to work with isar-based sbuild? I > just found a maillist for 'kas', but not for these projects. > For xenomai-images, we handle changes via xenomai@xenomai.org (please additional tag related patches with "[xenomai-images]"). meta-iot2050 takes pull requests on github. For both projects, changes that will work with current Isar can already be submitted. Changes that depend on your pending series could be proposed along with an Isar revision bump once 'next' picked up your queue. Or send them earlier with a clear remark that those are for reference and future use. But it is probably better to wait for the 'next' merge. Thanks, Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux