From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7057122428766388224 X-Received: by 2002:a05:6000:101:: with SMTP id o1mr24870063wrx.487.1643795175035; Wed, 02 Feb 2022 01:46:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:a78a:: with SMTP id q132ls2634119wme.0.gmail; Wed, 02 Feb 2022 01:46:14 -0800 (PST) X-Google-Smtp-Source: ABdhPJxiH/13Y1wqFbLVa0yLEIm6636cvpdr6kfFqc8VJnWlpE5dOnrCLBaTtmlU+jMT9dVHB8F6 X-Received: by 2002:a05:600c:4808:: with SMTP id i8mr5313511wmo.23.1643795174033; Wed, 02 Feb 2022 01:46:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1643795174; cv=pass; d=google.com; s=arc-20160816; b=IJdSwRN7cpXlkRkZt45+7+sEHCRkkZUo7+LphTK0vhu5s7zWKAvhlzAlqsVaU2co3l ZBOAAuMIhqT3JHvEEuKUI2F3jnwL0V2fgktyu5s+Vm5AbjQZC00/V6NwwBrsWGd/yVR7 e94VmHp6WTbqFg/Sp9sljbRUbowY4v5NR4UpmG6hcmTu9xsb2o3z1/oCeBLCFdIKyBsJ bLdUzpE4f0h1i1Hq4G7EmzVrXOV5Bfg3hvj1Xb9IjdvHO9hf12ZsSSsmEVwT4cUMuqQX lkTeA5/ECV9OU97/+zWIcwExU92WhG6hxQiJQZXsTytKdste3F3EDrOXyuWvoGnUqzba Zlfw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :dkim-signature; bh=8ShtZOQGSxK1sNZaaKvk7Q3HFk3pUirQv041y1FfqaQ=; b=hQcnO39wT6Nnb+X6Dkxn92G7DLvotWuPiO241hHk22XhHVk5KLBqTPyjQxyWq0Z/UR SUpRpyMtapWFDfXt1jPcJUY/nVosTVCTFrgvZ12YwJneJJFSk5nqIuGJvp3bwXAqU4Ww hedHGZyPC8wsL6whYsHzi1XroBYt9GfME4B9w/2Uv6JL83sGKNuTaC24SFG29CZOulMQ hiBstuE7CQvnsjm/f7JpU3d9pqh9uoRvxTylYDWRpyOEF0gdxZXldkWgTZtwRnBuaV/A G/8YcLqKFx+0v07uGxbwBjbJOxLUf5/zQwM/Fky4GRXhrmruC3EXDUlHNgsebZYcvmsZ bCTA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=sFvBf8Yb; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 40.107.7.80 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70080.outbound.protection.outlook.com. [40.107.7.80]) by gmr-mx.google.com with ESMTPS id l24si339143wmg.1.2022.02.02.01.46.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Feb 2022 01:46:13 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 40.107.7.80 as permitted sender) client-ip=40.107.7.80; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=sFvBf8Yb; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 40.107.7.80 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fv0Ag6QbSwUxSAzlsJ9ANjYAjXuCK7h6vU+cATuCP4D1f4V3kl1Mxu5qljTlIGSUHhFYwRRtTanXELwDnNJRyPzynE8OHn8caqBFlvommXSOT7TQLnhiz083xLYXNv+FvbjJ1/HIV6p2EILGJ3mDwGQYooVd7rXnzr68B6Hb3HLnpRjA5XQpWeL9Tqmka84j1Cc4VNMiiu/RxS9adsC5rwOKORZhnBOqOYk1mkT55v49beZUWNonsUW5AHB3uQM9Tpv04EbOxmvef+OQamhk0e54MVFzBuo6/H6faYJWDn7kSC6rcB3bska5euPPbyh6U4invK08DIuU6quCP1oCFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8ShtZOQGSxK1sNZaaKvk7Q3HFk3pUirQv041y1FfqaQ=; b=mvK2fG8ZVqAj/wRCi9ilrKx703t6YUvU9QRjuTmaqh5qxtIHK5UogA5ABJPUAbUH83NlbJE8z2qN01AC38OR39XkwJLkkjT70mFU+J53xOTb70r+MUOuxaPhib5ykufvYz9qTSPTBWlSCC7VLczmZ8aXZT6HbuvTpgPoJdh/Nub8HPYUbURVq/F7UWkY590PCGX1YijP04AQr7cpd6Dov/O51c7b6dGWLgjpPJDs/hTlG8NlbkTchNmu7lqdRLVPD7Q/8Qk6lk4TiGmbHqQm2MBH1IlHdJof1yufa+ELF68U7OIZ/HfYX6kRV7Jl10ZDrujJ9sJ1vnaqRfNzBLSQtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.70) smtp.rcpttodomain=ilbers.de smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8ShtZOQGSxK1sNZaaKvk7Q3HFk3pUirQv041y1FfqaQ=; b=sFvBf8Yb01njT0ksfry/fm1fdkfFiM3w0rIhOeXQ5DONrYDKSbO3HZlNdsXElfsg2VdgU/Fjnv+rORxTjtx1VhWU88bA0p1p4PiRw7xxa1qKtj1zcl3fbrdakYN5Jk59e5jAsnHefaI1GFnlcN++XZrVA0nPzr8RP5H9155HJYHT0LPuXb/NIJ69jbaJ/X006TpbQkQfw7QfZh/YyqrugyMdmjGMXOaCkMvmUxy8oGeE+Bbk4vi/BEHvjzizUhX5xvyLzyZnU4HC4Mr7ABRUtaMfM1Iq/mODFsk/4s/Zt4nsMo4SLbk5UUpYSxTKKmlxk6YHQpC81+AJVx6+5SxFiQ== Received: from DB9PR02CA0029.eurprd02.prod.outlook.com (2603:10a6:10:1d9::34) by VI1PR10MB3486.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:800:137::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.17; Wed, 2 Feb 2022 09:46:12 +0000 Received: from DB5EUR01FT006.eop-EUR01.prod.protection.outlook.com (2603:10a6:10:1d9:cafe::d) by DB9PR02CA0029.outlook.office365.com (2603:10a6:10:1d9::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.22 via Frontend Transport; Wed, 2 Feb 2022 09:46:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.70) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.70 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.70; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.70) by DB5EUR01FT006.mail.protection.outlook.com (10.152.4.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4951.12 via Frontend Transport; Wed, 2 Feb 2022 09:46:12 +0000 Received: from DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) by DEMCHDC9SJA.ad011.siemens.net (194.138.21.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Wed, 2 Feb 2022 10:46:11 +0100 Received: from [139.25.68.37] (139.25.68.37) by DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 2 Feb 2022 10:46:11 +0100 Message-ID: <64add802-9820-8d78-969c-c7dc1eac5fe0@siemens.com> Date: Wed, 2 Feb 2022 10:46:10 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [PATCH v5 07/12] sbuild: support of shell exports from dpkg_runbuild_prepend Content-Language: en-US To: Uladzimir Bely , "Moessbauer, Felix (T CED SES-DE)" , "isar-users@googlegroups.com" References: <20220201170038.5723-1-ubely@ilbers.de> <4880928.Qq0lBPeGtt@home> From: Jan Kiszka In-Reply-To: <4880928.Qq0lBPeGtt@home> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-Path: jan.kiszka@siemens.com X-Originating-IP: [139.25.68.37] X-ClientProxiedBy: DEMCHDC89XA.ad011.siemens.net (139.25.226.103) To DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9cddcdf1-fb55-434e-f476-08d9e630d915 X-MS-TrafficTypeDiagnostic: VI1PR10MB3486:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2089; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Dtk6BmJKcvHaTcY6I4a7hdYZ1vE//AITiWSYGSUKKo4hH4fHrGV+AIBNsBcDsVLkBJ9RUHPhrw2k5UYca+NwA/snRvJJRPghOCImYVKxWtUXSwfBp75joualDvzqNFet2QJfJka1pCIOFRIaOtNjC0wmsiBukxkTZs2rDHwPYP2WJtVQmnG60vxbjeFdV0GueRCAC7vT/C8VfjSIcF8Lnq7TqB5jgwldUS0L33dKkmq6/VTs0WnKQWJYQZugPJ0K1FHZ4ucUKSCk/wmFgaqYwNp16Eoqfuhb/tTW9v57DhEi3NPQ+32fSd71ZhlB7Huj0CsW1hDM13T04SmSG9b6CWkzWQHdmLH7lOOLab0K+Dz/iBi3u8X0AjBBHztbQI6fJ68VOPqGXghgLSt23hrVsWsOTN+hhRVhMCGT8LKiVPNb/cRexO12CfZQ991bYsUFOoOcfzlwWDSWBjO7r15DumGukvPm+QCAwMBFbKr6P8nB2wcUacfTLr24rlZcZv8FUuNkIXHlGxTy78m/WlS313HBZrYbIjEz+GfaotaGFWFDL1PrZKzzvZMpH2cij1crj/B+CSUIwd+SXuzkvvfhkDVJu1cMjG7tFttSDNOKc1ikidGAnWGT68sUdTEfBUx852SqeWNlK+VZ5YoukIRhRSkZNqZ827ChNZeAmkLp0BVx1KaR5aWJimCeTZl6szyfeRf5nkpQUYCTfZXgDVcVlPjA3Irznx/3bvSTnkdYZgxJUDf4/3EyyVyEwRGH+DVY X-Forefront-Antispam-Report: CIP:194.138.21.70;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:hybrid.siemens.com;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(6706004)(16576012)(31686004)(83380400001)(36860700001)(40460700003)(36756003)(316002)(110136005)(26005)(336012)(16526019)(186003)(47076005)(2616005)(956004)(2906002)(44832011)(7636003)(7596003)(82310400004)(53546011)(5660300002)(8676002)(508600001)(31696002)(356005)(82960400001)(70586007)(70206006)(86362001)(8936002)(3940600001)(43740500002);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2022 09:46:12.3810 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9cddcdf1-fb55-434e-f476-08d9e630d915 X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.70];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR01FT006.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR10MB3486 X-TUID: Y2vhRHPTphR3 On 02.02.22 10:13, Uladzimir Bely wrote: > In the email from Wednesday, 2 February 2022 11:52:29 +03 user Jan Kiszka > wrote: >> On 02.02.22 09:46, Moessbauer, Felix (T CED SES-DE) wrote: >>>> -----Original Message----- >>>> From: isar-users@googlegroups.com On >>>> Behalf Of Jan Kiszka >>>> Sent: Wednesday, February 2, 2022 7:57 AM >>>> To: Uladzimir Bely ; isar-users@googlegroups.com >>>> Subject: Re: [PATCH v5 07/12] sbuild: support of shell exports from >>>> dpkg_runbuild_prepend >>>> >>>> On 01.02.22 19:41, Uladzimir Bely wrote: >>>>> In the email from Tuesday, 1 February 2022 21:09:07 +03 user Jan Kiszka >>>> >>>> wrote: >>>>>> On 01.02.22 18:00, Uladzimir Bely wrote: >>>>>>> Many of recipes often use shell exports done in dpkg_run_prepend, so >>>>>>> that this changed environment is used during build. >>>>>>> >>>>>>> While sbuild is performed in isolated environment, we need a way to >>>>>>> pass these variables to it. This is done by storing environment >>>>>>> before dpkg_runbuild (after prepare_build) and finding just before >>>>>>> the actual build what was changed or added. >>>>>>> >>>>>>> Signed-off-by: Uladzimir Bely >>>>>>> --- >>>>>>> >>>>>>> meta/classes/dpkg.bbclass | 18 ++++++++++++++++++ >>>>>>> 1 file changed, 18 insertions(+) >>>>>>> >>>>>>> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass >>>>>>> index 66db7ec5..c252e9b3 100644 >>>>>>> --- a/meta/classes/dpkg.bbclass >>>>>>> +++ b/meta/classes/dpkg.bbclass >>>>>>> @@ -29,12 +29,30 @@ do_install_builddeps[lockfiles] += >>>>>>> "${REPO_ISAR_DIR}/isar.lock"> addtask devshell after >>>>>>> do_install_builddeps >>>>>>> >>>>>>> +DPKG_PREBUILD_ENV_FILE="${WORKDIR}/dpkg_prebuild.env" >>>>>>> + >>>>>>> +do_prepare_build_append() { >>>>>>> + env > ${DPKG_PREBUILD_ENV_FILE} } >>>>>>> + >>>>>>> >>>>>>> # Build package from sources using build script >>>>>>> dpkg_runbuild() { >>>>>>> >>>>>>> E="${@ isar_export_proxies(d)}" >>>>>>> E="${@ isar_export_ccache(d)}" >>>>>>> export PARALLEL_MAKE="${PARALLEL_MAKE}" >>>>>>> >>>>>>> + env | while read -r line; do >>>>>>> + # Filter the same lines >>>>>>> + grep -q "^${line}" ${DPKG_PREBUILD_ENV_FILE} && continue >>>>>>> + # Filter some standard variables >>>>>>> + echo ${line} | grep -q "^HOME=" && continue >>>>>>> + echo ${line} | grep -q "^PWD=" && continue >>>>>>> + >>>>>>> + var=$(echo "${line}" | cut -d '=' -f1) >>>>>>> + value=$(echo "${line}" | cut -d '=' -f2-) >>>>>>> + sbuild_export $var "$value" >>>>>>> + done >>>>>>> + >>>>>>> >>>>>>> profiles=$(grep "DEB_BUILD_PROFILES" ${SBUILD_CONFIG} | tail -n1 >>>>>>> | >>>>>>> cut -d "'" -f 4) if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then >>>>>>> >>>>>>> profiles="${profiles} cross nocheck" >>>>>> >>>>>> So, this basically decouples "Avoid using shell environment during >>>>>> the build" and similar conversions downstream from this series, >>>>>> right? It's indeed good to have a compat path now. >>>>>> >>>>>> However, what are the patterns we want to push? Avoiding exports? >>>>>> Then we should probably warn here that this compat path should be >>>>>> avoided and might be removed in the future. >>>>>> >>>>>> If we want to keep both, we can probably leave several recipes alone >>>>>> in that other series. >>>>>> >>>>>> Jan >>>>> >>>>> You might be right. Current solution doesn't force users to avoid >>>>> using exports while they are silently supported. >>>>> >>>>> The first internal solution was keeping dpkg_build_export function but >>>>> also passing custom shell exports (from dpkg_runbuild_prepend) to >>>>> sbuild environment. And a warning about migration was shown. >>>>> >>>>> Later I removed this to make new patchset less invasive for downstreams. >>>>> Anyway, if we agree that "dpkg_build_export" (like API function) is >>>>> more preferable, I can get back this solution with warnings. >>>> >>>> This is not necessarily what I'm suggesting. Your conversion series works >>>> without that API, primarily using template. We also have build profiles. >>>> But we first of all need to define what we actually want. >>>> >>>> The benefit of pushing things into the build env, rather than feeding it >>>> from the recipe, is likely that the resulting deb-src of such generated >>>> packages can actually be used for rebuilding independently of Isar. So >>>> there is likely value in deprecating classic export. >>> >>> Using environment variables to control the build in Debian/rules is >>> generally discouraged (except for DEB_BUILD_PROFILES and >>> DEB_BUILD_OPTIONS). But even for these variables, semantics are >>> documented [1] and we / an ISAR user should strictly follow them when >>> packaging. >>> >>> There are also reasons why you want to build a package in a clean >>> environment (e.g. to avoid accidentally setting CC or CXX to icc or >>> clang). Hence, I highly vote for not forwarding the environment into the >>> sbuild build environment. >> ...without a warning about a grace period during that we will still do >> that to allow downstream migration. Then we have a plan, I would say. >> >> Jan > > To make things clear, there are now three options: > > 1) (Current) Just support passing shell exports user done in > dpkg_runbuild_prepend to (s)build environment. Users of downstreams are not > notified, so they continue using these exports. > > 2) Continue support custom shell exports by passing them to (s)build env, BUT > show warnings, that user should use some 'dpkg_build_export'. In general, this > doesn't differ much from the previous approach. > > 3) Continue support custom shell exports by passing them to (s)build env, BUT > show warnings, that this stuff should be converted to templates. After some > period, stop supporting exports. Templates or build profiles. > > 4) Doesn't support shell exports passing to (s)build env, so it will > immediately break the builds. > > As I understand correctly, the way (3) is the best solution. > That is also my understanding. Thanks, Jan -- Siemens AG, Technology Competence Center Embedded Linux