From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7197016923412889600 X-Received: by 2002:a2e:a379:0:b0:295:8ef2:8711 with SMTP id i25-20020a2ea379000000b002958ef28711mr3679325ljn.10.1677248676838; Fri, 24 Feb 2023 06:24:36 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:211b:b0:295:a2a6:79c with SMTP id a27-20020a05651c211b00b00295a2a6079cls501122ljq.11.-pod-prod-gmail; Fri, 24 Feb 2023 06:24:35 -0800 (PST) X-Google-Smtp-Source: AK7set9r3+HzIQ1FdQFam8i0fr1ui+Iz6mYyLnBY79z63W+1+8cHFVT3PcAqBeECqwlFiVAQ3wjC X-Received: by 2002:a05:651c:b28:b0:295:a3a5:3b79 with SMTP id b40-20020a05651c0b2800b00295a3a53b79mr1352241ljr.13.1677248675272; Fri, 24 Feb 2023 06:24:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677248675; cv=none; d=google.com; s=arc-20160816; b=dTJgI4yEIfNEIOa1qrvHsCPdhB4qM+9yqqiCNXRT4R4mDId6Y1ueNXKvTKasg2yI8H NpXlSG0PFyILQ4OD8bOB3p08VIIomtCm32TddF85RBNvEGEygy4iqHK4aVQgchfBVxDL DQBw44WJOYOlRutduDCKi0Hz5en4inw1oD2Izg61iMHsFAq6NwmyTxrtZS4Ymagad9K6 GDFybTUr5f7EwPLgOEcMKmPWZB6Qk5C9COP3ovBXRaH55GrWS0pjpO5ROTRqLO+PLPhp T5PxGKonvOuAeU/R2HfxuzNO7GByh4mDJx+tWfY9C/HhQnZS4M04z3ChTXlevDj9Wolk Qsew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=5YzTe7YeTI4jO1O/L6pU9bm/IuPQDXsRgM6p+2Ct11o=; b=IIaTTa+h0k0ZaKn9d1WjeL001X/EdJOlqFcgv19cKy46DBTTelyDbD3lhFghTk7KUs gvWzCTQjIUJ+X1zxkWhqD1AQkX5wxOdI4rvJPLSsjYQPpUGrfYnNbOkIjLPqm0qGxrgT g6ZG0TsGR/T0u23pTYc8Yaj53kRPZTugVsrPJLZMZh2h9KVCkFFnECRwnrjx5DUQnwLi nugNe8RnFDSdpRTwxtiPcx1XpAT5Ofe9cC+jmN5rScsMUFPcCceBKrJa9kWkmtPpb57t 8UfuDbRsZVj+M141QcnclKT1Um0mSAfhMIv83FYWKW9sOhlJiPL2/xiAUQS8L0E4Dg+j F87w== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=LWnV9P7v; spf=pass (google.com: domain of fm-526363-20230224142434f49d0face41f4cc11d-wdmouh@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-526363-20230224142434f49d0face41f4cc11d-wdMoUH@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from mta-64-227.siemens.flowmailer.net (mta-64-227.siemens.flowmailer.net. [185.136.64.227]) by gmr-mx.google.com with ESMTPS id z12-20020a2e884c000000b0029599709f7bsi353260ljj.2.2023.02.24.06.24.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Feb 2023 06:24:35 -0800 (PST) Received-SPF: pass (google.com: domain of fm-526363-20230224142434f49d0face41f4cc11d-wdmouh@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) client-ip=185.136.64.227; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=LWnV9P7v; spf=pass (google.com: domain of fm-526363-20230224142434f49d0face41f4cc11d-wdmouh@rts-flowmailer.siemens.com designates 185.136.64.227 as permitted sender) smtp.mailfrom=fm-526363-20230224142434f49d0face41f4cc11d-wdMoUH@rts-flowmailer.siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: by mta-64-227.siemens.flowmailer.net with ESMTPSA id 20230224142434f49d0face41f4cc11d for ; Fri, 24 Feb 2023 15:24:34 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=adriaan.schmidt@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=5YzTe7YeTI4jO1O/L6pU9bm/IuPQDXsRgM6p+2Ct11o=; b=LWnV9P7vOmrmNlHHj5qEyh2GdDMNsm+G3ch93pgziO3OZ6/IR6pioDJaElzgbfVF0SzRdJ CxVZSek2pLEGL2CueXg45NakM1Kzx8ORWPpHXfCm78OG5x/2htULUjBwycYoKswwCEzXbD8K VhQeuE/9covSJ/vdJNYobY63XkAq8=; From: Adriaan Schmidt To: isar-users@googlegroups.com Cc: Adriaan Schmidt Subject: [PATCH v3 2/4] add multiarch support Date: Fri, 24 Feb 2023 15:24:30 +0100 Message-Id: <20230224142432.3767746-3-adriaan.schmidt@siemens.com> In-Reply-To: <20230224142432.3767746-1-adriaan.schmidt@siemens.com> References: <20230224142432.3767746-1-adriaan.schmidt@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-526363:519-21489:flowmailer X-TUID: 3biVA2Tl2NhZ This adds support for building packages for native and compat architectures to dpdk-base.bbclass. Thus, all package recipes automatically have a *-native and *-compat target, which can be used in DEPENDS/RDEPENDS definitions. Additionally those targets can be used in IMAGE_INSTALL, where they are automatically converted to install the correct debian package: foo-compat -> foo:${COMPAT_DISTRO_ARCH} foo-native -> foo:${HOST_ARCH} Note that the switch ISAR_ENABLE_COMPAT_ARCH still exist and controls addition of the compat architecture during bootstrapping. Signed-off-by: Adriaan Schmidt --- .../recipes-app/samefile/samefile_2.14.bb | 2 +- meta/classes/compat.bbclass | 40 +++++++++ meta/classes/debianize.bbclass | 2 +- meta/classes/dpkg-base.bbclass | 1 + meta/classes/image.bbclass | 4 +- meta/classes/multiarch.bbclass | 88 +++++++++++++++++++ meta/classes/native.bbclass | 10 +++ meta/classes/sdk.bbclass | 2 +- meta/conf/bitbake.conf | 1 + .../isar-bootstrap/isar-bootstrap.inc | 2 + .../sbuild-chroot/sbuild-chroot.inc | 14 +-- 11 files changed, 156 insertions(+), 10 deletions(-) create mode 100644 meta/classes/compat.bbclass create mode 100644 meta/classes/multiarch.bbclass create mode 100644 meta/classes/native.bbclass diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb index 5e36a2ac..c53c9445 100644 --- a/meta-isar/recipes-app/samefile/samefile_2.14.bb +++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb @@ -21,7 +21,7 @@ do_prepare_build() { # deb_debianize. Pre-exisiting files will not be recreated, changelog # will be prepended unless its latest entry is for CHANGELOG_V. cat << EOF > ${WORKDIR}/changelog -${PN} (0.1) unstable; urgency=low +${BPN} (0.1) unstable; urgency=low * a long long time ago there was an early version diff --git a/meta/classes/compat.bbclass b/meta/classes/compat.bbclass new file mode 100644 index 00000000..1ca3e960 --- /dev/null +++ b/meta/classes/compat.bbclass @@ -0,0 +1,40 @@ +# This software is a part of ISAR. +# Copyright (C) 2023 Siemens AG +# +# SPDX-License-Identifier: MIT + +# this class is "dual-use": it can be inherited (e.g., by bootstrap and image +# classes) to access variables and functions, and it's also added via BBCLASSEXTEND +# when inheriting multiconfig.bbclass. + +################################################################################ +# generic functions +################################################################################ + +# calculate COMPAT_DISTRO_ARCH and ISAR_ENABLE_COMPAT_ARCH +# this must always use the DISTRO_ARCH override (not PACKAGE_ARCH), so needs +# to happen in a modified environment +python() { + distro_arch = d.getVar('DISTRO_ARCH', True) + package_arch = d.getVar('PACKAGE_ARCH', True) + overrides = d.getVar('OVERRIDES', True).split(':') + + localdata = bb.data.createCopy(d) + new_overrides = [distro_arch] + [o for o in overrides if not o == package_arch] + localdata.setVar('OVERRIDES', ':'.join(new_overrides)) + isar_enable_compat_arch = localdata.getVar('ISAR_ENABLE_COMPAT_ARCH', True) + compat_distro_arch = localdata.getVar('COMPAT_DISTRO_ARCH', True) + + d.setVar('COMPAT_DISTRO_ARCH', compat_distro_arch) + d.setVar('ISAR_ENABLE_COMPAT_ARCH', isar_enable_compat_arch) +} + +def isar_can_build_compat(d): + return (d.getVar('COMPAT_DISTRO_ARCH', True) is not None and + d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == '1') + +################################################################################ +# package recipe modifications when building *-compat: +################################################################################ + +PACKAGE_ARCH:class-compat = "${COMPAT_DISTRO_ARCH}" diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass index a6694a00..1b98c02d 100644 --- a/meta/classes/debianize.bbclass +++ b/meta/classes/debianize.bbclass @@ -31,7 +31,7 @@ deb_add_changelog() { date=$(LANG=C date -R -d @${timestamp}) cat < ${S}/debian/changelog -${PN} (${changelog_v}) UNRELEASED; urgency=low +${BPN} (${changelog_v}) UNRELEASED; urgency=low * generated by Isar diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index ad28f7b3..55cc6655 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -5,6 +5,7 @@ # SPDX-License-Identifier: MIT inherit sbuild +inherit multiarch inherit debianize inherit terminal inherit repository diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index ef7d5a2a..ce7c549c 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -79,9 +79,11 @@ image_do_mounts() { buildchroot_do_mounts } +inherit multiarch + ROOTFSDIR = "${IMAGE_ROOTFS}" ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache" -ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" +ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${@isar_multiarch_packages('IMAGE_INSTALL', d)}" ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}" diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass new file mode 100644 index 00000000..21a4f610 --- /dev/null +++ b/meta/classes/multiarch.bbclass @@ -0,0 +1,88 @@ +# This software is a part of ISAR. +# Copyright (C) 2021-2022 Siemens AG +# +# SPDX-License-Identifier: MIT + +BPN = "${PN}" + +inherit compat +python() { + # provide compat only when we can build it + if isar_can_build_compat(d): + d.appendVar('BBCLASSEXTEND', ' compat') + + # build native separately only when it differs from the target variant + if d.getVar('HOST_ARCH', True) == d.getVar('DISTRO_ARCH', True): + pn = d.getVar('PN', True) + if not pn.endswith('-native') and not pn.endswith('-compat'): + provides = (d.getVar('PROVIDES', True) or '').split() + for p in provides: + d.appendVar('PROVIDES', f' {p}-native') + d.appendVar('PROVIDES', f' {pn}-native') + else: + d.appendVar('BBCLASSEXTEND', ' native') +} + +python multiarch_virtclass_handler() { + # In compat/native builds, ${PN} includes the -compat/-native suffix, + # so recipe-writers need to be careful when using it. Most of the time, + # they probably want to use ${BPN}, and in general, it's their responsibility + # to do so. If they don't, then it's ok for the build of the compat/native + # variant to fail. However, some variables are evaluated at parse time, + # and this will break the recipe even when compat/native is not requested. + # e.g., SRC_URI="file://${PN}" will try to checksum the local file at + # parse time, and parsing always happens for all build variants. So in those + # few variables, we automatically replace ${PN} with ${BPN}. + def fixup_pn_in_vars(d): + vars = 'SRC_URI FILESPATH'.split() + for var in vars: + v = d.getVar(var, False) + if v is not None: + d.setVar(var, v.replace('${PN}', '${BPN}')) + + # When building compat/native, the corresponding suffix needs to be + # propagated to all bitbake dependency definitions. + def fixup_depends(suffix, d): + vars = 'PROVIDES RPROVIDES DEPENDS RDEPENDS'.split() + for var in vars: + multiarch_var = [] + val = d.getVar(var, True) + if val is None: + continue + for v in val.split(): + if v.endswith('-compat') or v.endswith('-native'): + multiarch_var.append(v) + else: + multiarch_var.append(v + suffix) + d.setVar(var, ' '.join(multiarch_var)) + + pn = e.data.getVar('PN') + if pn.endswith('-compat'): + e.data.setVar('BPN', pn[:-len('-compat')]) + e.data.appendVar('OVERRIDES', ':class-compat') + fixup_pn_in_vars(e.data) + fixup_depends('-compat', e.data) + elif pn.endswith('-native'): + e.data.setVar('BPN', pn[:-len('-native')]) + e.data.appendVar('OVERRIDES', ':class-native') + fixup_pn_in_vars(e.data) + fixup_depends('-native', e.data) +} +addhandler multiarch_virtclass_handler +multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" + +# function to convert bitbake targets to installable debian packages, +# e.g., "hello-compat" to "hello:i386". +def isar_multiarch_packages(var, d): + bb_targets = (d.getVar(var, True) or '').split() + packages = [] + compat_distro_arch = d.getVar('COMPAT_DISTRO_ARCH', True) + host_arch = d.getVar('HOST_ARCH', True) + for t in bb_targets: + if t.endswith('-compat') and compat_distro_arch is not None: + packages.append(t[:-len('-compat')] + ':' + compat_distro_arch) + elif t.endswith('-native'): + packages.append(t[:-len('-native')] + ':' + host_arch) + else: + packages.append(t) + return ' '.join(packages) diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass new file mode 100644 index 00000000..d2581ac1 --- /dev/null +++ b/meta/classes/native.bbclass @@ -0,0 +1,10 @@ +# This software is a part of ISAR. +# Copyright (C) 2023 Siemens AG +# +# SPDX-License-Identifier: MIT + +################################################################################ +# package recipe modifications when building *-native: +################################################################################ + +PACKAGE_ARCH:class-native = "${HOST_ARCH}" diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass index 0a98ea04..bfd47d0e 100644 --- a/meta/classes/sdk.bbclass +++ b/meta/classes/sdk.bbclass @@ -58,7 +58,7 @@ python __anonymous() { # rootfs/image overrides for the SDK ROOTFS_ARCH:class-sdk = "${HOST_ARCH}" ROOTFS_DISTRO:class-sdk = "${HOST_DISTRO}" -ROOTFS_PACKAGES:class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${SDK_INSTALL}" +ROOTFS_PACKAGES:class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${@isar_multiarch_packages('SDK_INSTALL', d)}" ROOTFS_FEATURES:append:class-sdk = " clean-package-cache generate-manifest export-dpkg-status" ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}" ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}" diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 05ccb7b8..dd21319a 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -81,6 +81,7 @@ QEMU_ARCH:riscv64 = "riscv64" # Codename of the repository created by the caching class DEBDISTRONAME ?= "isar" +NATIVELSBSTRING ?= "isarnative" # Strings used in sstate signature files TARGET_VENDOR = "" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 99d75e21..21a2d92f 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -274,6 +274,8 @@ do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +inherit compat + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc index fb061dac..fd8bb648 100644 --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc @@ -9,6 +9,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260 PV = "1.0" inherit rootfs +inherit compat python() { distro_gcc = d.getVar('DISTRO_GCC') @@ -20,18 +21,19 @@ python() { d.appendVar('SBUILD_CHROOT_PREINSTALL_COMMON', ' libstdc++-{}-dev:{}'.format(distro_gcc, distro_arch)) - if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1': - compat_arch = d.getVar('COMPAT_DISTRO_ARCH') + if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1': + compat_arch = d.getVar('COMPAT_DISTRO_ARCH') + d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', + ' libc6-dev:{}'.format(compat_arch)) + d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', + ' crossbuild-essential-{}'.format(compat_arch)) + if d.getVar('DISTRO_GCC'): d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', ' libgcc-{}-dev:{}'.format(distro_gcc, compat_arch)) d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', ' libstdc++-{}-dev:{}'.format(distro_gcc, compat_arch)) } -SBUILD_CHROOT_COMPAT_PREINSTALL:compat-arch = " \ - libc6-dev:${COMPAT_DISTRO_ARCH} \ - crossbuild-essential-${COMPAT_DISTRO_ARCH}" - SBUILD_CHROOT_PREINSTALL_COMMON = " \ ${SBUILD_CHROOT_COMPAT_PREINSTALL} \ libc6-dev:${DISTRO_ARCH} \ -- 2.30.2