* [PATCH v3 0/4] multiarch support
@ 2023-02-24 14:24 Adriaan Schmidt
2023-02-24 14:24 ` [PATCH v3 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt
` (4 more replies)
0 siblings, 5 replies; 8+ messages in thread
From: Adriaan Schmidt @ 2023-02-24 14:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
This adds `<package>-compat` and `<package>-native` bitbake
targets to all recipes inheriting dpkg-base.
The new -compat build variant replaces the old compat mechanism.
Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to
ensure that the bootstrap and buildchroot is prepared correctly.
Regarding testing of the new features:
- we have (limited) testing of compat, by adding `hello-isar-compat` to
amd64 and arm64 builds.
- currently no testing of native. There are two main use cases for the feature:
- SDK that needs `<package>-native`. This could be turned into a test
easily, by setting/appending SDK_INSTALL. Would require a new option
in testsuite, similar to `image_install` introduced in 57a0ade9a.
- A build tool that is provided by a recipe, when using it in cross
compilation. E.g. if someone needed a patched cmake, they would
write cmake.bb, and applications that need it would (in case cross
compilation is enabled) DEPEND+="cmake-native". Or, another real-world
example from a downstream layer: packaging applications with goreleaser,
which is not found in the Debian apt repos. Instead, I'm fetching it
via a dpkg-prebuilt recipe, written to support multiple architectures
(architecture is part of the download URL). Now if I want to cross
compile go applications, they DEPEND+="goreleaser-native", and
DEBIAN_BUILD_DEPENDS_append=", goreleaser:native".
These cases are more complex, but maybe we can find an example along
those lines and add it to meta-isar.
As testing of the native feature would (in the simple case) need an
extension of the test classes (for SDK_INSTALL), or introduce something
completely new into meta-isar, I'd like to do that after discussion, in
a later series.
Adriaan
changes since v2:
- fixed a bug that completely broke things for targets without a compat
arch (e.g., i386). The compat variant of packages is now only available
when it can actually be built.
- the native variant is only generated if it differs from the target.
If DISTRO_ARCH==HOST_ARCH, then `<package>-native` is automatically
provided by the target package.
- also do the bitbake-target->debian-package transformation on SDK_INSTALL
- fix compat packages in testsuite: when we add hello-isar-compat, we
need to remove hello-isar.
changes since v1:
- fixed an issue that prevented arch overrides of
ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite
- added `-native` expansion to contents of IMAGE_INSTALL
- documentation in user_manual
Adriaan Schmidt (4):
bitbake.conf: use PACKAGE_ARCH in overrides
add multiarch support
remove obsolete compat-arch override
doc: add compat/native targets to user manual
doc/user_manual.md | 19 ++--
.../recipes-app/hello-isar/hello-isar.bb | 3 -
meta-isar/recipes-app/libhello/libhello.bb | 3 -
.../recipes-app/samefile/samefile_2.14.bb | 2 +-
meta/classes/compat.bbclass | 40 +++++++++
meta/classes/debianize.bbclass | 2 +-
meta/classes/dpkg-base.bbclass | 1 +
meta/classes/image.bbclass | 4 +-
meta/classes/multiarch.bbclass | 88 +++++++++++++++++++
meta/classes/native.bbclass | 10 +++
meta/classes/sdk.bbclass | 2 +-
meta/conf/bitbake.conf | 6 +-
.../isar-bootstrap/isar-bootstrap.inc | 2 +
.../sbuild-chroot/sbuild-chroot.inc | 14 +--
testsuite/cibuilder.py | 4 +
15 files changed, 174 insertions(+), 26 deletions(-)
create mode 100644 meta/classes/compat.bbclass
create mode 100644 meta/classes/multiarch.bbclass
create mode 100644 meta/classes/native.bbclass
--
2.30.2
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v3 1/4] bitbake.conf: use PACKAGE_ARCH in overrides 2023-02-24 14:24 [PATCH v3 0/4] multiarch support Adriaan Schmidt @ 2023-02-24 14:24 ` Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 2/4] add multiarch support Adriaan Schmidt ` (3 subsequent siblings) 4 siblings, 0 replies; 8+ messages in thread From: Adriaan Schmidt @ 2023-02-24 14:24 UTC (permalink / raw) To: isar-users; +Cc: Adriaan Schmidt This replaces `DISTRO_ARCH` with `PACKAGE_ARCH` in the `OVERRIDES` list. Note that `PACKAGE_ARCH` defaults to `DISTRO_ARCH`, so this only has an effect when `PACKAGE_ARCH` is explicitly set in a recipe. Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> --- meta/conf/bitbake.conf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index c9f52a86..05ccb7b8 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -67,8 +67,8 @@ KERNEL_FILE:mipsel ?= "vmlinux" KERNEL_FILE:riscv64 ?= "vmlinux" KERNEL_FILE:arm64 ?= "vmlinux" -OVERRIDES = "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable" -FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}" +OVERRIDES = "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable" +FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}" COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}" # Setting default QEMU_ARCH variables for different DISTRO_ARCH: -- 2.30.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 2/4] add multiarch support 2023-02-24 14:24 [PATCH v3 0/4] multiarch support Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt @ 2023-02-24 14:24 ` Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 3/4] remove obsolete compat-arch override Adriaan Schmidt ` (2 subsequent siblings) 4 siblings, 0 replies; 8+ messages in thread From: Adriaan Schmidt @ 2023-02-24 14:24 UTC (permalink / raw) To: isar-users; +Cc: Adriaan Schmidt This adds support for building packages for native and compat architectures to dpdk-base.bbclass. Thus, all package recipes automatically have a *-native and *-compat target, which can be used in DEPENDS/RDEPENDS definitions. Additionally those targets can be used in IMAGE_INSTALL, where they are automatically converted to install the correct debian package: foo-compat -> foo:${COMPAT_DISTRO_ARCH} foo-native -> foo:${HOST_ARCH} Note that the switch ISAR_ENABLE_COMPAT_ARCH still exist and controls addition of the compat architecture during bootstrapping. Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> --- .../recipes-app/samefile/samefile_2.14.bb | 2 +- meta/classes/compat.bbclass | 40 +++++++++ meta/classes/debianize.bbclass | 2 +- meta/classes/dpkg-base.bbclass | 1 + meta/classes/image.bbclass | 4 +- meta/classes/multiarch.bbclass | 88 +++++++++++++++++++ meta/classes/native.bbclass | 10 +++ meta/classes/sdk.bbclass | 2 +- meta/conf/bitbake.conf | 1 + .../isar-bootstrap/isar-bootstrap.inc | 2 + .../sbuild-chroot/sbuild-chroot.inc | 14 +-- 11 files changed, 156 insertions(+), 10 deletions(-) create mode 100644 meta/classes/compat.bbclass create mode 100644 meta/classes/multiarch.bbclass create mode 100644 meta/classes/native.bbclass diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb index 5e36a2ac..c53c9445 100644 --- a/meta-isar/recipes-app/samefile/samefile_2.14.bb +++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb @@ -21,7 +21,7 @@ do_prepare_build() { # deb_debianize. Pre-exisiting files will not be recreated, changelog # will be prepended unless its latest entry is for CHANGELOG_V. cat << EOF > ${WORKDIR}/changelog -${PN} (0.1) unstable; urgency=low +${BPN} (0.1) unstable; urgency=low * a long long time ago there was an early version diff --git a/meta/classes/compat.bbclass b/meta/classes/compat.bbclass new file mode 100644 index 00000000..1ca3e960 --- /dev/null +++ b/meta/classes/compat.bbclass @@ -0,0 +1,40 @@ +# This software is a part of ISAR. +# Copyright (C) 2023 Siemens AG +# +# SPDX-License-Identifier: MIT + +# this class is "dual-use": it can be inherited (e.g., by bootstrap and image +# classes) to access variables and functions, and it's also added via BBCLASSEXTEND +# when inheriting multiconfig.bbclass. + +################################################################################ +# generic functions +################################################################################ + +# calculate COMPAT_DISTRO_ARCH and ISAR_ENABLE_COMPAT_ARCH +# this must always use the DISTRO_ARCH override (not PACKAGE_ARCH), so needs +# to happen in a modified environment +python() { + distro_arch = d.getVar('DISTRO_ARCH', True) + package_arch = d.getVar('PACKAGE_ARCH', True) + overrides = d.getVar('OVERRIDES', True).split(':') + + localdata = bb.data.createCopy(d) + new_overrides = [distro_arch] + [o for o in overrides if not o == package_arch] + localdata.setVar('OVERRIDES', ':'.join(new_overrides)) + isar_enable_compat_arch = localdata.getVar('ISAR_ENABLE_COMPAT_ARCH', True) + compat_distro_arch = localdata.getVar('COMPAT_DISTRO_ARCH', True) + + d.setVar('COMPAT_DISTRO_ARCH', compat_distro_arch) + d.setVar('ISAR_ENABLE_COMPAT_ARCH', isar_enable_compat_arch) +} + +def isar_can_build_compat(d): + return (d.getVar('COMPAT_DISTRO_ARCH', True) is not None and + d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == '1') + +################################################################################ +# package recipe modifications when building *-compat: +################################################################################ + +PACKAGE_ARCH:class-compat = "${COMPAT_DISTRO_ARCH}" diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass index a6694a00..1b98c02d 100644 --- a/meta/classes/debianize.bbclass +++ b/meta/classes/debianize.bbclass @@ -31,7 +31,7 @@ deb_add_changelog() { date=$(LANG=C date -R -d @${timestamp}) cat <<EOF > ${S}/debian/changelog -${PN} (${changelog_v}) UNRELEASED; urgency=low +${BPN} (${changelog_v}) UNRELEASED; urgency=low * generated by Isar diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index ad28f7b3..55cc6655 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -5,6 +5,7 @@ # SPDX-License-Identifier: MIT inherit sbuild +inherit multiarch inherit debianize inherit terminal inherit repository diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index ef7d5a2a..ce7c549c 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -79,9 +79,11 @@ image_do_mounts() { buildchroot_do_mounts } +inherit multiarch + ROOTFSDIR = "${IMAGE_ROOTFS}" ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache" -ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" +ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${@isar_multiarch_packages('IMAGE_INSTALL', d)}" ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}" diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass new file mode 100644 index 00000000..21a4f610 --- /dev/null +++ b/meta/classes/multiarch.bbclass @@ -0,0 +1,88 @@ +# This software is a part of ISAR. +# Copyright (C) 2021-2022 Siemens AG +# +# SPDX-License-Identifier: MIT + +BPN = "${PN}" + +inherit compat +python() { + # provide compat only when we can build it + if isar_can_build_compat(d): + d.appendVar('BBCLASSEXTEND', ' compat') + + # build native separately only when it differs from the target variant + if d.getVar('HOST_ARCH', True) == d.getVar('DISTRO_ARCH', True): + pn = d.getVar('PN', True) + if not pn.endswith('-native') and not pn.endswith('-compat'): + provides = (d.getVar('PROVIDES', True) or '').split() + for p in provides: + d.appendVar('PROVIDES', f' {p}-native') + d.appendVar('PROVIDES', f' {pn}-native') + else: + d.appendVar('BBCLASSEXTEND', ' native') +} + +python multiarch_virtclass_handler() { + # In compat/native builds, ${PN} includes the -compat/-native suffix, + # so recipe-writers need to be careful when using it. Most of the time, + # they probably want to use ${BPN}, and in general, it's their responsibility + # to do so. If they don't, then it's ok for the build of the compat/native + # variant to fail. However, some variables are evaluated at parse time, + # and this will break the recipe even when compat/native is not requested. + # e.g., SRC_URI="file://${PN}" will try to checksum the local file at + # parse time, and parsing always happens for all build variants. So in those + # few variables, we automatically replace ${PN} with ${BPN}. + def fixup_pn_in_vars(d): + vars = 'SRC_URI FILESPATH'.split() + for var in vars: + v = d.getVar(var, False) + if v is not None: + d.setVar(var, v.replace('${PN}', '${BPN}')) + + # When building compat/native, the corresponding suffix needs to be + # propagated to all bitbake dependency definitions. + def fixup_depends(suffix, d): + vars = 'PROVIDES RPROVIDES DEPENDS RDEPENDS'.split() + for var in vars: + multiarch_var = [] + val = d.getVar(var, True) + if val is None: + continue + for v in val.split(): + if v.endswith('-compat') or v.endswith('-native'): + multiarch_var.append(v) + else: + multiarch_var.append(v + suffix) + d.setVar(var, ' '.join(multiarch_var)) + + pn = e.data.getVar('PN') + if pn.endswith('-compat'): + e.data.setVar('BPN', pn[:-len('-compat')]) + e.data.appendVar('OVERRIDES', ':class-compat') + fixup_pn_in_vars(e.data) + fixup_depends('-compat', e.data) + elif pn.endswith('-native'): + e.data.setVar('BPN', pn[:-len('-native')]) + e.data.appendVar('OVERRIDES', ':class-native') + fixup_pn_in_vars(e.data) + fixup_depends('-native', e.data) +} +addhandler multiarch_virtclass_handler +multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise" + +# function to convert bitbake targets to installable debian packages, +# e.g., "hello-compat" to "hello:i386". +def isar_multiarch_packages(var, d): + bb_targets = (d.getVar(var, True) or '').split() + packages = [] + compat_distro_arch = d.getVar('COMPAT_DISTRO_ARCH', True) + host_arch = d.getVar('HOST_ARCH', True) + for t in bb_targets: + if t.endswith('-compat') and compat_distro_arch is not None: + packages.append(t[:-len('-compat')] + ':' + compat_distro_arch) + elif t.endswith('-native'): + packages.append(t[:-len('-native')] + ':' + host_arch) + else: + packages.append(t) + return ' '.join(packages) diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass new file mode 100644 index 00000000..d2581ac1 --- /dev/null +++ b/meta/classes/native.bbclass @@ -0,0 +1,10 @@ +# This software is a part of ISAR. +# Copyright (C) 2023 Siemens AG +# +# SPDX-License-Identifier: MIT + +################################################################################ +# package recipe modifications when building *-native: +################################################################################ + +PACKAGE_ARCH:class-native = "${HOST_ARCH}" diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass index 0a98ea04..bfd47d0e 100644 --- a/meta/classes/sdk.bbclass +++ b/meta/classes/sdk.bbclass @@ -58,7 +58,7 @@ python __anonymous() { # rootfs/image overrides for the SDK ROOTFS_ARCH:class-sdk = "${HOST_ARCH}" ROOTFS_DISTRO:class-sdk = "${HOST_DISTRO}" -ROOTFS_PACKAGES:class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${SDK_INSTALL}" +ROOTFS_PACKAGES:class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${@isar_multiarch_packages('SDK_INSTALL', d)}" ROOTFS_FEATURES:append:class-sdk = " clean-package-cache generate-manifest export-dpkg-status" ROOTFS_MANIFEST_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}" ROOTFS_DPKGSTATUS_DEPLOY_DIR:class-sdk = "${DEPLOY_DIR_SDKCHROOT}" diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 05ccb7b8..dd21319a 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -81,6 +81,7 @@ QEMU_ARCH:riscv64 = "riscv64" # Codename of the repository created by the caching class DEBDISTRONAME ?= "isar" +NATIVELSBSTRING ?= "isarnative" # Strings used in sstate signature files TARGET_VENDOR = "" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 99d75e21..21a2d92f 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -274,6 +274,8 @@ do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" +inherit compat + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc index fb061dac..fd8bb648 100644 --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc @@ -9,6 +9,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260 PV = "1.0" inherit rootfs +inherit compat python() { distro_gcc = d.getVar('DISTRO_GCC') @@ -20,18 +21,19 @@ python() { d.appendVar('SBUILD_CHROOT_PREINSTALL_COMMON', ' libstdc++-{}-dev:{}'.format(distro_gcc, distro_arch)) - if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1': - compat_arch = d.getVar('COMPAT_DISTRO_ARCH') + if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1': + compat_arch = d.getVar('COMPAT_DISTRO_ARCH') + d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', + ' libc6-dev:{}'.format(compat_arch)) + d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', + ' crossbuild-essential-{}'.format(compat_arch)) + if d.getVar('DISTRO_GCC'): d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', ' libgcc-{}-dev:{}'.format(distro_gcc, compat_arch)) d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL', ' libstdc++-{}-dev:{}'.format(distro_gcc, compat_arch)) } -SBUILD_CHROOT_COMPAT_PREINSTALL:compat-arch = " \ - libc6-dev:${COMPAT_DISTRO_ARCH} \ - crossbuild-essential-${COMPAT_DISTRO_ARCH}" - SBUILD_CHROOT_PREINSTALL_COMMON = " \ ${SBUILD_CHROOT_COMPAT_PREINSTALL} \ libc6-dev:${DISTRO_ARCH} \ -- 2.30.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 3/4] remove obsolete compat-arch override 2023-02-24 14:24 [PATCH v3 0/4] multiarch support Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 2/4] add multiarch support Adriaan Schmidt @ 2023-02-24 14:24 ` Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 4/4] doc: add compat/native targets to user manual Adriaan Schmidt 2023-03-03 7:31 ` [PATCH v3 0/4] multiarch support Uladzimir Bely 4 siblings, 0 replies; 8+ messages in thread From: Adriaan Schmidt @ 2023-02-24 14:24 UTC (permalink / raw) To: isar-users; +Cc: Adriaan Schmidt The compat-arch override is no longer needed, as PACKAGE_ARCH is controlled by DEPENDing on <package>-compat. Also change the compat test: adding the compat package now happens via IMAGE_INSTALL in the config. Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> --- meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 --- meta-isar/recipes-app/libhello/libhello.bb | 3 --- meta/conf/bitbake.conf | 3 +-- testsuite/cibuilder.py | 4 ++++ 4 files changed, 5 insertions(+), 8 deletions(-) diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb b/meta-isar/recipes-app/hello-isar/hello-isar.bb index 39ddecb9..7d9f8322 100644 --- a/meta-isar/recipes-app/hello-isar/hello-isar.bb +++ b/meta-isar/recipes-app/hello-isar/hello-isar.bb @@ -20,7 +20,4 @@ SRC_URI = " \ file://yet-another-change.txt;apply=yes;striplevel=0" SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" -# NOTE: This is just to test 32-bit building on 64-bit archs. -PACKAGE_ARCH:compat-arch = "${COMPAT_DISTRO_ARCH}" - inherit dpkg diff --git a/meta-isar/recipes-app/libhello/libhello.bb b/meta-isar/recipes-app/libhello/libhello.bb index 3770fdb4..8b10842f 100644 --- a/meta-isar/recipes-app/libhello/libhello.bb +++ b/meta-isar/recipes-app/libhello/libhello.bb @@ -13,7 +13,4 @@ PV = "0.1-98f2e41" SRC_URI = "git://github.com/ilbers/libhello.git;protocol=https;branch=master;destsuffix=${P}" SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" -# NOTE: This is just to test 32-bit building on 64-bit archs. -PACKAGE_ARCH:compat-arch = "${COMPAT_DISTRO_ARCH}" - inherit dpkg diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index dd21319a..0c79a5b6 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -67,9 +67,8 @@ KERNEL_FILE:mipsel ?= "vmlinux" KERNEL_FILE:riscv64 ?= "vmlinux" KERNEL_FILE:arm64 ?= "vmlinux" -OVERRIDES = "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable" +OVERRIDES = "${PACKAGE_ARCH}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable" FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}" -COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}" # Setting default QEMU_ARCH variables for different DISTRO_ARCH: QEMU_ARCH:amd64 = "x86_64" diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py index 72522f4d..89d01531 100755 --- a/testsuite/cibuilder.py +++ b/testsuite/cibuilder.py @@ -96,7 +96,11 @@ class CIBuilder(Test): with open(self.build_dir + '/conf/ci_build.conf', 'w') as f: if compat_arch: f.write('ISAR_ENABLE_COMPAT_ARCH:amd64 = "1"\n') + f.write('IMAGE_INSTALL:remove:amd64 = "hello-isar"\n') + f.write('IMAGE_INSTALL:append:amd64 = " hello-isar-compat"\n') f.write('ISAR_ENABLE_COMPAT_ARCH:arm64 = "1"\n') + f.write('IMAGE_INSTALL:remove:arm64 = "hello-isar"\n') + f.write('IMAGE_INSTALL:append:arm64 = " hello-isar-compat"\n') f.write('IMAGE_INSTALL += "kselftest"\n') if cross: f.write('ISAR_CROSS_COMPILE = "1"\n') -- 2.30.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 4/4] doc: add compat/native targets to user manual 2023-02-24 14:24 [PATCH v3 0/4] multiarch support Adriaan Schmidt ` (2 preceding siblings ...) 2023-02-24 14:24 ` [PATCH v3 3/4] remove obsolete compat-arch override Adriaan Schmidt @ 2023-02-24 14:24 ` Adriaan Schmidt 2023-03-03 7:31 ` [PATCH v3 0/4] multiarch support Uladzimir Bely 4 siblings, 0 replies; 8+ messages in thread From: Adriaan Schmidt @ 2023-02-24 14:24 UTC (permalink / raw) To: isar-users; +Cc: Adriaan Schmidt Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> --- doc/user_manual.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/doc/user_manual.md b/doc/user_manual.md index b9a0bb64..3db5eb26 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -963,18 +963,24 @@ Debian cross-compilation works out of the box. Currently the following build con Experimental support for riscv64 is available as well. -### Cross-building for a compat architecture +### Building for a compat and/or native architecture Some architectures, under Isar amd64 and arm64 so far, support running 32-bit legacy applications on 64-bit kernels. Debian supports this via the multiarch concept. Isar can build 32-bit packages as part of a 64-bit image build and also enable -the image with the necessary packages. To activate the compat mode of a build, -set `ISAR_ENABLE_COMPAT_ARCH = "1"` in `local.conf`. Packages that shall be -built for the compat arch need to be tagged individually by setting -`PACKAGE_ARCH = "${COMPAT_DISTRO_ARCH}"` in the package recipe. Non-tagged -packages will continue to be built for the primary target architecture. +the image with the necessary packages. To activate compat support, +set `ISAR_ENABLE_COMPAT_ARCH = "1"` in `local.conf`. This will install neccessary +build dependencies in the buildchroot. + +For all dpkg package recipes, Isar automatically provides a `<package>-compat` +target that builds the package for the `COMPAT_DISTRO_ARCH`. This can be +referenced using the `DEPENDS` and `IMAGE_INSTALL` variables. + +To explicitly build a package for the build host architecture (in cross build +scenarios, or when generating an SDK), Isar automatically provides a +`<package>-native` target for all dpkg package recipes. ### Cross Support for Imagers @@ -987,7 +993,6 @@ In case your setup does not support cross-imaging, you can disable this just for the particular image by adding `ISAR_CROSS_COMPILE = "0"` to your image recipe. - ## Examining and debugging package generation inside their buildchroot Just like OpenEmbedded, Isar supports a devshell target for all dpkg package -- 2.30.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3 0/4] multiarch support 2023-02-24 14:24 [PATCH v3 0/4] multiarch support Adriaan Schmidt ` (3 preceding siblings ...) 2023-02-24 14:24 ` [PATCH v3 4/4] doc: add compat/native targets to user manual Adriaan Schmidt @ 2023-03-03 7:31 ` Uladzimir Bely 2023-03-03 7:39 ` Jan Kiszka 4 siblings, 1 reply; 8+ messages in thread From: Uladzimir Bely @ 2023-03-03 7:31 UTC (permalink / raw) To: isar-users, jan.kiszka; +Cc: Adriaan Schmidt In mail from Friday, 24 February 2023 17:24:28 +03 user Adriaan Schmidt wrote: > This adds `<package>-compat` and `<package>-native` bitbake > targets to all recipes inheriting dpkg-base. > > The new -compat build variant replaces the old compat mechanism. > Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to > ensure that the bootstrap and buildchroot is prepared correctly. > > Regarding testing of the new features: > - we have (limited) testing of compat, by adding `hello-isar-compat` to > amd64 and arm64 builds. > - currently no testing of native. There are two main use cases for the > feature: - SDK that needs `<package>-native`. This could be turned into a > test easily, by setting/appending SDK_INSTALL. Would require a new option > in testsuite, similar to `image_install` introduced in 57a0ade9a. - A build > tool that is provided by a recipe, when using it in cross compilation. E.g. > if someone needed a patched cmake, they would write cmake.bb, and > applications that need it would (in case cross compilation is enabled) > DEPEND+="cmake-native". Or, another real-world example from a downstream > layer: packaging applications with goreleaser, which is not found in the > Debian apt repos. Instead, I'm fetching it via a dpkg-prebuilt recipe, > written to support multiple architectures (architecture is part of the > download URL). Now if I want to cross compile go applications, they > DEPEND+="goreleaser-native", and DEBIAN_BUILD_DEPENDS_append=", > goreleaser:native". > These cases are more complex, but maybe we can find an example along > those lines and add it to meta-isar. > > As testing of the native feature would (in the simple case) need an > extension of the test classes (for SDK_INSTALL), or introduce something > completely new into meta-isar, I'd like to do that after discussion, in > a later series. > > Adriaan > > changes since v2: > - fixed a bug that completely broke things for targets without a compat > arch (e.g., i386). The compat variant of packages is now only available > when it can actually be built. > - the native variant is only generated if it differs from the target. > If DISTRO_ARCH==HOST_ARCH, then `<package>-native` is automatically > provided by the target package. > - also do the bitbake-target->debian-package transformation on SDK_INSTALL > - fix compat packages in testsuite: when we add hello-isar-compat, we > need to remove hello-isar. > > changes since v1: > - fixed an issue that prevented arch overrides of > ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite > - added `-native` expansion to contents of IMAGE_INSTALL > - documentation in user_manual > > > Adriaan Schmidt (4): > bitbake.conf: use PACKAGE_ARCH in overrides > add multiarch support > remove obsolete compat-arch override > doc: add compat/native targets to user manual > > doc/user_manual.md | 19 ++-- > .../recipes-app/hello-isar/hello-isar.bb | 3 - > meta-isar/recipes-app/libhello/libhello.bb | 3 - > .../recipes-app/samefile/samefile_2.14.bb | 2 +- > meta/classes/compat.bbclass | 40 +++++++++ > meta/classes/debianize.bbclass | 2 +- > meta/classes/dpkg-base.bbclass | 1 + > meta/classes/image.bbclass | 4 +- > meta/classes/multiarch.bbclass | 88 +++++++++++++++++++ > meta/classes/native.bbclass | 10 +++ > meta/classes/sdk.bbclass | 2 +- > meta/conf/bitbake.conf | 6 +- > .../isar-bootstrap/isar-bootstrap.inc | 2 + > .../sbuild-chroot/sbuild-chroot.inc | 14 +-- > testsuite/cibuilder.py | 4 + > 15 files changed, 174 insertions(+), 26 deletions(-) > create mode 100644 meta/classes/compat.bbclass > create mode 100644 meta/classes/multiarch.bbclass > create mode 100644 meta/classes/native.bbclass The patchset passes CI (there was a delay in testing due to internal problems with it) and is ready for merge. I'd like just to remind that Jan requested mentioning changes in RECIPE-API- CHANGELOG, not only in user_manual. If this point is still valid, we need v4; if not, we could proceed with patchset merge. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3 0/4] multiarch support 2023-03-03 7:31 ` [PATCH v3 0/4] multiarch support Uladzimir Bely @ 2023-03-03 7:39 ` Jan Kiszka 2023-03-03 8:10 ` Schmidt, Adriaan 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2023-03-03 7:39 UTC (permalink / raw) To: Uladzimir Bely, isar-users; +Cc: Adriaan Schmidt On 03.03.23 08:31, Uladzimir Bely wrote: > In mail from Friday, 24 February 2023 17:24:28 +03 user Adriaan Schmidt wrote: >> This adds `<package>-compat` and `<package>-native` bitbake >> targets to all recipes inheriting dpkg-base. >> >> The new -compat build variant replaces the old compat mechanism. >> Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to >> ensure that the bootstrap and buildchroot is prepared correctly. >> >> Regarding testing of the new features: >> - we have (limited) testing of compat, by adding `hello-isar-compat` to >> amd64 and arm64 builds. >> - currently no testing of native. There are two main use cases for the >> feature: - SDK that needs `<package>-native`. This could be turned into a >> test easily, by setting/appending SDK_INSTALL. Would require a new option >> in testsuite, similar to `image_install` introduced in 57a0ade9a. - A build >> tool that is provided by a recipe, when using it in cross compilation. E.g. >> if someone needed a patched cmake, they would write cmake.bb, and >> applications that need it would (in case cross compilation is enabled) >> DEPEND+="cmake-native". Or, another real-world example from a downstream >> layer: packaging applications with goreleaser, which is not found in the >> Debian apt repos. Instead, I'm fetching it via a dpkg-prebuilt recipe, >> written to support multiple architectures (architecture is part of the >> download URL). Now if I want to cross compile go applications, they >> DEPEND+="goreleaser-native", and DEBIAN_BUILD_DEPENDS_append=", >> goreleaser:native". >> These cases are more complex, but maybe we can find an example along >> those lines and add it to meta-isar. >> >> As testing of the native feature would (in the simple case) need an >> extension of the test classes (for SDK_INSTALL), or introduce something >> completely new into meta-isar, I'd like to do that after discussion, in >> a later series. >> >> Adriaan >> >> changes since v2: >> - fixed a bug that completely broke things for targets without a compat >> arch (e.g., i386). The compat variant of packages is now only available >> when it can actually be built. >> - the native variant is only generated if it differs from the target. >> If DISTRO_ARCH==HOST_ARCH, then `<package>-native` is automatically >> provided by the target package. >> - also do the bitbake-target->debian-package transformation on SDK_INSTALL >> - fix compat packages in testsuite: when we add hello-isar-compat, we >> need to remove hello-isar. >> >> changes since v1: >> - fixed an issue that prevented arch overrides of >> ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite >> - added `-native` expansion to contents of IMAGE_INSTALL >> - documentation in user_manual >> >> >> Adriaan Schmidt (4): >> bitbake.conf: use PACKAGE_ARCH in overrides >> add multiarch support >> remove obsolete compat-arch override >> doc: add compat/native targets to user manual >> >> doc/user_manual.md | 19 ++-- >> .../recipes-app/hello-isar/hello-isar.bb | 3 - >> meta-isar/recipes-app/libhello/libhello.bb | 3 - >> .../recipes-app/samefile/samefile_2.14.bb | 2 +- >> meta/classes/compat.bbclass | 40 +++++++++ >> meta/classes/debianize.bbclass | 2 +- >> meta/classes/dpkg-base.bbclass | 1 + >> meta/classes/image.bbclass | 4 +- >> meta/classes/multiarch.bbclass | 88 +++++++++++++++++++ >> meta/classes/native.bbclass | 10 +++ >> meta/classes/sdk.bbclass | 2 +- >> meta/conf/bitbake.conf | 6 +- >> .../isar-bootstrap/isar-bootstrap.inc | 2 + >> .../sbuild-chroot/sbuild-chroot.inc | 14 +-- >> testsuite/cibuilder.py | 4 + >> 15 files changed, 174 insertions(+), 26 deletions(-) >> create mode 100644 meta/classes/compat.bbclass >> create mode 100644 meta/classes/multiarch.bbclass >> create mode 100644 meta/classes/native.bbclass > > The patchset passes CI (there was a delay in testing due to internal problems > with it) and is ready for merge. > > I'd like just to remind that Jan requested mentioning changes in RECIPE-API- > CHANGELOG, not only in user_manual. > > If this point is still valid, we need v4; if not, we could proceed with > patchset merge. > The point remains valid due to patch 3, but I suppose we can quickly add a related patch for the changelog on top. Adriaan? Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH v3 0/4] multiarch support 2023-03-03 7:39 ` Jan Kiszka @ 2023-03-03 8:10 ` Schmidt, Adriaan 0 siblings, 0 replies; 8+ messages in thread From: Schmidt, Adriaan @ 2023-03-03 8:10 UTC (permalink / raw) To: Kiszka, Jan, Uladzimir Bely, isar-users Kiszka, Jan (T CED), Freitag, 3. März 2023 08:40: > On 03.03.23 08:31, Uladzimir Bely wrote: > > In mail from Friday, 24 February 2023 17:24:28 +03 user Adriaan Schmidt > wrote: > >> This adds `<package>-compat` and `<package>-native` bitbake > >> targets to all recipes inheriting dpkg-base. > >> > >> The new -compat build variant replaces the old compat mechanism. > >> Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to > >> ensure that the bootstrap and buildchroot is prepared correctly. > >> > >> Regarding testing of the new features: > >> - we have (limited) testing of compat, by adding `hello-isar-compat` to > >> amd64 and arm64 builds. > >> - currently no testing of native. There are two main use cases for the > >> feature: - SDK that needs `<package>-native`. This could be turned into a > >> test easily, by setting/appending SDK_INSTALL. Would require a new option > >> in testsuite, similar to `image_install` introduced in 57a0ade9a. - A > build > >> tool that is provided by a recipe, when using it in cross compilation. > E.g. > >> if someone needed a patched cmake, they would write cmake.bb, and > >> applications that need it would (in case cross compilation is enabled) > >> DEPEND+="cmake-native". Or, another real-world example from a downstream > >> layer: packaging applications with goreleaser, which is not found in the > >> Debian apt repos. Instead, I'm fetching it via a dpkg-prebuilt recipe, > >> written to support multiple architectures (architecture is part of the > >> download URL). Now if I want to cross compile go applications, they > >> DEPEND+="goreleaser-native", and DEBIAN_BUILD_DEPENDS_append=", > >> goreleaser:native". > >> These cases are more complex, but maybe we can find an example along > >> those lines and add it to meta-isar. > >> > >> As testing of the native feature would (in the simple case) need an > >> extension of the test classes (for SDK_INSTALL), or introduce something > >> completely new into meta-isar, I'd like to do that after discussion, in > >> a later series. > >> > >> Adriaan > >> > >> changes since v2: > >> - fixed a bug that completely broke things for targets without a compat > >> arch (e.g., i386). The compat variant of packages is now only available > >> when it can actually be built. > >> - the native variant is only generated if it differs from the target. > >> If DISTRO_ARCH==HOST_ARCH, then `<package>-native` is automatically > >> provided by the target package. > >> - also do the bitbake-target->debian-package transformation on SDK_INSTALL > >> - fix compat packages in testsuite: when we add hello-isar-compat, we > >> need to remove hello-isar. > >> > >> changes since v1: > >> - fixed an issue that prevented arch overrides of > >> ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite > >> - added `-native` expansion to contents of IMAGE_INSTALL > >> - documentation in user_manual > >> > >> > >> Adriaan Schmidt (4): > >> bitbake.conf: use PACKAGE_ARCH in overrides > >> add multiarch support > >> remove obsolete compat-arch override > >> doc: add compat/native targets to user manual > >> > >> doc/user_manual.md | 19 ++-- > >> .../recipes-app/hello-isar/hello-isar.bb | 3 - > >> meta-isar/recipes-app/libhello/libhello.bb | 3 - > >> .../recipes-app/samefile/samefile_2.14.bb | 2 +- > >> meta/classes/compat.bbclass | 40 +++++++++ > >> meta/classes/debianize.bbclass | 2 +- > >> meta/classes/dpkg-base.bbclass | 1 + > >> meta/classes/image.bbclass | 4 +- > >> meta/classes/multiarch.bbclass | 88 +++++++++++++++++++ > >> meta/classes/native.bbclass | 10 +++ > >> meta/classes/sdk.bbclass | 2 +- > >> meta/conf/bitbake.conf | 6 +- > >> .../isar-bootstrap/isar-bootstrap.inc | 2 + > >> .../sbuild-chroot/sbuild-chroot.inc | 14 +-- > >> testsuite/cibuilder.py | 4 + > >> 15 files changed, 174 insertions(+), 26 deletions(-) > >> create mode 100644 meta/classes/compat.bbclass > >> create mode 100644 meta/classes/multiarch.bbclass > >> create mode 100644 meta/classes/native.bbclass > > > > The patchset passes CI (there was a delay in testing due to internal > problems > > with it) and is ready for merge. > > > > I'd like just to remind that Jan requested mentioning changes in RECIPE- > API- > > CHANGELOG, not only in user_manual. > > > > If this point is still valid, we need v4; if not, we could proceed with > > patchset merge. > > > > The point remains valid due to patch 3, but I suppose we can quickly add > a related patch for the changelog on top. Adriaan? I'm already preparing v4 with cosmetic changes that came up in review (mainly dropping the expand=True in calls to getVar()). I will add a note to the recipe api changelog. Adriaan > Jan > > -- > Siemens AG, Technology > Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-03-03 8:10 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-02-24 14:24 [PATCH v3 0/4] multiarch support Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 2/4] add multiarch support Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 3/4] remove obsolete compat-arch override Adriaan Schmidt 2023-02-24 14:24 ` [PATCH v3 4/4] doc: add compat/native targets to user manual Adriaan Schmidt 2023-03-03 7:31 ` [PATCH v3 0/4] multiarch support Uladzimir Bely 2023-03-03 7:39 ` Jan Kiszka 2023-03-03 8:10 ` Schmidt, Adriaan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox