* [PATCH v2 0/4] multiarch support
@ 2023-02-20 9:05 Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt
` (4 more replies)
0 siblings, 5 replies; 8+ messages in thread
From: Adriaan Schmidt @ 2023-02-20 9:05 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
This adds `<package>-compat` and `<package>-native` bitbake
targets to all recipes inheriting dpkg-base.
The new -compat build variant replaces the old compat mechanism.
Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to
ensure that the bootstrap and buildchroot is prepared correctly.
Adriaan
changes since v1:
- fixed an issue that prevented arch overrides of
ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite
- added `-native` expansion to contents of IMAGE_INSTALL
- documentation in user_manual
Adriaan Schmidt (4):
bitbake.conf: use PACKAGE_ARCH in overrides
add multiarch support
remove obsolete compat-arch override
doc: add compat/native targets to user manual
doc/user_manual.md | 19 +++--
.../recipes-app/hello-isar/hello-isar.bb | 3 -
meta-isar/recipes-app/libhello/libhello.bb | 3 -
.../recipes-app/samefile/samefile_2.14.bb | 2 +-
meta/classes/compat.bbclass | 46 ++++++++++++
meta/classes/debianize.bbclass | 2 +-
meta/classes/dpkg-base.bbclass | 1 +
meta/classes/image.bbclass | 4 +-
meta/classes/multiarch.bbclass | 71 +++++++++++++++++++
meta/classes/native.bbclass | 10 +++
meta/conf/bitbake.conf | 6 +-
.../isar-bootstrap/isar-bootstrap.inc | 2 +
.../sbuild-chroot/sbuild-chroot.inc | 14 ++--
testsuite/cibuilder.py | 2 +
14 files changed, 160 insertions(+), 25 deletions(-)
create mode 100644 meta/classes/compat.bbclass
create mode 100644 meta/classes/multiarch.bbclass
create mode 100644 meta/classes/native.bbclass
--
2.30.2
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/4] bitbake.conf: use PACKAGE_ARCH in overrides
2023-02-20 9:05 [PATCH v2 0/4] multiarch support Adriaan Schmidt
@ 2023-02-20 9:05 ` Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 2/4] add multiarch support Adriaan Schmidt
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Adriaan Schmidt @ 2023-02-20 9:05 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
This replaces `DISTRO_ARCH` with `PACKAGE_ARCH` in the `OVERRIDES` list.
Note that `PACKAGE_ARCH` defaults to `DISTRO_ARCH`, so this only has an
effect when `PACKAGE_ARCH` is explicitly set in a recipe.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta/conf/bitbake.conf | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index c9f52a86..05ccb7b8 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -67,8 +67,8 @@ KERNEL_FILE:mipsel ?= "vmlinux"
KERNEL_FILE:riscv64 ?= "vmlinux"
KERNEL_FILE:arm64 ?= "vmlinux"
-OVERRIDES = "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
-FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
+OVERRIDES = "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
+FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
# Setting default QEMU_ARCH variables for different DISTRO_ARCH:
--
2.30.2
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 2/4] add multiarch support
2023-02-20 9:05 [PATCH v2 0/4] multiarch support Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt
@ 2023-02-20 9:05 ` Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 3/4] remove obsolete compat-arch override Adriaan Schmidt
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Adriaan Schmidt @ 2023-02-20 9:05 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
This adds support for building packages for native and compat architectures
to dpdk-base.bbclass.
Thus, all package recipes automatically have a *-native and *-compat target,
which can be used in DEPENDS/RDEPENDS definitions.
Additionally those targets can be used in IMAGE_INSTALL, where they
are automatically converted to install the correct debian package:
foo-compat -> foo:${COMPAT_DISTRO_ARCH}
foo-native -> foo:${HOST_ARCH}
Note that the switch ISAR_ENABLE_COMPAT_ARCH still exist and controls
addition of the compat architecture during bootstrapping.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
.../recipes-app/samefile/samefile_2.14.bb | 2 +-
meta/classes/compat.bbclass | 46 ++++++++++++++
meta/classes/debianize.bbclass | 2 +-
meta/classes/dpkg-base.bbclass | 1 +
meta/classes/image.bbclass | 4 +-
meta/classes/multiarch.bbclass | 60 +++++++++++++++++++
meta/classes/native.bbclass | 10 ++++
meta/conf/bitbake.conf | 1 +
8 files changed, 123 insertions(+), 3 deletions(-)
create mode 100644 meta/classes/compat.bbclass
create mode 100644 meta/classes/multiarch.bbclass
create mode 100644 meta/classes/native.bbclass
diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb
index 5e36a2ac..c53c9445 100644
--- a/meta-isar/recipes-app/samefile/samefile_2.14.bb
+++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb
@@ -21,7 +21,7 @@ do_prepare_build() {
# deb_debianize. Pre-exisiting files will not be recreated, changelog
# will be prepended unless its latest entry is for CHANGELOG_V.
cat << EOF > ${WORKDIR}/changelog
-${PN} (0.1) unstable; urgency=low
+${BPN} (0.1) unstable; urgency=low
* a long long time ago there was an early version
diff --git a/meta/classes/compat.bbclass b/meta/classes/compat.bbclass
new file mode 100644
index 00000000..2221bbf7
--- /dev/null
+++ b/meta/classes/compat.bbclass
@@ -0,0 +1,46 @@
+# This software is a part of ISAR.
+# Copyright (C) 2023 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+# this class is "dual-use": it can be inherited (e.g., by bootstrap and image
+# classes) to access variables and functions, and it's also addedd via BBCLASSEXTEND
+# when inheriting multiconfig.bbclass.
+
+################################################################################
+# generic functions
+################################################################################
+
+# calculate COMPAT_DISTRO_ARCH and ISAR_ENABLE_COMPAT_ARCH
+# this must always use the DISTRO_ARCH override (not PACKAGE_ARCH), so needs
+# to happen in a modified environment
+python() {
+ distro_arch = d.getVar('DISTRO_ARCH', True)
+ package_arch = d.getVar('PACKAGE_ARCH', True)
+ overrides = d.getVar('OVERRIDES', True).split(':')
+
+ localdata = bb.data.createCopy(d)
+ new_overrides = [distro_arch] + [o for o in overrides if not o == package_arch]
+ localdata.setVar('OVERRIDES', ':'.join(new_overrides))
+ isar_enable_compat_arch = localdata.getVar('ISAR_ENABLE_COMPAT_ARCH', True)
+ compat_distro_arch = localdata.getVar('COMPAT_DISTRO_ARCH', True)
+
+ if compat_distro_arch is None and 'class-compat' in d.getVar('OVERRIDES').split(':'):
+ bb.fatal(f"{distro_arch} does not have a compat arch")
+
+ d.setVar('COMPAT_DISTRO_ARCH', compat_distro_arch)
+ d.setVar('ISAR_ENABLE_COMPAT_ARCH', isar_enable_compat_arch)
+}
+
+
+################################################################################
+# package recipe modifications when actually building *-compat:
+################################################################################
+
+# check that we can actually build compat (ISAR_ENABLE_COMPAT_ARCH is set)
+python do_fetch:prepend:class-compat() {
+ if not d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == '1':
+ bb.fatal("compat package requested, but ISAR_ENABLE_COMPAT_ARCH is not set.")
+}
+
+PACKAGE_ARCH:class-compat = "${COMPAT_DISTRO_ARCH}"
diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index a6694a00..1b98c02d 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -31,7 +31,7 @@ deb_add_changelog() {
date=$(LANG=C date -R -d @${timestamp})
cat <<EOF > ${S}/debian/changelog
-${PN} (${changelog_v}) UNRELEASED; urgency=low
+${BPN} (${changelog_v}) UNRELEASED; urgency=low
* generated by Isar
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index ad28f7b3..55cc6655 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -5,6 +5,7 @@
# SPDX-License-Identifier: MIT
inherit sbuild
+inherit multiarch
inherit debianize
inherit terminal
inherit repository
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index ef7d5a2a..ce7c549c 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -79,9 +79,11 @@ image_do_mounts() {
buildchroot_do_mounts
}
+inherit multiarch
+
ROOTFSDIR = "${IMAGE_ROOTFS}"
ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache"
-ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
+ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${@isar_multiarch_packages('IMAGE_INSTALL', d)}"
ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
ROOTFS_DPKGSTATUS_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}"
ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass
new file mode 100644
index 00000000..09aaff52
--- /dev/null
+++ b/meta/classes/multiarch.bbclass
@@ -0,0 +1,60 @@
+# This software is a part of ISAR.
+# Copyright (C) 2021-2022 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+BBCLASSEXTEND += "native compat"
+BPN = "${PN}"
+
+python multiarch_virtclass_handler() {
+ def fixup_pn_in_vars(d):
+ vars = 'SRC_URI FILESPATH'.split()
+ for var in vars:
+ v = d.getVar(var, False)
+ if v is not None:
+ d.setVar(var, v.replace('${PN}', '${BPN}'))
+
+ def fixup_depends(suffix, d):
+ vars = 'PROVIDES RPROVIDES DEPENDS RDEPENDS'.split()
+ for var in vars:
+ multiarch_var = []
+ val = d.getVar(var, True)
+ if val is None:
+ continue
+ for v in val.split():
+ if v.endswith('-compat') or v.endswith('-native'):
+ multiarch_var.append(v)
+ else:
+ multiarch_var.append(v + suffix)
+ d.setVar(var, ' '.join(multiarch_var))
+
+ pn = e.data.getVar('PN')
+ if pn.endswith('-compat'):
+ e.data.setVar('BPN', pn[:-len('-compat')])
+ e.data.appendVar('OVERRIDES', ':class-compat')
+ fixup_pn_in_vars(e.data)
+ fixup_depends('-compat', e.data)
+ elif pn.endswith('-native'):
+ e.data.setVar('BPN', pn[:-len('-native')])
+ e.data.appendVar('OVERRIDES', ':class-native')
+ fixup_pn_in_vars(e.data)
+ fixup_depends('-native', e.data)
+}
+addhandler multiarch_virtclass_handler
+multiarch_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
+
+# function to convert bitbake targets to installable debian packages,
+# e.g., "hello-compat" to "hello:i386".
+def isar_multiarch_packages(var, d):
+ bb_targets = (d.getVar(var, True) or '').split()
+ packages = []
+ compat_distro_arch = d.getVar('COMPAT_DISTRO_ARCH', True)
+ host_arch = d.getVar('HOST_ARCH', True)
+ for t in bb_targets:
+ if t.endswith('-compat'):
+ packages.append(t[:-len('-compat')] + ':' + compat_distro_arch)
+ elif t.endswith('-native'):
+ packages.append(t[:-len('-native')] + ':' + host_arch)
+ else:
+ packages.append(t)
+ return ' '.join(packages)
diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
new file mode 100644
index 00000000..ef20e8bc
--- /dev/null
+++ b/meta/classes/native.bbclass
@@ -0,0 +1,10 @@
+# This software is a part of ISAR.
+# Copyright (C) 2023 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+################################################################################
+# package recipe modifications when actually building *-native:
+################################################################################
+
+PACKAGE_ARCH:class-native = "${HOST_ARCH}"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 05ccb7b8..dd21319a 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -81,6 +81,7 @@ QEMU_ARCH:riscv64 = "riscv64"
# Codename of the repository created by the caching class
DEBDISTRONAME ?= "isar"
+NATIVELSBSTRING ?= "isarnative"
# Strings used in sstate signature files
TARGET_VENDOR = ""
--
2.30.2
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 3/4] remove obsolete compat-arch override
2023-02-20 9:05 [PATCH v2 0/4] multiarch support Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 2/4] add multiarch support Adriaan Schmidt
@ 2023-02-20 9:05 ` Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 4/4] doc: add compat/native targets to user manual Adriaan Schmidt
2023-02-22 13:40 ` [PATCH v2 0/4] multiarch support Uladzimir Bely
4 siblings, 0 replies; 8+ messages in thread
From: Adriaan Schmidt @ 2023-02-20 9:05 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
The compat-arch override is no longer needed, as PACKAGE_ARCH is controlled
by DEPENDing on <package>-compat.
Also change the compat test: adding the compat package now happens via
IMAGE_INSTALL in the config.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 ---
meta-isar/recipes-app/libhello/libhello.bb | 3 ---
meta/classes/multiarch.bbclass | 11 +++++++++++
meta/conf/bitbake.conf | 3 +--
.../recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 ++
.../sbuild-chroot/sbuild-chroot.inc | 14 ++++++++------
testsuite/cibuilder.py | 2 ++
7 files changed, 24 insertions(+), 14 deletions(-)
diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb b/meta-isar/recipes-app/hello-isar/hello-isar.bb
index 39ddecb9..7d9f8322 100644
--- a/meta-isar/recipes-app/hello-isar/hello-isar.bb
+++ b/meta-isar/recipes-app/hello-isar/hello-isar.bb
@@ -20,7 +20,4 @@ SRC_URI = " \
file://yet-another-change.txt;apply=yes;striplevel=0"
SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d"
-# NOTE: This is just to test 32-bit building on 64-bit archs.
-PACKAGE_ARCH:compat-arch = "${COMPAT_DISTRO_ARCH}"
-
inherit dpkg
diff --git a/meta-isar/recipes-app/libhello/libhello.bb b/meta-isar/recipes-app/libhello/libhello.bb
index 3770fdb4..8b10842f 100644
--- a/meta-isar/recipes-app/libhello/libhello.bb
+++ b/meta-isar/recipes-app/libhello/libhello.bb
@@ -13,7 +13,4 @@ PV = "0.1-98f2e41"
SRC_URI = "git://github.com/ilbers/libhello.git;protocol=https;branch=master;destsuffix=${P}"
SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93"
-# NOTE: This is just to test 32-bit building on 64-bit archs.
-PACKAGE_ARCH:compat-arch = "${COMPAT_DISTRO_ARCH}"
-
inherit dpkg
diff --git a/meta/classes/multiarch.bbclass b/meta/classes/multiarch.bbclass
index 09aaff52..791f8e0d 100644
--- a/meta/classes/multiarch.bbclass
+++ b/meta/classes/multiarch.bbclass
@@ -7,6 +7,15 @@ BBCLASSEXTEND += "native compat"
BPN = "${PN}"
python multiarch_virtclass_handler() {
+ # In compat/native builds, ${PN} includes the -compat/-native suffix,
+ # so recipe-writers need to be careful when using it. Most of the time,
+ # they probably want to use ${BPN}, and in general, it's their responsibility
+ # to do so. If they don't, then it's ok for the build of the compat/native
+ # variant to fail. However, some variables are evaluated at parse time,
+ # and this will break the recipe even when compat/native is not requested.
+ # e.g., SRC_URI="file://${PN}" will try to checksum the local file at
+ # parse time, and parsing always happens for all build variants. So in those
+ # few variables, we automatically replace ${PN} with ${BPN}.
def fixup_pn_in_vars(d):
vars = 'SRC_URI FILESPATH'.split()
for var in vars:
@@ -14,6 +23,8 @@ python multiarch_virtclass_handler() {
if v is not None:
d.setVar(var, v.replace('${PN}', '${BPN}'))
+ # When building compat/native, the corresponding suffix needs to be
+ # propagated to all bitbake dependency definitions.
def fixup_depends(suffix, d):
vars = 'PROVIDES RPROVIDES DEPENDS RDEPENDS'.split()
for var in vars:
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index dd21319a..0c79a5b6 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -67,9 +67,8 @@ KERNEL_FILE:mipsel ?= "vmlinux"
KERNEL_FILE:riscv64 ?= "vmlinux"
KERNEL_FILE:arm64 ?= "vmlinux"
-OVERRIDES = "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
+OVERRIDES = "${PACKAGE_ARCH}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
-COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
# Setting default QEMU_ARCH variables for different DISTRO_ARCH:
QEMU_ARCH:amd64 = "x86_64"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 99d75e21..21a2d92f 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -274,6 +274,8 @@ do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}"
do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config"
do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}"
+inherit compat
+
do_bootstrap() {
if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then
if [ -z "${COMPAT_DISTRO_ARCH}" ]; then
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
index fb061dac..fd8bb648 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -9,6 +9,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
PV = "1.0"
inherit rootfs
+inherit compat
python() {
distro_gcc = d.getVar('DISTRO_GCC')
@@ -20,18 +21,19 @@ python() {
d.appendVar('SBUILD_CHROOT_PREINSTALL_COMMON',
' libstdc++-{}-dev:{}'.format(distro_gcc, distro_arch))
- if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1':
- compat_arch = d.getVar('COMPAT_DISTRO_ARCH')
+ if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1':
+ compat_arch = d.getVar('COMPAT_DISTRO_ARCH')
+ d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL',
+ ' libc6-dev:{}'.format(compat_arch))
+ d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL',
+ ' crossbuild-essential-{}'.format(compat_arch))
+ if d.getVar('DISTRO_GCC'):
d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL',
' libgcc-{}-dev:{}'.format(distro_gcc, compat_arch))
d.appendVar('SBUILD_CHROOT_COMPAT_PREINSTALL',
' libstdc++-{}-dev:{}'.format(distro_gcc, compat_arch))
}
-SBUILD_CHROOT_COMPAT_PREINSTALL:compat-arch = " \
- libc6-dev:${COMPAT_DISTRO_ARCH} \
- crossbuild-essential-${COMPAT_DISTRO_ARCH}"
-
SBUILD_CHROOT_PREINSTALL_COMMON = " \
${SBUILD_CHROOT_COMPAT_PREINSTALL} \
libc6-dev:${DISTRO_ARCH} \
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index 72522f4d..f0ea2702 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -96,7 +96,9 @@ class CIBuilder(Test):
with open(self.build_dir + '/conf/ci_build.conf', 'w') as f:
if compat_arch:
f.write('ISAR_ENABLE_COMPAT_ARCH:amd64 = "1"\n')
+ f.write('IMAGE_INSTALL:append:amd64 = " hello-isar-compat"\n')
f.write('ISAR_ENABLE_COMPAT_ARCH:arm64 = "1"\n')
+ f.write('IMAGE_INSTALL:append:arm64 = " hello-isar-compat"\n')
f.write('IMAGE_INSTALL += "kselftest"\n')
if cross:
f.write('ISAR_CROSS_COMPILE = "1"\n')
--
2.30.2
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 4/4] doc: add compat/native targets to user manual
2023-02-20 9:05 [PATCH v2 0/4] multiarch support Adriaan Schmidt
` (2 preceding siblings ...)
2023-02-20 9:05 ` [PATCH v2 3/4] remove obsolete compat-arch override Adriaan Schmidt
@ 2023-02-20 9:05 ` Adriaan Schmidt
2023-02-22 13:40 ` [PATCH v2 0/4] multiarch support Uladzimir Bely
4 siblings, 0 replies; 8+ messages in thread
From: Adriaan Schmidt @ 2023-02-20 9:05 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
doc/user_manual.md | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index b9a0bb64..3db5eb26 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -963,18 +963,24 @@ Debian cross-compilation works out of the box. Currently the following build con
Experimental support for riscv64 is available as well.
-### Cross-building for a compat architecture
+### Building for a compat and/or native architecture
Some architectures, under Isar amd64 and arm64 so far, support running 32-bit
legacy applications on 64-bit kernels. Debian supports this via the multiarch
concept.
Isar can build 32-bit packages as part of a 64-bit image build and also enable
-the image with the necessary packages. To activate the compat mode of a build,
-set `ISAR_ENABLE_COMPAT_ARCH = "1"` in `local.conf`. Packages that shall be
-built for the compat arch need to be tagged individually by setting
-`PACKAGE_ARCH = "${COMPAT_DISTRO_ARCH}"` in the package recipe. Non-tagged
-packages will continue to be built for the primary target architecture.
+the image with the necessary packages. To activate compat support,
+set `ISAR_ENABLE_COMPAT_ARCH = "1"` in `local.conf`. This will install neccessary
+build dependencies in the buildchroot.
+
+For all dpkg package recipes, Isar automatically provides a `<package>-compat`
+target that builds the package for the `COMPAT_DISTRO_ARCH`. This can be
+referenced using the `DEPENDS` and `IMAGE_INSTALL` variables.
+
+To explicitly build a package for the build host architecture (in cross build
+scenarios, or when generating an SDK), Isar automatically provides a
+`<package>-native` target for all dpkg package recipes.
### Cross Support for Imagers
@@ -987,7 +993,6 @@ In case your setup does not support cross-imaging, you can disable this
just for the particular image by adding `ISAR_CROSS_COMPILE = "0"` to your
image recipe.
-
## Examining and debugging package generation inside their buildchroot
Just like OpenEmbedded, Isar supports a devshell target for all dpkg package
--
2.30.2
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 0/4] multiarch support
2023-02-20 9:05 [PATCH v2 0/4] multiarch support Adriaan Schmidt
` (3 preceding siblings ...)
2023-02-20 9:05 ` [PATCH v2 4/4] doc: add compat/native targets to user manual Adriaan Schmidt
@ 2023-02-22 13:40 ` Uladzimir Bely
2023-02-22 17:27 ` Schmidt, Adriaan
4 siblings, 1 reply; 8+ messages in thread
From: Uladzimir Bely @ 2023-02-22 13:40 UTC (permalink / raw)
To: isar-users, Adriaan Schmidt
In the email from Monday, 20 February 2023 12:05:07 +03 user Adriaan Schmidt
wrote:
> This adds `<package>-compat` and `<package>-native` bitbake
> targets to all recipes inheriting dpkg-base.
>
> The new -compat build variant replaces the old compat mechanism.
> Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to
> ensure that the bootstrap and buildchroot is prepared correctly.
>
> Adriaan
>
> changes since v1:
> - fixed an issue that prevented arch overrides of
> ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite
> - added `-native` expansion to contents of IMAGE_INSTALL
> - documentation in user_manual
>
> Adriaan Schmidt (4):
> bitbake.conf: use PACKAGE_ARCH in overrides
> add multiarch support
> remove obsolete compat-arch override
> doc: add compat/native targets to user manual
>
> doc/user_manual.md | 19 +++--
> .../recipes-app/hello-isar/hello-isar.bb | 3 -
> meta-isar/recipes-app/libhello/libhello.bb | 3 -
> .../recipes-app/samefile/samefile_2.14.bb | 2 +-
> meta/classes/compat.bbclass | 46 ++++++++++++
> meta/classes/debianize.bbclass | 2 +-
> meta/classes/dpkg-base.bbclass | 1 +
> meta/classes/image.bbclass | 4 +-
> meta/classes/multiarch.bbclass | 71 +++++++++++++++++++
> meta/classes/native.bbclass | 10 +++
> meta/conf/bitbake.conf | 6 +-
> .../isar-bootstrap/isar-bootstrap.inc | 2 +
> .../sbuild-chroot/sbuild-chroot.inc | 14 ++--
> testsuite/cibuilder.py | 2 +
> 14 files changed, 160 insertions(+), 25 deletions(-)
> create mode 100644 meta/classes/compat.bbclass
> create mode 100644 meta/classes/multiarch.bbclass
> create mode 100644 meta/classes/native.bbclass
Didn't look too deeply into the problem, but CI completely fails with the
patchset, producing multiple errors like:
16:29:04 ERROR: /workspace/build/isar_ub_devel_2_fast/125/meta-isar/recipes-
app/cowsay/cowsay_git.bb: armhf does not have a compat arch
16:29:04 ERROR: /workspace/build/isar_ub_devel_2_fast/125/meta/recipes-
support/sshd-regen-keys/sshd-regen-keys_0.4.bb: i386 does not have a compat
arch
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH v2 0/4] multiarch support
2023-02-22 13:40 ` [PATCH v2 0/4] multiarch support Uladzimir Bely
@ 2023-02-22 17:27 ` Schmidt, Adriaan
2023-02-22 18:01 ` Uladzimir Bely
0 siblings, 1 reply; 8+ messages in thread
From: Schmidt, Adriaan @ 2023-02-22 17:27 UTC (permalink / raw)
To: Uladzimir Bely, isar-users
Uladzimir Bely <ubely@ilbers.de>, Mittwoch, 22. Februar 2023 14:41
> In the email from Monday, 20 February 2023 12:05:07 +03 user Adriaan Schmidt
> wrote:
> > This adds `<package>-compat` and `<package>-native` bitbake
> > targets to all recipes inheriting dpkg-base.
> >
> > The new -compat build variant replaces the old compat mechanism.
> > Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to
> > ensure that the bootstrap and buildchroot is prepared correctly.
> >
> > Adriaan
> >
> > changes since v1:
> > - fixed an issue that prevented arch overrides of
> > ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite
> > - added `-native` expansion to contents of IMAGE_INSTALL
> > - documentation in user_manual
> >
> > Adriaan Schmidt (4):
> > bitbake.conf: use PACKAGE_ARCH in overrides
> > add multiarch support
> > remove obsolete compat-arch override
> > doc: add compat/native targets to user manual
> >
> > doc/user_manual.md | 19 +++--
> > .../recipes-app/hello-isar/hello-isar.bb | 3 -
> > meta-isar/recipes-app/libhello/libhello.bb | 3 -
> > .../recipes-app/samefile/samefile_2.14.bb | 2 +-
> > meta/classes/compat.bbclass | 46 ++++++++++++
> > meta/classes/debianize.bbclass | 2 +-
> > meta/classes/dpkg-base.bbclass | 1 +
> > meta/classes/image.bbclass | 4 +-
> > meta/classes/multiarch.bbclass | 71 +++++++++++++++++++
> > meta/classes/native.bbclass | 10 +++
> > meta/conf/bitbake.conf | 6 +-
> > .../isar-bootstrap/isar-bootstrap.inc | 2 +
> > .../sbuild-chroot/sbuild-chroot.inc | 14 ++--
> > testsuite/cibuilder.py | 2 +
> > 14 files changed, 160 insertions(+), 25 deletions(-)
> > create mode 100644 meta/classes/compat.bbclass
> > create mode 100644 meta/classes/multiarch.bbclass
> > create mode 100644 meta/classes/native.bbclass
>
> Didn't look too deeply into the problem, but CI completely fails with the
> patchset, producing multiple errors like:
>
> 16:29:04 ERROR: /workspace/build/isar_ub_devel_2_fast/125/meta-isar/recipes-
> app/cowsay/cowsay_git.bb: armhf does not have a compat arch
> 16:29:04 ERROR: /workspace/build/isar_ub_devel_2_fast/125/meta/recipes-
> support/sshd-regen-keys/sshd-regen-keys_0.4.bb: i386 does not have a compat
> arch
Ah yes, of course I only tested the cases where a compat arch exists...
I looked into this, and this error (or the check that prints it) happens at
parse time. That's not good, because `*-compat` variants of recipes are
always parsed, whether someone wants to build them or not.
I have found two ways of solving this:
1. If compat cannot be built (either because ISAR_ENABLE_COMPAT_ARCH is not set,
or because there is no COMPAT_DISTRO_ARCH), don't provide the `*-compat` build targets.
Then a `bitbake hello-compat` would fail with:
===
builder@022f45995a49:/build$ bitbake hello-compat
Loading cache: 100% |####################################################################################| Time: 0:00:00
Loaded 113 entries from dependency cache.
Parsing recipes: 100% |##################################################################################| Time: 0:00:00
Parsing of 45 .bb files complete (6 cached, 39 parsed). 80 targets, 0 skipped, 0 masked, 0 errors.
ERROR: Nothing PROVIDES 'hello-compat'
===
2. Alternatively, I can check when actually starting the build. The best I could find
is prepending `do_fetch`, and then we would fail like this:
===
builder@022f45995a49:/build$ bitbake hello-compat
Loading cache: 100% |####################################################################################| Time: 0:00:00
Loaded 113 entries from dependency cache.
Parsing recipes: 100% |##################################################################################| Time: 0:00:00
Parsing of 45 .bb files complete (8 cached, 37 parsed). 113 targets, 0 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |###############################################################################| Time: 0:00:00
Sstate summary: Wanted 1 Local 0 Mirrors 0 Missed 1 Current 2 (0% match, 66% complete)
NOTE: Executing Tasks
ERROR: hello-compat-1.0-r0 do_fetch: i386 does not have a compat arch
ERROR: Logfile of failure stored in: /build/tmp/work/debian-bullseye-i386/hello-compat/1.0-r0/temp/log.do_fetch.83315
ERROR: Task (virtual:compat:/build/../repo/meta-isar/recipes-app/hello/hello.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 21 tasks of which 19 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
virtual:compat:/build/../repo/meta-isar/recipes-app/hello/hello.bb:do_fetch
Summary: There was 1 ERROR message, returning a non-zero exit code.
===
I strongly prefer (1.) because it fails earlier, before any tasks start running.
The downside is that we can't give a more detailed explanation why we're failing.
Any preferences?
Adriaan
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 0/4] multiarch support
2023-02-22 17:27 ` Schmidt, Adriaan
@ 2023-02-22 18:01 ` Uladzimir Bely
0 siblings, 0 replies; 8+ messages in thread
From: Uladzimir Bely @ 2023-02-22 18:01 UTC (permalink / raw)
To: isar-users, Schmidt, Adriaan
In the email from Wednesday, 22 February 2023 20:27:28 +03 user Schmidt, Adriaan wrote:
> Uladzimir Bely <ubely@ilbers.de>, Mittwoch, 22. Februar 2023 14:41
> > In the email from Monday, 20 February 2023 12:05:07 +03 user Adriaan Schmidt
> > wrote:
> > > This adds `<package>-compat` and `<package>-native` bitbake
> > > targets to all recipes inheriting dpkg-base.
> > >
> > > The new -compat build variant replaces the old compat mechanism.
> > > Note that `ISAR_ENABLE_COMPAT_ARCH="1"` is still required to
> > > ensure that the bootstrap and buildchroot is prepared correctly.
> > >
> > > Adriaan
> > >
> > > changes since v1:
> > > - fixed an issue that prevented arch overrides of
> > > ISAR_ENABLE_COMPAT_ARCH, which is used in testsuite
> > > - added `-native` expansion to contents of IMAGE_INSTALL
> > > - documentation in user_manual
> > >
> > > Adriaan Schmidt (4):
> > > bitbake.conf: use PACKAGE_ARCH in overrides
> > > add multiarch support
> > > remove obsolete compat-arch override
> > > doc: add compat/native targets to user manual
> > >
> > > doc/user_manual.md | 19 +++--
> > > .../recipes-app/hello-isar/hello-isar.bb | 3 -
> > > meta-isar/recipes-app/libhello/libhello.bb | 3 -
> > > .../recipes-app/samefile/samefile_2.14.bb | 2 +-
> > > meta/classes/compat.bbclass | 46 ++++++++++++
> > > meta/classes/debianize.bbclass | 2 +-
> > > meta/classes/dpkg-base.bbclass | 1 +
> > > meta/classes/image.bbclass | 4 +-
> > > meta/classes/multiarch.bbclass | 71 +++++++++++++++++++
> > > meta/classes/native.bbclass | 10 +++
> > > meta/conf/bitbake.conf | 6 +-
> > > .../isar-bootstrap/isar-bootstrap.inc | 2 +
> > > .../sbuild-chroot/sbuild-chroot.inc | 14 ++--
> > > testsuite/cibuilder.py | 2 +
> > > 14 files changed, 160 insertions(+), 25 deletions(-)
> > > create mode 100644 meta/classes/compat.bbclass
> > > create mode 100644 meta/classes/multiarch.bbclass
> > > create mode 100644 meta/classes/native.bbclass
> >
> > Didn't look too deeply into the problem, but CI completely fails with the
> > patchset, producing multiple errors like:
> >
> > 16:29:04 ERROR: /workspace/build/isar_ub_devel_2_fast/125/meta-isar/recipes-
> > app/cowsay/cowsay_git.bb: armhf does not have a compat arch
> > 16:29:04 ERROR: /workspace/build/isar_ub_devel_2_fast/125/meta/recipes-
> > support/sshd-regen-keys/sshd-regen-keys_0.4.bb: i386 does not have a compat
> > arch
>
> Ah yes, of course I only tested the cases where a compat arch exists...
>
> I looked into this, and this error (or the check that prints it) happens at
> parse time. That's not good, because `*-compat` variants of recipes are
> always parsed, whether someone wants to build them or not.
>
> I have found two ways of solving this:
>
> 1. If compat cannot be built (either because ISAR_ENABLE_COMPAT_ARCH is not set,
> or because there is no COMPAT_DISTRO_ARCH), don't provide the `*-compat` build targets.
> Then a `bitbake hello-compat` would fail with:
> ===
> builder@022f45995a49:/build$ bitbake hello-compat
> Loading cache: 100% |####################################################################################| Time: 0:00:00
> Loaded 113 entries from dependency cache.
> Parsing recipes: 100% |##################################################################################| Time: 0:00:00
> Parsing of 45 .bb files complete (6 cached, 39 parsed). 80 targets, 0 skipped, 0 masked, 0 errors.
> ERROR: Nothing PROVIDES 'hello-compat'
> ===
>
> 2. Alternatively, I can check when actually starting the build. The best I could find
> is prepending `do_fetch`, and then we would fail like this:
> ===
> builder@022f45995a49:/build$ bitbake hello-compat
> Loading cache: 100% |####################################################################################| Time: 0:00:00
> Loaded 113 entries from dependency cache.
> Parsing recipes: 100% |##################################################################################| Time: 0:00:00
> Parsing of 45 .bb files complete (8 cached, 37 parsed). 113 targets, 0 skipped, 0 masked, 0 errors.
> NOTE: Resolving any missing task queue dependencies
> Initialising tasks: 100% |###############################################################################| Time: 0:00:00
> Sstate summary: Wanted 1 Local 0 Mirrors 0 Missed 1 Current 2 (0% match, 66% complete)
> NOTE: Executing Tasks
> ERROR: hello-compat-1.0-r0 do_fetch: i386 does not have a compat arch
> ERROR: Logfile of failure stored in: /build/tmp/work/debian-bullseye-i386/hello-compat/1.0-r0/temp/log.do_fetch.83315
> ERROR: Task (virtual:compat:/build/../repo/meta-isar/recipes-app/hello/hello.bb:do_fetch) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 21 tasks of which 19 didn't need to be rerun and 1 failed.
>
> Summary: 1 task failed:
> virtual:compat:/build/../repo/meta-isar/recipes-app/hello/hello.bb:do_fetch
> Summary: There was 1 ERROR message, returning a non-zero exit code.
> ===
>
> I strongly prefer (1.) because it fails earlier, before any tasks start running.
> The downside is that we can't give a more detailed explanation why we're failing.
>
> Any preferences?
>
> Adriaan
>
It seems (2.) also fails early enough, but we have to wait for debootstrap parallel tasks finished. So I'm for (1.).
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-02-22 18:01 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-20 9:05 [PATCH v2 0/4] multiarch support Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 1/4] bitbake.conf: use PACKAGE_ARCH in overrides Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 2/4] add multiarch support Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 3/4] remove obsolete compat-arch override Adriaan Schmidt
2023-02-20 9:05 ` [PATCH v2 4/4] doc: add compat/native targets to user manual Adriaan Schmidt
2023-02-22 13:40 ` [PATCH v2 0/4] multiarch support Uladzimir Bely
2023-02-22 17:27 ` Schmidt, Adriaan
2023-02-22 18:01 ` Uladzimir Bely
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox