* [PATCH 0/2] 32-bit compat arch support @ 2020-09-04 12:47 Jan Kiszka 2020-09-04 12:47 ` [PATCH 1/2] Add compat architecture support via multiarch Jan Kiszka 2020-09-04 12:47 ` [PATCH 2/2] ci: Add compat arch support Jan Kiszka 0 siblings, 2 replies; 8+ messages in thread From: Jan Kiszka @ 2020-09-04 12:47 UTC (permalink / raw) To: isar-users This allows to build and install 32-bit packages for/on amd64 and arm64 targets. SDK is enhanced as well. Jan Jan Kiszka (2): Add compat architecture support via multiarch ci: Add compat arch support meta-isar/conf/local.conf.sample | 5 +++++ meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ meta-isar/recipes-app/libhello/libhello.bb | 3 +++ meta/classes/dpkg.bbclass | 8 +++++--- meta/conf/bitbake.conf | 3 ++- meta/conf/distro/debian-common.conf | 3 +++ .../recipes-core/isar-bootstrap/isar-bootstrap.inc | 14 +++++++++++++- meta/recipes-devtools/buildchroot/buildchroot.inc | 5 +++++ meta/recipes-devtools/sdkchroot/sdkchroot.bb | 1 + scripts/ci_build.sh | 6 ++++++ 10 files changed, 46 insertions(+), 5 deletions(-) -- 2.26.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/2] Add compat architecture support via multiarch 2020-09-04 12:47 [PATCH 0/2] 32-bit compat arch support Jan Kiszka @ 2020-09-04 12:47 ` Jan Kiszka 2020-09-04 12:47 ` [PATCH 2/2] ci: Add compat arch support Jan Kiszka 1 sibling, 0 replies; 8+ messages in thread From: Jan Kiszka @ 2020-09-04 12:47 UTC (permalink / raw) To: isar-users From: Jan Kiszka <jan.kiszka@siemens.com> Allow building and installing 32-bit packages an 64-bit architectures that support this. Namely, these are amd64 and arm64 for us so far, minus stretch-amd64 which lacks crossbuild-essential-i386 in that version. Control the target architecture of a package is added via the PACKAGE_ARCH variable that defaults to DISTRO_ARCH. When set to COMPAT_DISTRO_ARCH, the package is built for i386 on amd64 and armhf on arm64. If the build environment and the target image should contain the compat arch is controlled via setting ISAR_ENABLE_COMPAT_ARCH to 1. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> --- meta-isar/conf/local.conf.sample | 5 +++++ meta/classes/dpkg.bbclass | 8 +++++--- meta/conf/bitbake.conf | 3 ++- meta/conf/distro/debian-common.conf | 3 +++ .../recipes-core/isar-bootstrap/isar-bootstrap.inc | 14 +++++++++++++- meta/recipes-devtools/buildchroot/buildchroot.inc | 5 +++++ meta/recipes-devtools/sdkchroot/sdkchroot.bb | 1 + 7 files changed, 34 insertions(+), 5 deletions(-) diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index e842c129..2f82020d 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -175,6 +175,11 @@ IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsc # Enable cross-compilation support ISAR_CROSS_COMPILE ?= "0" +# +# Uncomment to enable 32-bit compat architecture support +# NOTE: this works for amd64 and arm64 targets so far +#ISAR_ENABLE_COMPAT_ARCH ?= "1" + # # Uncomment this to enable use of cached base repository #ISAR_USE_CACHED_BASE_REPO ?= "1" diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index acccc1f2..a24386df 100644 --- a/meta/classes/dpkg.bbclass +++ b/meta/classes/dpkg.bbclass @@ -3,16 +3,18 @@ inherit dpkg-base +PACKAGE_ARCH ?= "${DISTRO_ARCH}" + # Install build dependencies for package do_install_builddeps() { dpkg_do_mounts E="${@ isar_export_proxies(d)}" deb_dl_dir_import "${BUILDCHROOT_DIR}" sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ - ${PP}/${PPS} ${DISTRO_ARCH} --download-only + ${PP}/${PPS} ${PACKAGE_ARCH} --download-only deb_dl_dir_export "${BUILDCHROOT_DIR}" sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ - ${PP}/${PPS} ${DISTRO_ARCH} + ${PP}/${PPS} ${PACKAGE_ARCH} dpkg_undo_mounts } @@ -27,5 +29,5 @@ dpkg_runbuild() { E="${@ isar_export_proxies(d)}" export PARALLEL_MAKE="${PARALLEL_MAKE}" sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ - /isar/build.sh ${PP}/${PPS} ${DISTRO_ARCH} + /isar/build.sh ${PP}/${PPS} ${PACKAGE_ARCH} } diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index d2834f53..3f136ca1 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -62,8 +62,9 @@ KERNEL_FILE ?= "vmlinuz" KERNEL_FILE_mipsel ?= "vmlinux" KERNEL_FILE_riscv64 ?= "vmlinux" -OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable" +OVERRIDES = "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:forcevariable" FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}" +COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}" # Setting default QEMU_ARCH variables for different DISTRO_ARCH: QEMU_ARCH_amd64 = "x86_64" diff --git a/meta/conf/distro/debian-common.conf b/meta/conf/distro/debian-common.conf index 7d681c49..582678f6 100644 --- a/meta/conf/distro/debian-common.conf +++ b/meta/conf/distro/debian-common.conf @@ -22,3 +22,6 @@ GRUB_BOOTLOADER_INSTALL_armhf = "grub-efi-arm-bin" GRUB_BOOTLOADER_INSTALL_arm64 = "grub-efi-arm64-bin" SYSLINUX_BOOTLOADER_INSTALL = "syslinux syslinux-common" + +COMPAT_DISTRO_ARCH_amd64 = "i386" +COMPAT_DISTRO_ARCH_arm64 = "armhf" diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index da0d436b..fbfe669d 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -222,7 +222,7 @@ def get_host_release(): rel = platform.release() return rel -do_bootstrap[vardeps] += "DISTRO_APT_PREMIRRORS" +do_bootstrap[vardeps] += "DISTRO_APT_PREMIRRORS ISAR_ENABLE_COMPAT_ARCH" do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" do_bootstrap[depends] = "base-apt:do_cache" @@ -236,6 +236,14 @@ isar_bootstrap() { esac shift done + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + if [ -z "${COMPAT_DISTRO_ARCH}" ]; then + bbfatal "${DISTRO_ARCH} does not have a compat arch" + fi + if [ "${@get_distro_suite(d, True)}-${COMPAT_DISTRO_ARCH}" = "stretch-i386" ]; then + bbfatal "compat arch build for stretch-i386 not supported" + fi + fi debootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" @@ -330,6 +338,10 @@ isar_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} fi + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} + fi + chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc index b4d7b764..835968de 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot.inc +++ b/meta/recipes-devtools/buildchroot/buildchroot.inc @@ -22,7 +22,12 @@ ROOTFS_CLEAN_FILES = "" ROOTFS_MANIFEST_DEPLOY_DIR = "${DEPLOY_DIR_BUILDCHROOT}" ROOTFS_FEATURES += "generate-manifest" +BUILDCHROOT_COMPAT_PREINSTALL_compat-arch = " \ + libc6:${COMPAT_DISTRO_ARCH} \ + crossbuild-essential-${COMPAT_DISTRO_ARCH}" + BUILDCHROOT_PREINSTALL_COMMON = " \ + ${BUILDCHROOT_COMPAT_PREINSTALL} \ make \ debhelper \ autotools-dev \ diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb index bf3f6fb4..2bf3f5bc 100644 --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb @@ -22,6 +22,7 @@ DEPENDS += "${SDK_INSTALL}" TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}" TOOLCHAIN_${HOST_ARCH} = "build-essential" TOOLCHAIN_i386 = "build-essential" +TOOLCHAIN_append_compat-arch = " crossbuild-essential-${COMPAT_DISTRO_ARCH}" inherit rootfs ROOTFS_ARCH = "${HOST_ARCH}" -- 2.26.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/2] ci: Add compat arch support 2020-09-04 12:47 [PATCH 0/2] 32-bit compat arch support Jan Kiszka 2020-09-04 12:47 ` [PATCH 1/2] Add compat architecture support via multiarch Jan Kiszka @ 2020-09-04 12:47 ` Jan Kiszka 2020-09-08 8:53 ` Jan Kiszka 1 sibling, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2020-09-04 12:47 UTC (permalink / raw) To: isar-users From: Jan Kiszka <jan.kiszka@siemens.com> Build hello-isar and libisar for the compat arch if that is enabled. Set ISAR_ENABLE_COMPAT_ARCH for all supported combinations in CI. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> --- meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ meta-isar/recipes-app/libhello/libhello.bb | 3 +++ scripts/ci_build.sh | 6 ++++++ 3 files changed, 12 insertions(+) diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb b/meta-isar/recipes-app/hello-isar/hello-isar.bb index 144a433e..8c3ba8b2 100644 --- a/meta-isar/recipes-app/hello-isar/hello-isar.bb +++ b/meta-isar/recipes-app/hello-isar/hello-isar.bb @@ -20,4 +20,7 @@ SRC_URI = " \ file://yet-another-change.txt;apply=yes;striplevel=0" SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" +# NOTE: This is just to test 32-bit building on 64-bit archs. +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" + inherit dpkg diff --git a/meta-isar/recipes-app/libhello/libhello.bb b/meta-isar/recipes-app/libhello/libhello.bb index ab271b58..5c44de50 100644 --- a/meta-isar/recipes-app/libhello/libhello.bb +++ b/meta-isar/recipes-app/libhello/libhello.bb @@ -13,4 +13,7 @@ PV = "0.1-98f2e41" SRC_URI = "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}" SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" +# NOTE: This is just to test 32-bit building on 64-bit archs. +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" + inherit dpkg diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index d2c707b8..461fd5cc 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -139,6 +139,12 @@ if [ ! -d "$BUILD_DIR" ]; then fi source isar-init-build-env "$BUILD_DIR" +cat >>conf/local.conf <<EOF +ISAR_ENABLE_COMPAT_ARCH_amd64 = "1" +ISAR_ENABLE_COMPAT_ARCH_arm64 = "1" +ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0" +EOF + if [ -n "$CROSS_BUILD" ]; then sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf fi -- 2.26.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] ci: Add compat arch support 2020-09-04 12:47 ` [PATCH 2/2] ci: Add compat arch support Jan Kiszka @ 2020-09-08 8:53 ` Jan Kiszka 2020-09-09 13:44 ` Henning Schild 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2020-09-08 8:53 UTC (permalink / raw) To: isar-users On 04.09.20 14:47, [ext] Jan Kiszka wrote: > From: Jan Kiszka <jan.kiszka@siemens.com> > > Build hello-isar and libisar for the compat arch if that is enabled. > Set ISAR_ENABLE_COMPAT_ARCH for all supported combinations in CI. > > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> > --- > meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ > meta-isar/recipes-app/libhello/libhello.bb | 3 +++ > scripts/ci_build.sh | 6 ++++++ > 3 files changed, 12 insertions(+) > > diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb b/meta-isar/recipes-app/hello-isar/hello-isar.bb > index 144a433e..8c3ba8b2 100644 > --- a/meta-isar/recipes-app/hello-isar/hello-isar.bb > +++ b/meta-isar/recipes-app/hello-isar/hello-isar.bb > @@ -20,4 +20,7 @@ SRC_URI = " \ > file://yet-another-change.txt;apply=yes;striplevel=0" > SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" > > +# NOTE: This is just to test 32-bit building on 64-bit archs. > +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" > + > inherit dpkg > diff --git a/meta-isar/recipes-app/libhello/libhello.bb b/meta-isar/recipes-app/libhello/libhello.bb > index ab271b58..5c44de50 100644 > --- a/meta-isar/recipes-app/libhello/libhello.bb > +++ b/meta-isar/recipes-app/libhello/libhello.bb > @@ -13,4 +13,7 @@ PV = "0.1-98f2e41" > SRC_URI = "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}" > SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" > > +# NOTE: This is just to test 32-bit building on 64-bit archs. > +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" > + > inherit dpkg > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh > index d2c707b8..461fd5cc 100755 > --- a/scripts/ci_build.sh > +++ b/scripts/ci_build.sh > @@ -139,6 +139,12 @@ if [ ! -d "$BUILD_DIR" ]; then > fi > source isar-init-build-env "$BUILD_DIR" > > +cat >>conf/local.conf <<EOF > +ISAR_ENABLE_COMPAT_ARCH_amd64 = "1" > +ISAR_ENABLE_COMPAT_ARCH_arm64 = "1" > +ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0" > +EOF > + > if [ -n "$CROSS_BUILD" ]; then > sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf > fi > This triggers a problem: In a multiconfig setup like our CI, building the 32-bit package from both the native 32-bit arch as well as the 64-bit arch now races. If the 64-bit arch tries to install its self-built compat package, that may just be under re-deployment by the 32-bit arch, and the build fails. Options: - somehow express that both archs provide the same package (though that may defeat the purpose of testing the compat build) - split isar-apt per arch, not just per distro Suggestions? Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] ci: Add compat arch support 2020-09-08 8:53 ` Jan Kiszka @ 2020-09-09 13:44 ` Henning Schild 2020-09-10 6:51 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Henning Schild @ 2020-09-09 13:44 UTC (permalink / raw) To: [ext] Jan Kiszka; +Cc: isar-users On Tue, 8 Sep 2020 10:53:58 +0200 "[ext] Jan Kiszka" <jan.kiszka@siemens.com> wrote: > On 04.09.20 14:47, [ext] Jan Kiszka wrote: > > From: Jan Kiszka <jan.kiszka@siemens.com> > > > > Build hello-isar and libisar for the compat arch if that is enabled. > > Set ISAR_ENABLE_COMPAT_ARCH for all supported combinations in CI. > > > > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> > > --- > > meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ > > meta-isar/recipes-app/libhello/libhello.bb | 3 +++ > > scripts/ci_build.sh | 6 ++++++ > > 3 files changed, 12 insertions(+) > > > > diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb > > b/meta-isar/recipes-app/hello-isar/hello-isar.bb index > > 144a433e..8c3ba8b2 100644 --- > > a/meta-isar/recipes-app/hello-isar/hello-isar.bb +++ > > b/meta-isar/recipes-app/hello-isar/hello-isar.bb @@ -20,4 +20,7 @@ > > SRC_URI = " \ file://yet-another-change.txt;apply=yes;striplevel=0" > > SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" > > > > +# NOTE: This is just to test 32-bit building on 64-bit archs. > > +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" > > + > > inherit dpkg > > diff --git a/meta-isar/recipes-app/libhello/libhello.bb > > b/meta-isar/recipes-app/libhello/libhello.bb index > > ab271b58..5c44de50 100644 --- > > a/meta-isar/recipes-app/libhello/libhello.bb +++ > > b/meta-isar/recipes-app/libhello/libhello.bb @@ -13,4 +13,7 @@ PV = > > "0.1-98f2e41" SRC_URI = > > "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}" > > SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" > > +# NOTE: This is just to test 32-bit building on 64-bit archs. > > +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" > > + > > inherit dpkg > > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh > > index d2c707b8..461fd5cc 100755 > > --- a/scripts/ci_build.sh > > +++ b/scripts/ci_build.sh > > @@ -139,6 +139,12 @@ if [ ! -d "$BUILD_DIR" ]; then > > fi > > source isar-init-build-env "$BUILD_DIR" > > > > +cat >>conf/local.conf <<EOF > > +ISAR_ENABLE_COMPAT_ARCH_amd64 = "1" > > +ISAR_ENABLE_COMPAT_ARCH_arm64 = "1" > > +ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0" > > +EOF > > + > > if [ -n "$CROSS_BUILD" ]; then > > sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= > > "1"/g' conf/local.conf fi > > > > This triggers a problem: > > In a multiconfig setup like our CI, building the 32-bit package from > both the native 32-bit arch as well as the 64-bit arch now races. If > the 64-bit arch tries to install its self-built compat package, that > may just be under re-deployment by the 32-bit arch, and the build > fails. > > Options: > - somehow express that both archs provide the same package (though > that may defeat the purpose of testing the compat build) The purpose is that people want to use compat because they have just one image. > - split isar-apt per arch, not just per distro But would the compat not need to include both now, and the mc enemy would still mess with one of them? - better/more locking around isar-apt readers and writers > Suggestions? Well one thing ... but i do not want to repeat myself ... Henning > Jan > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] ci: Add compat arch support 2020-09-09 13:44 ` Henning Schild @ 2020-09-10 6:51 ` Jan Kiszka 2020-09-10 6:54 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2020-09-10 6:51 UTC (permalink / raw) To: Henning Schild; +Cc: isar-users On 09.09.20 15:44, Henning Schild wrote: > On Tue, 8 Sep 2020 10:53:58 +0200 > "[ext] Jan Kiszka" <jan.kiszka@siemens.com> wrote: > >> On 04.09.20 14:47, [ext] Jan Kiszka wrote: >>> From: Jan Kiszka <jan.kiszka@siemens.com> >>> >>> Build hello-isar and libisar for the compat arch if that is enabled. >>> Set ISAR_ENABLE_COMPAT_ARCH for all supported combinations in CI. >>> >>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> >>> --- >>> meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ >>> meta-isar/recipes-app/libhello/libhello.bb | 3 +++ >>> scripts/ci_build.sh | 6 ++++++ >>> 3 files changed, 12 insertions(+) >>> >>> diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb >>> b/meta-isar/recipes-app/hello-isar/hello-isar.bb index >>> 144a433e..8c3ba8b2 100644 --- >>> a/meta-isar/recipes-app/hello-isar/hello-isar.bb +++ >>> b/meta-isar/recipes-app/hello-isar/hello-isar.bb @@ -20,4 +20,7 @@ >>> SRC_URI = " \ file://yet-another-change.txt;apply=yes;striplevel=0" >>> SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" >>> >>> +# NOTE: This is just to test 32-bit building on 64-bit archs. >>> +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" >>> + >>> inherit dpkg >>> diff --git a/meta-isar/recipes-app/libhello/libhello.bb >>> b/meta-isar/recipes-app/libhello/libhello.bb index >>> ab271b58..5c44de50 100644 --- >>> a/meta-isar/recipes-app/libhello/libhello.bb +++ >>> b/meta-isar/recipes-app/libhello/libhello.bb @@ -13,4 +13,7 @@ PV = >>> "0.1-98f2e41" SRC_URI = >>> "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}" >>> SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" >>> +# NOTE: This is just to test 32-bit building on 64-bit archs. >>> +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" >>> + >>> inherit dpkg >>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh >>> index d2c707b8..461fd5cc 100755 >>> --- a/scripts/ci_build.sh >>> +++ b/scripts/ci_build.sh >>> @@ -139,6 +139,12 @@ if [ ! -d "$BUILD_DIR" ]; then >>> fi >>> source isar-init-build-env "$BUILD_DIR" >>> >>> +cat >>conf/local.conf <<EOF >>> +ISAR_ENABLE_COMPAT_ARCH_amd64 = "1" >>> +ISAR_ENABLE_COMPAT_ARCH_arm64 = "1" >>> +ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0" >>> +EOF >>> + >>> if [ -n "$CROSS_BUILD" ]; then >>> sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= >>> "1"/g' conf/local.conf fi >>> >> >> This triggers a problem: >> >> In a multiconfig setup like our CI, building the 32-bit package from >> both the native 32-bit arch as well as the 64-bit arch now races. If >> the 64-bit arch tries to install its self-built compat package, that >> may just be under re-deployment by the 32-bit arch, and the build >> fails. >> >> Options: >> - somehow express that both archs provide the same package (though >> that may defeat the purpose of testing the compat build) > > The purpose is that people want to use compat because they have just > one image. Well, one could imagine building a legacy application for both an old ARMv7 device as well as for a new ARMv8 design. All that in one build run (mc:...). Then the user would have to express that my_beloved_legacy_app is actually a dependency that only needs to be built once, and it would not matter if that is done by the armhf buildchroot or via compat by the arm64 one. > >> - split isar-apt per arch, not just per distro > > But would the compat not need to include both now, and the mc enemy > would still mess with one of them? We would have isar-apt/apt/debian-buster-armhf, debian-buster-arm64 etc., but all of them able to carry packages for any arch. > > - better/more locking around isar-apt readers and writers > Nope, that would only paper over the fact that a package is built twice and the one to be used would be chosen by chance. Bad for reliable testing. Jan >> Suggestions? > > Well one thing ... but i do not want to repeat myself ... > > Henning > >> Jan >> > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] ci: Add compat arch support 2020-09-10 6:51 ` Jan Kiszka @ 2020-09-10 6:54 ` Jan Kiszka 2020-09-10 10:34 ` Henning Schild 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2020-09-10 6:54 UTC (permalink / raw) To: Henning Schild; +Cc: isar-users On 10.09.20 08:51, [ext] Jan Kiszka wrote: > On 09.09.20 15:44, Henning Schild wrote: >> On Tue, 8 Sep 2020 10:53:58 +0200 >> "[ext] Jan Kiszka" <jan.kiszka@siemens.com> wrote: >> >>> On 04.09.20 14:47, [ext] Jan Kiszka wrote: >>>> From: Jan Kiszka <jan.kiszka@siemens.com> >>>> >>>> Build hello-isar and libisar for the compat arch if that is enabled. >>>> Set ISAR_ENABLE_COMPAT_ARCH for all supported combinations in CI. >>>> >>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> >>>> --- >>>> meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ >>>> meta-isar/recipes-app/libhello/libhello.bb | 3 +++ >>>> scripts/ci_build.sh | 6 ++++++ >>>> 3 files changed, 12 insertions(+) >>>> >>>> diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb >>>> b/meta-isar/recipes-app/hello-isar/hello-isar.bb index >>>> 144a433e..8c3ba8b2 100644 --- >>>> a/meta-isar/recipes-app/hello-isar/hello-isar.bb +++ >>>> b/meta-isar/recipes-app/hello-isar/hello-isar.bb @@ -20,4 +20,7 @@ >>>> SRC_URI = " \ file://yet-another-change.txt;apply=yes;striplevel=0" >>>> SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" >>>> >>>> +# NOTE: This is just to test 32-bit building on 64-bit archs. >>>> +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" >>>> + >>>> inherit dpkg >>>> diff --git a/meta-isar/recipes-app/libhello/libhello.bb >>>> b/meta-isar/recipes-app/libhello/libhello.bb index >>>> ab271b58..5c44de50 100644 --- >>>> a/meta-isar/recipes-app/libhello/libhello.bb +++ >>>> b/meta-isar/recipes-app/libhello/libhello.bb @@ -13,4 +13,7 @@ PV = >>>> "0.1-98f2e41" SRC_URI = >>>> "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}" >>>> SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" >>>> +# NOTE: This is just to test 32-bit building on 64-bit archs. >>>> +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" >>>> + >>>> inherit dpkg >>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh >>>> index d2c707b8..461fd5cc 100755 >>>> --- a/scripts/ci_build.sh >>>> +++ b/scripts/ci_build.sh >>>> @@ -139,6 +139,12 @@ if [ ! -d "$BUILD_DIR" ]; then >>>> fi >>>> source isar-init-build-env "$BUILD_DIR" >>>> >>>> +cat >>conf/local.conf <<EOF >>>> +ISAR_ENABLE_COMPAT_ARCH_amd64 = "1" >>>> +ISAR_ENABLE_COMPAT_ARCH_arm64 = "1" >>>> +ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0" >>>> +EOF >>>> + >>>> if [ -n "$CROSS_BUILD" ]; then >>>> sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= >>>> "1"/g' conf/local.conf fi >>>> >>> >>> This triggers a problem: >>> >>> In a multiconfig setup like our CI, building the 32-bit package from >>> both the native 32-bit arch as well as the 64-bit arch now races. If >>> the 64-bit arch tries to install its self-built compat package, that >>> may just be under re-deployment by the 32-bit arch, and the build >>> fails. >>> >>> Options: >>> - somehow express that both archs provide the same package (though >>> that may defeat the purpose of testing the compat build) >> >> The purpose is that people want to use compat because they have just >> one image. > > Well, one could imagine building a legacy application for both an old > ARMv7 device as well as for a new ARMv8 design. All that in one build > run (mc:...). Then the user would have to express that > my_beloved_legacy_app is actually a dependency that only needs to be > built once, and it would not matter if that is done by the armhf > buildchroot or via compat by the arm64 one. > I think I know the answer for our testing scenario: call the the compat-variant of hello-isar "hello-isar-compat" and request that package in the 64-bit image. Then both build paths are stressed, and everyone gets what is desired. Let me try that later... Jan >> >>> - split isar-apt per arch, not just per distro >> >> But would the compat not need to include both now, and the mc enemy >> would still mess with one of them? > > We would have isar-apt/apt/debian-buster-armhf, debian-buster-arm64 > etc., but all of them able to carry packages for any arch. > >> >> - better/more locking around isar-apt readers and writers >> > > Nope, that would only paper over the fact that a package is built twice > and the one to be used would be chosen by chance. Bad for reliable testing. > > Jan > >>> Suggestions? >> >> Well one thing ... but i do not want to repeat myself ... >> >> Henning >> >>> Jan >>> >> > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] ci: Add compat arch support 2020-09-10 6:54 ` Jan Kiszka @ 2020-09-10 10:34 ` Henning Schild 0 siblings, 0 replies; 8+ messages in thread From: Henning Schild @ 2020-09-10 10:34 UTC (permalink / raw) To: Jan Kiszka; +Cc: isar-users Am Thu, 10 Sep 2020 08:54:00 +0200 schrieb Jan Kiszka <jan.kiszka@siemens.com>: > On 10.09.20 08:51, [ext] Jan Kiszka wrote: > > On 09.09.20 15:44, Henning Schild wrote: > >> On Tue, 8 Sep 2020 10:53:58 +0200 > >> "[ext] Jan Kiszka" <jan.kiszka@siemens.com> wrote: > >> > >>> On 04.09.20 14:47, [ext] Jan Kiszka wrote: > >>>> From: Jan Kiszka <jan.kiszka@siemens.com> > >>>> > >>>> Build hello-isar and libisar for the compat arch if that is > >>>> enabled. Set ISAR_ENABLE_COMPAT_ARCH for all supported > >>>> combinations in CI. > >>>> > >>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> > >>>> --- > >>>> meta-isar/recipes-app/hello-isar/hello-isar.bb | 3 +++ > >>>> meta-isar/recipes-app/libhello/libhello.bb | 3 +++ > >>>> scripts/ci_build.sh | 6 ++++++ > >>>> 3 files changed, 12 insertions(+) > >>>> > >>>> diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb > >>>> b/meta-isar/recipes-app/hello-isar/hello-isar.bb index > >>>> 144a433e..8c3ba8b2 100644 --- > >>>> a/meta-isar/recipes-app/hello-isar/hello-isar.bb +++ > >>>> b/meta-isar/recipes-app/hello-isar/hello-isar.bb @@ -20,4 +20,7 > >>>> @@ SRC_URI = " \ > >>>> file://yet-another-change.txt;apply=yes;striplevel=0" SRCREV = > >>>> "a18c14cc11ce6b003f3469e89223cffb4016861d" > >>>> +# NOTE: This is just to test 32-bit building on 64-bit archs. > >>>> +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" > >>>> + > >>>> inherit dpkg > >>>> diff --git a/meta-isar/recipes-app/libhello/libhello.bb > >>>> b/meta-isar/recipes-app/libhello/libhello.bb index > >>>> ab271b58..5c44de50 100644 --- > >>>> a/meta-isar/recipes-app/libhello/libhello.bb +++ > >>>> b/meta-isar/recipes-app/libhello/libhello.bb @@ -13,4 +13,7 @@ > >>>> PV = "0.1-98f2e41" SRC_URI = > >>>> "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}" > >>>> SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" > >>>> +# NOTE: This is just to test 32-bit building on 64-bit archs. > >>>> +PACKAGE_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}" > >>>> + > >>>> inherit dpkg > >>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh > >>>> index d2c707b8..461fd5cc 100755 > >>>> --- a/scripts/ci_build.sh > >>>> +++ b/scripts/ci_build.sh > >>>> @@ -139,6 +139,12 @@ if [ ! -d "$BUILD_DIR" ]; then > >>>> fi > >>>> source isar-init-build-env "$BUILD_DIR" > >>>> > >>>> +cat >>conf/local.conf <<EOF > >>>> +ISAR_ENABLE_COMPAT_ARCH_amd64 = "1" > >>>> +ISAR_ENABLE_COMPAT_ARCH_arm64 = "1" > >>>> +ISAR_ENABLE_COMPAT_ARCH_debian-stretch_amd64 = "0" > >>>> +EOF > >>>> + > >>>> if [ -n "$CROSS_BUILD" ]; then > >>>> sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= > >>>> "1"/g' conf/local.conf fi > >>>> > >>> > >>> This triggers a problem: > >>> > >>> In a multiconfig setup like our CI, building the 32-bit package > >>> from both the native 32-bit arch as well as the 64-bit arch now > >>> races. If the 64-bit arch tries to install its self-built compat > >>> package, that may just be under re-deployment by the 32-bit arch, > >>> and the build fails. > >>> > >>> Options: > >>> - somehow express that both archs provide the same package > >>> (though that may defeat the purpose of testing the compat build) > >> > >> The purpose is that people want to use compat because they have > >> just one image. > > > > Well, one could imagine building a legacy application for both an > > old ARMv7 device as well as for a new ARMv8 design. All that in one > > build run (mc:...). Then the user would have to express that > > my_beloved_legacy_app is actually a dependency that only needs to be > > built once, and it would not matter if that is done by the armhf > > buildchroot or via compat by the arm64 one. > > > > I think I know the answer for our testing scenario: call the the > compat-variant of hello-isar "hello-isar-compat" and request that > package in the 64-bit image. Then both build paths are stressed, and > everyone gets what is desired. If you manage to just rename it on the bitbake side ... maybe. Think about a compat rebuild of a lib where you want to keep using debian packages using that shared lib and not mess with all deps. You would also create conflicting packages if you rename in debian. Henning > Let me try that later... > > Jan > > >> > >>> - split isar-apt per arch, not just per distro > >> > >> But would the compat not need to include both now, and the mc enemy > >> would still mess with one of them? > > > > We would have isar-apt/apt/debian-buster-armhf, debian-buster-arm64 > > etc., but all of them able to carry packages for any arch. > > > >> > >> - better/more locking around isar-apt readers and writers > >> > > > > Nope, that would only paper over the fact that a package is built > > twice and the one to be used would be chosen by chance. Bad for > > reliable testing. > > > > Jan > > > >>> Suggestions? > >> > >> Well one thing ... but i do not want to repeat myself ... > >> > >> Henning > >> > >>> Jan > >>> > >> > > > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-09-10 10:34 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-09-04 12:47 [PATCH 0/2] 32-bit compat arch support Jan Kiszka 2020-09-04 12:47 ` [PATCH 1/2] Add compat architecture support via multiarch Jan Kiszka 2020-09-04 12:47 ` [PATCH 2/2] ci: Add compat arch support Jan Kiszka 2020-09-08 8:53 ` Jan Kiszka 2020-09-09 13:44 ` Henning Schild 2020-09-10 6:51 ` Jan Kiszka 2020-09-10 6:54 ` Jan Kiszka 2020-09-10 10:34 ` Henning Schild
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox