RE: [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

Koch, Stefan (DI PA DCP R&D 3) <stefan-koch@siemens.com>, Monday, August 21, 2023 3:45 PM:
> When using a cross build this patch does introduce optionally
> host and target specific kernel kbuild packages that
> ship the "scripts" and "tools" binaries.
> 
> By default at cross builds a kbuild-cross package will be
> generated that provides the host specific kbuild package.
> 
> The "-compat" and "-native" multiarch bitbake targets are useable to run
> additional target or host specific builds for kbuild scripts and tools.
> 
> Using the "-compat" bitbake target enables the build of
> a target specific kbuild package at cross builds.
> 
> Using the "-native" bitbake target enables the build of
> a host specific kbuild package at cross builds.
> 
> This solves this from doc/custom_kernel.inc:
> - The kernel headers package has not supported both native
>   and cross compilation of kernel modules when itself was cross built
> - Future roadmap: Generate kernel headers package for both host
>   and target when using a cross build
> 
> Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> ---
>  .../linux/files/debian/control.tmpl           | 16 +++++-
>  .../linux/files/debian/isar/build.tmpl        | 12 ++++-
>  .../linux/files/debian/isar/common.tmpl       |  8 +++
>  .../linux/files/debian/isar/install.tmpl      | 26 ++++++----
>  .../linux/files/debian/rules.tmpl             |  2 +-
>  meta/recipes-kernel/linux/linux-custom.inc    | 49 ++++++++++++++++---
>  6 files changed, 94 insertions(+), 19 deletions(-)
> 
> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> b/meta/recipes-kernel/linux/files/debian/control.tmpl
> index b19ca2c9..c012048b 100644
> --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> @@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS}
>  Homepage: http://www.kernel.org/
> 
>  Package: linux-image-${KERNEL_NAME_PROVIDED}
> +Build-Profiles: <!targetbuild !hostbuild>
>  Architecture: any
>  Depends: ${KERNEL_DEBIAN_DEPENDS}
>  Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
> @@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version
> @KR@
>   files, version: @KR@.
> 
>  Package: linux-headers-${KERNEL_NAME_PROVIDED}
> +Build-Profiles: <!targetbuild !hostbuild>
>  Architecture: any
>  Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
>  Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
> @@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers
> for @KR@
>   This is useful for people who need to build external modules
> 
>  Package: linux-libc-dev
> -Build-Profiles: <!nolibcdev>
> +Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
>  Section: devel
>  Provides: linux-kernel-headers
>  Architecture: any
> @@ -30,6 +32,7 @@ Description: Linux support headers for userspace
> development
>   are used by the installed headers for GNU glibc and other system libraries.
> 
>  Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> +Build-Profiles: <!targetbuild !hostbuild>
>  Section: debug
>  Architecture: any
>  Description: Linux kernel debugging symbols for @KR@
> @@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols for @KR@
>   all the necessary debug symbols for the kernel and its modules.
> 
>  Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> +Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
>  Architecture: any
>  Depends: ${perl:Depends}, ${shlib:Depends}
>  Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
>   This package provides kernel kbuild scripts and tools for @KR@
>   This is useful for people who need to build external modules
> +
> +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> +Build-Profiles: <!targetbuild !hostbuild crosskbuild>
> +Architecture: any
> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
> +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
> + This package provides kernel kbuild scripts and tools
> + as ${HOST_ARCH} cross binaries for @KR@
> + This is useful for those who need to cross build
> + external modules using ISAR's sbuild-chroot-host
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> index 906dc580..7f293528 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> @@ -22,7 +22,17 @@ do_build() {
>      sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-
> ${KERNEL_NAME_PROVIDED}.*
> 
>      # Build the Linux kernel
> -    ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> KAFLAGS="${KAFLAGS}"
> +    if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> "hostbuild"; then # Build kernel scripts and tools
> +        ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> KAFLAGS="${KAFLAGS}" scripts
> +        if grep -q -E "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
> ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
> +            ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
> +        fi
> +        if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config; then
> +            ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
> +        fi
> +    else # Build the Linux kernel
> +        ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> KAFLAGS="${KAFLAGS}"
> +    fi
> 
>      # Stop tracing
>      set +x
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> index f4c0519f..36e01691 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> @@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-
> ${KERNEL_NAME_PROVIDED}
>  KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
>  KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> 
> +if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
> +	# Force creating kernel kbuild debian package with valid host arch
> +	# Use a cross build to comply with arch specific kernel defconfigs
> +	# The scripts and tools are always created for host arch
> +	eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
> +	CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
> +fi
> +
>  # Constants
>  KCONF=.config
> 
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> index 236b67c4..08b8a9a5 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> @@ -34,18 +34,24 @@ do_install() {
>      set -x
> 
>      # Run the install steps
> -    install_image
> -    if [ "${ARCH}" != "um" ]; then
> -        install_config
> -        install_map
> +    if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> "hostbuild"; then
> +        install_image
> +        if [ "${ARCH}" != "um" ]; then
> +            install_config
> +            install_map
> +        fi
> +        install_hooks
> +        install_dtbs
> +        install_kmods
> +        install_headers
>      fi
> -    install_hooks
> -    install_dtbs
> -    install_kmods
> -    install_headers
> 
> -    # Cleanup and install kernel scripts and tools
> -    rm -rf ${deb_kern_kbuild_dir}
> +    if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q "crosskbuild"; then
> +        # Install cross kernel scripts and tools
> +        install_kbuild ${deb_kern_kbuild_dir}-cross
> +    fi
> +
> +    # Install kernel scripts and tools
>      install_kbuild ${deb_kern_kbuild_dir}
> 
>      # Stop tracing
> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> index e8ae3daa..1f838a41 100755
> --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> @@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
>  	true
> 
>  override_dh_strip:
> -	unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-automatic-dbgsym
> +	dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-cross --no-
> automatic-dbgsym
> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-
> kernel/linux/linux-custom.inc
> index 94ed84a2..85faa2d7 100644
> --- a/meta/recipes-kernel/linux/linux-custom.inc
> +++ b/meta/recipes-kernel/linux/linux-custom.inc
> @@ -85,9 +85,6 @@ TEMPLATE_VARS += "                \
>      DISTRIBUTOR                   \
>  "
> 
> -inherit dpkg
> -inherit template
> -
>  # Add custom cflags to the kernel build
>  KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
>  KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> @@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
>  # Derive name of the kernel packages from the name of this recipe
>  KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-')[2]}"
> 
> -# Make bitbake know we will be producing linux-image and linux-headers
> packages
>  python() {
>      kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
> -    d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> -                         'linux-headers-' + kernel_name)
> +    distro_arch = d.getVar("DISTRO_ARCH")
> +    host_arch = d.getVar("HOST_ARCH")
>      headers_depends = "linux-kbuild-" + kernel_name
> 
> +    # The "-compat" and "-native" multiarch bitbake targets are used to run
> +    # additional target or host specific builds for kbuild scripts and
> tools.
> +    # The "-compat" bitbake target requests a "target" build
> +    # and the "-native" bitbake target requests a "host" build.
> +    #
> +    # For different distro and host archs
> +    # - Add dependency for sbuild-chroot-target
> +    #   to allow building arch specific kbuild scripts and tools
> +    # - Set correct variable overrides
> +    is_compat = "compat" in d.getVar("PN", True)
> +    is_native = "native" in d.getVar("PN", True)

Just a minor thing: The intended/preferred interface for recipes are the
"class-native" and "class-compat" overrides set in multiarch.bbclass.
This is nice for bitbake (just set MY_VAR:class-native = "value"), but
unfortunately not so easy to use from Python code. Still, I'd avoid
looking at PN again in the recipes, and instead go for

is_compat = "class-compat" in d.getVar("OVERRIDES").split(":")

> +
> +    # Determine creation of a kbuild-cross or (standard) kbuild package for
> the host
> +    is_cross_profile = distro_arch != host_arch and
> d.getVar("ISAR_CROSS_COMPILE", True) == "1"
> +
> +    if is_cross_profile:
> +        headers_depends = headers_depends + "-cross | " + headers_depends
> +        build_depends = "crossbuild-essential-" + distro_arch + " [" +
> host_arch + "], "
> +        d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
> +        d.prependVar("KBUILD_DEPENDS", build_depends)
> +        d.appendVarFlag("do_prepare_build", "depends", " sbuild-chroot-
> target:do_build")
> +        d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
> +        d.setVar("COMPAT_DISTRO_ARCH", distro_arch)

This feels hacky... it overrides the compat mechanism, resulting in a
"*-compat" package that is not actually for the compat arch.

Adriaan

> +
> +        if is_compat:
> +            d.appendVar("DEB_BUILD_PROFILES", " targetbuild")
> +            d.setVar("ISAR_CROSS_COMPILE", "0")
> +        elif is_native:
> +            d.appendVar("DEB_BUILD_PROFILES", " hostbuild")
> +    else:
> +        d.appendVar("DEB_BUILD_PROFILES", " defaultkbuild")
> +
> +    # Make bitbake know we will be producing
> +    # linux-image and linux-headers packages
> +    if not is_compat and not is_native:
> +        d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
> +                             "linux-headers-" + kernel_name)
> +
>      # Set dependency for kernel headers
>      d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
>  }
> 
> +inherit dpkg
> +inherit template
> +
>  def get_kernel_arch(d):
>      distro_arch = d.getVar("DISTRO_ARCH")
>      if distro_arch in ["amd64", "i386"]:
> --
> 2.39.2