Re: [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages

["Koch, Stefan" <stefan-koch@siemens.com>]

On Wed, 2023-09-06 at 06:38 +0000, Schmidt, Adriaan (T CED SES-DE)
wrote:
> Koch, Stefan (DI PA DCP R&D 3) <stefan-koch@siemens.com>, Monday,
> August 21, 2023 3:45 PM:
> > When using a cross build this patch does introduce optionally
> > host and target specific kernel kbuild packages that
> > ship the "scripts" and "tools" binaries.
> >
> > By default at cross builds a kbuild-cross package will be
> > generated that provides the host specific kbuild package.
> >
> > The "-compat" and "-native" multiarch bitbake targets are useable
> > to run
> > additional target or host specific builds for kbuild scripts and
> > tools.
> >
> > Using the "-compat" bitbake target enables the build of
> > a target specific kbuild package at cross builds.
> >
> > Using the "-native" bitbake target enables the build of
> > a host specific kbuild package at cross builds.
> >
> > This solves this from doc/custom_kernel.inc:
> > - The kernel headers package has not supported both native
> >   and cross compilation of kernel modules when itself was cross
> > built
> > - Future roadmap: Generate kernel headers package for both host
> >   and target when using a cross build
> >
> > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > ---
> >  .../linux/files/debian/control.tmpl           | 16 +++++-
> >  .../linux/files/debian/isar/build.tmpl        | 12 ++++-
> >  .../linux/files/debian/isar/common.tmpl       |  8 +++
> >  .../linux/files/debian/isar/install.tmpl      | 26 ++++++----
> >  .../linux/files/debian/rules.tmpl             |  2 +-
> >  meta/recipes-kernel/linux/linux-custom.inc    | 49
> > ++++++++++++++++---
> >  6 files changed, 94 insertions(+), 19 deletions(-)
> >
> > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > index b19ca2c9..c012048b 100644
> > --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > @@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS}
> >  Homepage:
> > http://www.kernel.org/
> >
> >  Package: linux-image-${KERNEL_NAME_PROVIDED}
> > +Build-Profiles: <!targetbuild !hostbuild>
> >  Architecture: any
> >  Depends: ${KERNEL_DEBIAN_DEPENDS}
> >  Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
> > @@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux
> > kernel, version
> > @KR@
> >   files, version: @KR@.
> >
> >  Package: linux-headers-${KERNEL_NAME_PROVIDED}
> > +Build-Profiles: <!targetbuild !hostbuild>
> >  Architecture: any
> >  Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > ${shlib:Depends}
> >  Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
> > @@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel
> > headers
> > for @KR@
> >   This is useful for people who need to build external modules
> >
> >  Package: linux-libc-dev
> > -Build-Profiles: <!nolibcdev>
> > +Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
> >  Section: devel
> >  Provides: linux-kernel-headers
> >  Architecture: any
> > @@ -30,6 +32,7 @@ Description: Linux support headers for userspace
> > development
> >   are used by the installed headers for GNU glibc and other system
> > libraries.
> >
> >  Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> > +Build-Profiles: <!targetbuild !hostbuild>
> >  Section: debug
> >  Architecture: any
> >  Description: Linux kernel debugging symbols for @KR@
> > @@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols for
> > @KR@
> >   all the necessary debug symbols for the kernel and its modules.
> >
> >  Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> > +Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
> >  Architecture: any
> >  Depends: ${perl:Depends}, ${shlib:Depends}
> >  Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > tools for @KR@
> >   This package provides kernel kbuild scripts and tools for @KR@
> >   This is useful for people who need to build external modules
> > +
> > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> > +Build-Profiles: <!targetbuild !hostbuild crosskbuild>
> > +Architecture: any
> > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > ${shlib:Depends}
> > +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > tools for @KR@
> > + This package provides kernel kbuild scripts and tools
> > + as ${HOST_ARCH} cross binaries for @KR@
> > + This is useful for those who need to cross build
> > + external modules using ISAR's sbuild-chroot-host
> > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > index 906dc580..7f293528 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > @@ -22,7 +22,17 @@ do_build() {
> >      sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-
> > image-
> > ${KERNEL_NAME_PROVIDED}.*
> >
> >      # Build the Linux kernel
> > -    ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > KAFLAGS="${KAFLAGS}"
> > +    if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> > "hostbuild"; then # Build kernel scripts and tools
> > +        ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > KAFLAGS="${KAFLAGS}" scripts
> > +        if grep -q -E
> > "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
> > ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
> > +            ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
> > +        fi
> > +        if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config;
> > then
> > +            ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
> > +        fi
> > +    else # Build the Linux kernel
> > +        ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > KAFLAGS="${KAFLAGS}"
> > +    fi
> >
> >      # Stop tracing
> >      set +x
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/common.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > index f4c0519f..36e01691 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > @@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-
> > ${KERNEL_NAME_PROVIDED}
> >  KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> >  KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> >
> > +if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
> > +       # Force creating kernel kbuild debian package with valid
> > host arch
> > +       # Use a cross build to comply with arch specific kernel
> > defconfigs
> > +       # The scripts and tools are always created for host arch
> > +       eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
> > +       CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
> > +fi
> > +
> >  # Constants
> >  KCONF=.config
> >
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/install.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > index 236b67c4..08b8a9a5 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > @@ -34,18 +34,24 @@ do_install() {
> >      set -x
> >
> >      # Run the install steps
> > -    install_image
> > -    if [ "${ARCH}" != "um" ]; then
> > -        install_config
> > -        install_map
> > +    if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -
> > e
> > "hostbuild"; then
> > +        install_image
> > +        if [ "${ARCH}" != "um" ]; then
> > +            install_config
> > +            install_map
> > +        fi
> > +        install_hooks
> > +        install_dtbs
> > +        install_kmods
> > +        install_headers
> >      fi
> > -    install_hooks
> > -    install_dtbs
> > -    install_kmods
> > -    install_headers
> >
> > -    # Cleanup and install kernel scripts and tools
> > -    rm -rf ${deb_kern_kbuild_dir}
> > +    if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -
> > e
> > "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q
> > "crosskbuild"; then
> > +        # Install cross kernel scripts and tools
> > +        install_kbuild ${deb_kern_kbuild_dir}-cross
> > +    fi
> > +
> > +    # Install kernel scripts and tools
> >      install_kbuild ${deb_kern_kbuild_dir}
> >
> >      # Stop tracing
> > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > index e8ae3daa..1f838a41 100755
> > --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > @@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
> >         true
> >
> >  override_dh_strip:
> > -       unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-
> > automatic-dbgsym
> > +       dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-
> > cross --no-
> > automatic-dbgsym
> > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > b/meta/recipes-
> > kernel/linux/linux-custom.inc
> > index 94ed84a2..85faa2d7 100644
> > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > @@ -85,9 +85,6 @@ TEMPLATE_VARS += "                \
> >      DISTRIBUTOR                   \
> >  "
> >
> > -inherit dpkg
> > -inherit template
> > -
> >  # Add custom cflags to the kernel build
> >  KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> >  KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > @@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> >  # Derive name of the kernel packages from the name of this recipe
> >  KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-
> > ')[2]}"
> >
> > -# Make bitbake know we will be producing linux-image and linux-
> > headers
> > packages
> >  python() {
> >      kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
> > -    d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> > -                         'linux-headers-' + kernel_name)
> > +    distro_arch = d.getVar("DISTRO_ARCH")
> > +    host_arch = d.getVar("HOST_ARCH")
> >      headers_depends = "linux-kbuild-" + kernel_name
> >
> > +    # The "-compat" and "-native" multiarch bitbake targets are
> > used to run
> > +    # additional target or host specific builds for kbuild scripts
> > and
> > tools.
> > +    # The "-compat" bitbake target requests a "target" build
> > +    # and the "-native" bitbake target requests a "host" build.
> > +    #
> > +    # For different distro and host archs
> > +    # - Add dependency for sbuild-chroot-target
> > +    #   to allow building arch specific kbuild scripts and tools
> > +    # - Set correct variable overrides
> > +    is_compat = "compat" in d.getVar("PN", True)
> > +    is_native = "native" in d.getVar("PN", True)
>
> Just a minor thing: The intended/preferred interface for recipes are
> the
> "class-native" and "class-compat" overrides set in multiarch.bbclass.
> This is nice for bitbake (just set MY_VAR:class-native = "value"),
> but
> unfortunately not so easy to use from Python code. Still, I'd avoid
> looking at PN again in the recipes, and instead go for
>
> is_compat = "class-compat" in d.getVar("OVERRIDES").split(":")
>
> > +
> > +    # Determine creation of a kbuild-cross or (standard) kbuild
> > package for
> > the host
> > +    is_cross_profile = distro_arch != host_arch and
> > d.getVar("ISAR_CROSS_COMPILE", True) == "1"
> > +
> > +    if is_cross_profile:
> > +        headers_depends = headers_depends + "-cross | " +
> > headers_depends
> > +        build_depends = "crossbuild-essential-" + distro_arch + "
> > [" +
> > host_arch + "], "
> > +        d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
> > +        d.prependVar("KBUILD_DEPENDS", build_depends)
> > +        d.appendVarFlag("do_prepare_build", "depends", " sbuild-
> > chroot-
> > target:do_build")
> > +        d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
> > +        d.setVar("COMPAT_DISTRO_ARCH", distro_arch)
>
> This feels hacky... it overrides the compat mechanism, resulting in a
> "*-compat" package that is not actually for the compat arch.

Yes, it seems a bit hacky...

I'm unsure if this special kernel specific approach to rebuild kbuild
tools/scripts, forcing non-cross QEMU build even if ISAR cross-build is
enabled, legimitates adding a new similar approach like the
compat/native mechanism within ISAR.

Reusing the "-compat" bitbake target enables the build of
a target specific kbuild package at cross builds avoiding deeper
changes to ISAR. Forcing a QEMU non-cross build even if
ISAR_CROSS_COMPILE is enabled. This is the important part: Running a
separate QEMU non-cross build for kernel scripts/tools. This is needed
because the tools/scripts kernel build does not support creation for
target arch while cross building.

Let me know if you have suggestions to archieve that without using the
compat mechanism and avoiding deeper changes to ISAR. Important is that
a second dpkg build runs separatly within target's schroot (using
forced non-cross QEMU) while main build runs cross in host's schroot.
Earlier patchsets of this series used a way without using the new
compat/native mechanism but made deeper changes to ISAR.

>
> Adriaan
>
> > +
> > +        if is_compat:
> > +            d.appendVar("DEB_BUILD_PROFILES", " targetbuild")
> > +            d.setVar("ISAR_CROSS_COMPILE", "0")
> > +        elif is_native:
> > +            d.appendVar("DEB_BUILD_PROFILES", " hostbuild")
> > +    else:
> > +        d.appendVar("DEB_BUILD_PROFILES", " defaultkbuild")
> > +
> > +    # Make bitbake know we will be producing
> > +    # linux-image and linux-headers packages
> > +    if not is_compat and not is_native:
> > +        d.setVar("PROVIDES", "linux-image-" + kernel_name + " " +
> > \
> > +                             "linux-headers-" + kernel_name)
> > +
> >      # Set dependency for kernel headers
> >      d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
> >  }
> >
> > +inherit dpkg
> > +inherit template
> > +
> >  def get_kernel_arch(d):
> >      distro_arch = d.getVar("DISTRO_ARCH")
> >      if distro_arch in ["amd64", "i386"]:
> > --
> > 2.39.2
>

--
Stefan Koch
Siemens AG
http://www.siemens.com/