Re: [PATCH v2] linux-custom: Speedup build of target specific linux-kbuild package

["Koch, Stefan" <stefan-koch@siemens.com>]

On Fri, 2024-06-07 at 17:07 +0200, Jan Kiszka wrote:
> On 07.06.24 16:11, Stefan Koch wrote:
> > Avoids time expensive QEMU-emulated merging of kernel config by
> > using
> > the existing kernel config for the target specific linux-kbuild
> > 
> > Using DEPENDS instead of RDEPENDS ensures creation of
> > kernel (including config) before build of target specific linux-
> > kbuild
> > 
> > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > ---
> >  .../linux/files/debian/isar/configure.tmpl     | 18
> > ++++++++++++++----
> >  meta/recipes-kernel/linux/linux-custom.inc     | 10 +++++++---
> >  2 files changed, 21 insertions(+), 7 deletions(-)
> > 
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/configure.tmpl b/meta/recipes-
> > kernel/linux/files/debian/isar/configure.tmpl
> > index 389c9a85..72eae448 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl
> > @@ -10,10 +10,20 @@ do_configure() {
> >      # Trace what we do here
> >      set -x
> >  
> > -    # Process kernel config target and fragments
> > -    ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET}
> > -    ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \
> > -        ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS}
> > +    kernelconfig="$(find /boot -maxdepth 1 -name "config-${PV}*" -
> > print -quit)"
> 
> Can we additionally limit this to the case were we expect the config
> there (kbuild && !kernel)? Just to reduce the risk that some "this
> should never be there" config is ever silently pulled by the base
> package build. Only if our dependency is installed, that rather
> generic
> config-<version> file is from there, or we failed on installing the
> dependency.
Build profile can be checked here as second condition, too.

find with "config-${PV}*" is used because some kernel specify "-rt18"
or "-isar" or something else that is not included in PV:

/boot/config-5.10.209-audis-rtp-rt18
/boot/config-6.6.11-isar

Some kernel do not specify an suffix to ${PV}. Maybe it depends on
LOCALVERISON from the kernel config? But as kernel config is not
available, we cannot find out this information...

find returns the first match. When using the old isar-buildchroot
instead of sbuild/schroot: Multiple configs could exist - if multiple
kernels are built. But if ${PV} is identical dpkg should have a
conflict with overwriting an existing config with the same version? (as
long this "-rt18"/"-isar" suffix is not renamed for that build.)

See below for dependency problem.
> 
> > +    if [ -e "${kernelconfig}" ]; then
> > +        # Prefer existing kernel config
> > +        # So, very expensive QEMU-emulated merge_config.sh
> > +        # can be skipped for target specific linux-kbuild package
> > +        mkdir -p ${KERNEL_BUILD_DIR}
> > +        cp "${kernelconfig}" ${KERNEL_BUILD_DIR}/.config
> > +        ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig
> > +    else
> > +        # Process kernel config target and fragments
> > +        ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET}
> > +        ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/
> > \
> > +            ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS}
> > +    fi
> >  
> >      # Stop tracing
> >      set +x
> > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > b/meta/recipes-kernel/linux/linux-custom.inc
> > index 647f09dd..dc354e13 100644
> > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > @@ -25,6 +25,7 @@ KBUILD_DEPENDS ?= "build-essential:native, \
> >                     flex, \
> >                     git, \
> >                     kmod, \
> > +                   linux-image-
> > ${KERNEL_NAME_PROVIDED}:${DISTRO_ARCH} <kbuild !kernel>, \
That ensures installation of the dependency.
> >                     rsync,"
> >  
> >  KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool,
> > \
> > @@ -112,7 +113,7 @@ BUILD_PROFILES = "kernel kbuild"
> >  BBCLASSEXTEND:append:cross-profile = " kbuildtarget"
> >  
> >  # When cross-profile is active:
> > -# build only kernel with the default variant of the recipe
> > +# build only kernel (including config) with the default variant of
> > the recipe
> >  BUILD_PROFILES:cross-profile = "kernel"
> >  
> >  # -native: kbuild package for host
> > @@ -122,14 +123,17 @@ RECIPE_PROVIDES:class-native = " \
> >      linux-kbuild-${KERNEL_NAME_PROVIDED}"
> >  # Use pseudo target to pull in the base variant of the recipe.
> >  # Will be auto-extended with -native by multiarch.bbclass.
> > -RDEPENDS:class-native += "${BPN}-pseudo"
> > +# Using DEPENDS, see below
> > +DEPENDS:class-native += "${BPN}-pseudo"
> 
> Need to help me with remember my own code: Why do we also need this
> build dependency? I thought you are only optimizing the kbuildtarget
> case.
Kernel config is also read from linux-image for linux-kbuild-native.
When choosing to call merge_config.sh here again, this change would not
be needed. Then a different build profile must be added, because
<kernel !kbuild> won't match anymore.
> 
> >  
> >  # -kbuildtarget: kbuild package for target, enforcing non-cross-
> > build
> >  BUILD_PROFILES:class-kbuildtarget = "kbuild"
> >  RECIPE_PROVIDES:class-kbuildtarget = " \
> >      linux-headers-${KERNEL_NAME_PROVIDED} \
> >      linux-kbuild-${KERNEL_NAME_PROVIDED}"
> > -RDEPENDS:class-kbuildtarget = "${BPN}"
> > +# Using DEPENDS instead of RDEPENDS to ensure creation of kernel
> > including
> > +# pregenerated kernel config before target specific linux-kbuild
> > package build
> > +DEPENDS:class-kbuildtarget = "${BPN}"
> >  ISAR_CROSS_COMPILE:class-kbuildtarget = "0"
> >  
> >  # Make bitbake know we will be producing linux-image and linux-
> > headers packages
> 
> Fine-tuning - this generally looks like a nice optimization now!
> 
> Jan
> 

-- 
Stefan Koch
Siemens AG
www.siemens.com