* [PATCH v2] linux-custom: Speedup build of target specific linux-kbuild package @ 2024-06-07 14:11 Stefan Koch 2024-06-07 15:07 ` Jan Kiszka 0 siblings, 1 reply; 3+ messages in thread From: Stefan Koch @ 2024-06-07 14:11 UTC (permalink / raw) To: isar-users Cc: stefan-koch, jan.kiszka, christian.storm, michael.adler, simon.sudler, cedric.hombourger, adriaan.schmidt, felix.moessbauer, ubely, amikan Avoids time expensive QEMU-emulated merging of kernel config by using the existing kernel config for the target specific linux-kbuild Using DEPENDS instead of RDEPENDS ensures creation of kernel (including config) before build of target specific linux-kbuild Signed-off-by: Stefan Koch <stefan-koch@siemens.com> --- .../linux/files/debian/isar/configure.tmpl | 18 ++++++++++++++---- meta/recipes-kernel/linux/linux-custom.inc | 10 +++++++--- 2 files changed, 21 insertions(+), 7 deletions(-) diff --git a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl index 389c9a85..72eae448 100644 --- a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl +++ b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl @@ -10,10 +10,20 @@ do_configure() { # Trace what we do here set -x - # Process kernel config target and fragments - ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} - ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \ - ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS} + kernelconfig="$(find /boot -maxdepth 1 -name "config-${PV}*" -print -quit)" + if [ -e "${kernelconfig}" ]; then + # Prefer existing kernel config + # So, very expensive QEMU-emulated merge_config.sh + # can be skipped for target specific linux-kbuild package + mkdir -p ${KERNEL_BUILD_DIR} + cp "${kernelconfig}" ${KERNEL_BUILD_DIR}/.config + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig + else + # Process kernel config target and fragments + ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} + ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \ + ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS} + fi # Stop tracing set +x diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 647f09dd..dc354e13 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -25,6 +25,7 @@ KBUILD_DEPENDS ?= "build-essential:native, \ flex, \ git, \ kmod, \ + linux-image-${KERNEL_NAME_PROVIDED}:${DISTRO_ARCH} <kbuild !kernel>, \ rsync," KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ @@ -112,7 +113,7 @@ BUILD_PROFILES = "kernel kbuild" BBCLASSEXTEND:append:cross-profile = " kbuildtarget" # When cross-profile is active: -# build only kernel with the default variant of the recipe +# build only kernel (including config) with the default variant of the recipe BUILD_PROFILES:cross-profile = "kernel" # -native: kbuild package for host @@ -122,14 +123,17 @@ RECIPE_PROVIDES:class-native = " \ linux-kbuild-${KERNEL_NAME_PROVIDED}" # Use pseudo target to pull in the base variant of the recipe. # Will be auto-extended with -native by multiarch.bbclass. -RDEPENDS:class-native += "${BPN}-pseudo" +# Using DEPENDS, see below +DEPENDS:class-native += "${BPN}-pseudo" # -kbuildtarget: kbuild package for target, enforcing non-cross-build BUILD_PROFILES:class-kbuildtarget = "kbuild" RECIPE_PROVIDES:class-kbuildtarget = " \ linux-headers-${KERNEL_NAME_PROVIDED} \ linux-kbuild-${KERNEL_NAME_PROVIDED}" -RDEPENDS:class-kbuildtarget = "${BPN}" +# Using DEPENDS instead of RDEPENDS to ensure creation of kernel including +# pregenerated kernel config before target specific linux-kbuild package build +DEPENDS:class-kbuildtarget = "${BPN}" ISAR_CROSS_COMPILE:class-kbuildtarget = "0" # Make bitbake know we will be producing linux-image and linux-headers packages -- 2.39.2 ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2] linux-custom: Speedup build of target specific linux-kbuild package 2024-06-07 14:11 [PATCH v2] linux-custom: Speedup build of target specific linux-kbuild package Stefan Koch @ 2024-06-07 15:07 ` Jan Kiszka 2024-06-07 15:37 ` Koch, Stefan 0 siblings, 1 reply; 3+ messages in thread From: Jan Kiszka @ 2024-06-07 15:07 UTC (permalink / raw) To: Stefan Koch, isar-users Cc: christian.storm, michael.adler, simon.sudler, cedric.hombourger, adriaan.schmidt, felix.moessbauer, ubely, amikan On 07.06.24 16:11, Stefan Koch wrote: > Avoids time expensive QEMU-emulated merging of kernel config by using > the existing kernel config for the target specific linux-kbuild > > Using DEPENDS instead of RDEPENDS ensures creation of > kernel (including config) before build of target specific linux-kbuild > > Signed-off-by: Stefan Koch <stefan-koch@siemens.com> > --- > .../linux/files/debian/isar/configure.tmpl | 18 ++++++++++++++---- > meta/recipes-kernel/linux/linux-custom.inc | 10 +++++++--- > 2 files changed, 21 insertions(+), 7 deletions(-) > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl > index 389c9a85..72eae448 100644 > --- a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl > @@ -10,10 +10,20 @@ do_configure() { > # Trace what we do here > set -x > > - # Process kernel config target and fragments > - ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} > - ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \ > - ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS} > + kernelconfig="$(find /boot -maxdepth 1 -name "config-${PV}*" -print -quit)" Can we additionally limit this to the case were we expect the config there (kbuild && !kernel)? Just to reduce the risk that some "this should never be there" config is ever silently pulled by the base package build. Only if our dependency is installed, that rather generic config-<version> file is from there, or we failed on installing the dependency. > + if [ -e "${kernelconfig}" ]; then > + # Prefer existing kernel config > + # So, very expensive QEMU-emulated merge_config.sh > + # can be skipped for target specific linux-kbuild package > + mkdir -p ${KERNEL_BUILD_DIR} > + cp "${kernelconfig}" ${KERNEL_BUILD_DIR}/.config > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig > + else > + # Process kernel config target and fragments > + ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} > + ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \ > + ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS} > + fi > > # Stop tracing > set +x > diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc > index 647f09dd..dc354e13 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -25,6 +25,7 @@ KBUILD_DEPENDS ?= "build-essential:native, \ > flex, \ > git, \ > kmod, \ > + linux-image-${KERNEL_NAME_PROVIDED}:${DISTRO_ARCH} <kbuild !kernel>, \ > rsync," > > KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > @@ -112,7 +113,7 @@ BUILD_PROFILES = "kernel kbuild" > BBCLASSEXTEND:append:cross-profile = " kbuildtarget" > > # When cross-profile is active: > -# build only kernel with the default variant of the recipe > +# build only kernel (including config) with the default variant of the recipe > BUILD_PROFILES:cross-profile = "kernel" > > # -native: kbuild package for host > @@ -122,14 +123,17 @@ RECIPE_PROVIDES:class-native = " \ > linux-kbuild-${KERNEL_NAME_PROVIDED}" > # Use pseudo target to pull in the base variant of the recipe. > # Will be auto-extended with -native by multiarch.bbclass. > -RDEPENDS:class-native += "${BPN}-pseudo" > +# Using DEPENDS, see below > +DEPENDS:class-native += "${BPN}-pseudo" Need to help me with remember my own code: Why do we also need this build dependency? I thought you are only optimizing the kbuildtarget case. > > # -kbuildtarget: kbuild package for target, enforcing non-cross-build > BUILD_PROFILES:class-kbuildtarget = "kbuild" > RECIPE_PROVIDES:class-kbuildtarget = " \ > linux-headers-${KERNEL_NAME_PROVIDED} \ > linux-kbuild-${KERNEL_NAME_PROVIDED}" > -RDEPENDS:class-kbuildtarget = "${BPN}" > +# Using DEPENDS instead of RDEPENDS to ensure creation of kernel including > +# pregenerated kernel config before target specific linux-kbuild package build > +DEPENDS:class-kbuildtarget = "${BPN}" > ISAR_CROSS_COMPILE:class-kbuildtarget = "0" > > # Make bitbake know we will be producing linux-image and linux-headers packages Fine-tuning - this generally looks like a nice optimization now! Jan -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2] linux-custom: Speedup build of target specific linux-kbuild package 2024-06-07 15:07 ` Jan Kiszka @ 2024-06-07 15:37 ` Koch, Stefan 0 siblings, 0 replies; 3+ messages in thread From: Koch, Stefan @ 2024-06-07 15:37 UTC (permalink / raw) To: isar-users, Kiszka, Jan Cc: Storm, Christian, cedric.hombourger, Adler, Michael, amikan, Sudler, Simon, ubely, MOESSBAUER, Felix, Schmidt, Adriaan On Fri, 2024-06-07 at 17:07 +0200, Jan Kiszka wrote: > On 07.06.24 16:11, Stefan Koch wrote: > > Avoids time expensive QEMU-emulated merging of kernel config by > > using > > the existing kernel config for the target specific linux-kbuild > > > > Using DEPENDS instead of RDEPENDS ensures creation of > > kernel (including config) before build of target specific linux- > > kbuild > > > > Signed-off-by: Stefan Koch <stefan-koch@siemens.com> > > --- > > .../linux/files/debian/isar/configure.tmpl | 18 > > ++++++++++++++---- > > meta/recipes-kernel/linux/linux-custom.inc | 10 +++++++--- > > 2 files changed, 21 insertions(+), 7 deletions(-) > > > > diff --git a/meta/recipes- > > kernel/linux/files/debian/isar/configure.tmpl b/meta/recipes- > > kernel/linux/files/debian/isar/configure.tmpl > > index 389c9a85..72eae448 100644 > > --- a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl > > +++ b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl > > @@ -10,10 +10,20 @@ do_configure() { > > # Trace what we do here > > set -x > > > > - # Process kernel config target and fragments > > - ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} > > - ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \ > > - ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS} > > + kernelconfig="$(find /boot -maxdepth 1 -name "config-${PV}*" - > > print -quit)" > > Can we additionally limit this to the case were we expect the config > there (kbuild && !kernel)? Just to reduce the risk that some "this > should never be there" config is ever silently pulled by the base > package build. Only if our dependency is installed, that rather > generic > config-<version> file is from there, or we failed on installing the > dependency. Build profile can be checked here as second condition, too. find with "config-${PV}*" is used because some kernel specify "-rt18" or "-isar" or something else that is not included in PV: /boot/config-5.10.209-audis-rtp-rt18 /boot/config-6.6.11-isar Some kernel do not specify an suffix to ${PV}. Maybe it depends on LOCALVERISON from the kernel config? But as kernel config is not available, we cannot find out this information... find returns the first match. When using the old isar-buildchroot instead of sbuild/schroot: Multiple configs could exist - if multiple kernels are built. But if ${PV} is identical dpkg should have a conflict with overwriting an existing config with the same version? (as long this "-rt18"/"-isar" suffix is not renamed for that build.) See below for dependency problem. > > > + if [ -e "${kernelconfig}" ]; then > > + # Prefer existing kernel config > > + # So, very expensive QEMU-emulated merge_config.sh > > + # can be skipped for target specific linux-kbuild package > > + mkdir -p ${KERNEL_BUILD_DIR} > > + cp "${kernelconfig}" ${KERNEL_BUILD_DIR}/.config > > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig > > + else > > + # Process kernel config target and fragments > > + ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} > > + ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ > > \ > > + ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS} > > + fi > > > > # Stop tracing > > set +x > > diff --git a/meta/recipes-kernel/linux/linux-custom.inc > > b/meta/recipes-kernel/linux/linux-custom.inc > > index 647f09dd..dc354e13 100644 > > --- a/meta/recipes-kernel/linux/linux-custom.inc > > +++ b/meta/recipes-kernel/linux/linux-custom.inc > > @@ -25,6 +25,7 @@ KBUILD_DEPENDS ?= "build-essential:native, \ > > flex, \ > > git, \ > > kmod, \ > > + linux-image- > > ${KERNEL_NAME_PROVIDED}:${DISTRO_ARCH} <kbuild !kernel>, \ That ensures installation of the dependency. > > rsync," > > > > KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, > > \ > > @@ -112,7 +113,7 @@ BUILD_PROFILES = "kernel kbuild" > > BBCLASSEXTEND:append:cross-profile = " kbuildtarget" > > > > # When cross-profile is active: > > -# build only kernel with the default variant of the recipe > > +# build only kernel (including config) with the default variant of > > the recipe > > BUILD_PROFILES:cross-profile = "kernel" > > > > # -native: kbuild package for host > > @@ -122,14 +123,17 @@ RECIPE_PROVIDES:class-native = " \ > > linux-kbuild-${KERNEL_NAME_PROVIDED}" > > # Use pseudo target to pull in the base variant of the recipe. > > # Will be auto-extended with -native by multiarch.bbclass. > > -RDEPENDS:class-native += "${BPN}-pseudo" > > +# Using DEPENDS, see below > > +DEPENDS:class-native += "${BPN}-pseudo" > > Need to help me with remember my own code: Why do we also need this > build dependency? I thought you are only optimizing the kbuildtarget > case. Kernel config is also read from linux-image for linux-kbuild-native. When choosing to call merge_config.sh here again, this change would not be needed. Then a different build profile must be added, because <kernel !kbuild> won't match anymore. > > > > > # -kbuildtarget: kbuild package for target, enforcing non-cross- > > build > > BUILD_PROFILES:class-kbuildtarget = "kbuild" > > RECIPE_PROVIDES:class-kbuildtarget = " \ > > linux-headers-${KERNEL_NAME_PROVIDED} \ > > linux-kbuild-${KERNEL_NAME_PROVIDED}" > > -RDEPENDS:class-kbuildtarget = "${BPN}" > > +# Using DEPENDS instead of RDEPENDS to ensure creation of kernel > > including > > +# pregenerated kernel config before target specific linux-kbuild > > package build > > +DEPENDS:class-kbuildtarget = "${BPN}" > > ISAR_CROSS_COMPILE:class-kbuildtarget = "0" > > > > # Make bitbake know we will be producing linux-image and linux- > > headers packages > > Fine-tuning - this generally looks like a nice optimization now! > > Jan > -- Stefan Koch Siemens AG www.siemens.com ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-06-07 15:37 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-06-07 14:11 [PATCH v2] linux-custom: Speedup build of target specific linux-kbuild package Stefan Koch 2024-06-07 15:07 ` Jan Kiszka 2024-06-07 15:37 ` Koch, Stefan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox