Re: [PATCH v5 9/9] linux-custom: Move cfg fragments applying to debian/rules

[Henning Schild <henning.schild@siemens.com>]

Am Thu, 24 Feb 2022 12:09:00 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:

> In the email from Wednesday, 23 February 2022 16:53:03 +03 user
> Henning Schild wrote:
> > I just ran into an issue with that not actually finding my config in
> > WORKDIR and now trying to go the KBUILD_DEFCONFIG way
> > 
> > Am Sun, 13 Feb 2022 08:41:11 +0100
> > 
> > schrieb Uladzimir Bely <ubely@ilbers.de>:  
> > > Prepare linux fragments applying in Debian way.
> > > 
> > > This allows to avoid pre-build chroot call used for merging
> > > kernel config file and custom config fragments. Also, shell
> > > exports are not used anymore in favor of template files.
> > > 
> > > Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> > > ---
> > > 
> > >  .../linux/files/debian/isar/configure.tmpl    | 19 +++++++
> > >  .../linux/files/debian/rules.tmpl             |  3 ++
> > >  meta/recipes-kernel/linux/linux-custom.inc    | 53
> > > 
> > > ++++++++++++------- 3 files changed, 55 insertions(+), 20
> > > deletions(-)
> > > 
> > >  create mode 100644
> > > 
> > > meta/recipes-kernel/linux/files/debian/isar/configure.tmpl
> > > 
> > > diff --git
> > > a/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl new
> > > file mode 100644 index 00000000..83871baa --- /dev/null
> > > +++ b/meta/recipes-kernel/linux/files/debian/isar/configure.tmpl
> > > @@ -0,0 +1,19 @@
> > > +#!/bin/bash
> > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019
> > > +# SPDX-License-Identifier: MIT
> > > +
> > > +# Load common stuff
> > > +. ${S}/debian/isar/common || exit ${?}
> > > +
> > > +do_configure() {
> > > +
> > > +    # Process kernel config target and fragments
> > > +    ${MAKE} O=${KERNEL_BUILD_DIR} ${KERNEL_CONFIG_TARGET} || exit
> > > ${?}
> > > +    ./scripts/kconfig/merge_config.sh -O ${KERNEL_BUILD_DIR}/ \
> > > +    ${KERNEL_BUILD_DIR}/.config ${KERNEL_CONFIG_FRAGMENTS}
> > > +
> > > +    # Stop tracing
> > > +    set +x
> > > +}
> > > +
> > > +main configure ${*}
> > > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/rules.tmpl index
> > > 05a26fe2..a1166287 100755 ---
> > > a/meta/recipes-kernel/linux/files/debian/rules.tmpl +++
> > > b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -22,6
> > > +22,9 @@
> > > 
> > > isar_env=$(strip \ override_dh_auto_clean:
> > >  	$(isar_env) && bash $(deb_top_dir)/isar/clean
> > > 
> > > +override_dh_auto_configure:
> > > +	$(isar_env) && bash $(deb_top_dir)/isar/configure
> > > +
> > > 
> > >  override_dh_auto_build:
> > >  	$(isar_env) && bash $(deb_top_dir)/isar/build
> > > 
> > > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > > b/meta/recipes-kernel/linux/linux-custom.inc index
> > > 59d42c84..ea1abf76 100644 ---
> > > a/meta/recipes-kernel/linux/linux-custom.inc +++
> > > b/meta/recipes-kernel/linux/linux-custom.inc @@ -52,6 +52,7 @@
> > > TEMPLATE_FILES += "                  \
> > > 
> > >      debian/isar/build.tmpl           \
> > >      debian/isar/clean.tmpl           \
> > >      debian/isar/common.tmpl          \
> > > 
> > > +    debian/isar/configure.tmpl       \
> > > 
> > >      debian/isar/install.tmpl         \
> > >      debian/isar/version.cfg.tmpl     \
> > >      debian/linux-image.postinst.tmpl \
> > > 
> > > @@ -71,6 +72,8 @@ TEMPLATE_VARS += "                \
> > > 
> > >      KERNEL_HEADERS_DEBIAN_DEPENDS \
> > >      LINUX_VERSION_EXTENSION       \
> > >      KERNEL_NAME_PROVIDED          \
> > > 
> > > +    KERNEL_CONFIG_TARGET          \
> > > +    KERNEL_CONFIG_FRAGMENTS       \
> > > 
> > >  "
> > >  
> > >  inherit dpkg
> > > 
> > > @@ -146,41 +149,51 @@ do_prepare_build_prepend() {
> > > 
> > >  # build directory for our "full" kernel build
> > >  KERNEL_BUILD_DIR = "build-full"
> > > 
> > > +def get_kernel_config_target(d):
> > > +    kernel_defconfig = d.getVar('KERNEL_DEFCONFIG', True)
> > > +
> > > +    config_target = kernel_defconfig
> > > +
> > > +    if kernel_defconfig:
> > > +        workdir=d.getVar('WORKDIR', True)  
> >   
> > > +        if os.path.isfile(workdir + "/" + kernel_defconfig):  
> > Doing that at parse-time when assigning the variable
> > KERNEL_CONFIG_TARGET can not work. Some task in the task chain might
> > need to create that file first ... fetch/unpack/fiddle-with
> > 
> > Checking if the file exists at parse time seems very wrong. I guess
> > this patch needs to be reverted and rewritten and i guess the test
> > suite does not properly cover kernels custom defconfig
> >   
> 
> Yes, you are right, this moment is wrong.
> 
> I guess, it can be fixed by passing KERNEL_CONFIG_TARGET to
> 'debian/*' by direct appending from the recipe task (instead of
> template way).
> 
> I'll try prepare a solution soon.

Cool,

i would even suggest a revert because that breaks kernel builds in
layers. I reverted just that one and my layer worked again.

I will look into writing a patch to the testsuite, covering the
olddefconfig path.

Henning

> > This is how one can rebuild a debian kernel
> > 
> > KBUILD_DEPENDS += ",linux-config-${PV}"
> > 
> > KERNEL_DEFCONFIG="${P}.conf"
> > 
> > do_take_debian_config() {
> >     xzcat
> > ${BUILDCHROOT_DIR}/usr/src/linux-config-${PV}/config.${DISTRO_ARCH}_none_${
> > DISTRO_ARCH}.xz > ${S}/${KERNEL_DEFCONFIG} }
> > 
> > addtask take_debian_config after do_install_builddeps before
> > do_dpkg_build
> > 
> > Maybe that could be used for writing a kernel test recipe actually
> > going the "olddefconfig" way in case we do not yet have one.
> > 
> > Henning
> >   
> > > +            config_target = "olddefconfig"
> > > +        else:
> > > +            config_target = "defconfig KBUILD_DEFCONFIG=" +
> > > kernel_defconfig
> > > +    else:
> > > +        config_target = "defconfig"
> > > +
> > > +    return config_target
> > > +
> > > +KERNEL_CONFIG_TARGET = "${@get_kernel_config_target(d)}"
> > > +
> > > +def get_kernel_config_fragments(d):
> > > +    src_frags = " ".join(config_fragments(d))
> > > +    out_frags = " ".join(map(lambda frag: 'debian/fragments/' +
> > > frag, config_fragments(d))) +
> > > +    linux_version_extension = d.getVar('LINUX_VERSION_EXTENSION',
> > > True)
> > > +    if linux_version_extension:
> > > +        out_frags += " debian/isar/version.cfg"
> > > +
> > > +    return out_frags
> > > +
> > > +KERNEL_CONFIG_FRAGMENTS = "${@get_kernel_config_fragments(d)}"
> > > +
> > > 
> > >  dpkg_configure_kernel() {
> > > 
> > > -	config_target="${KERNEL_DEFCONFIG}"
> > > 
> > >  	rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p
> > > 
> > > ${S}/${KERNEL_BUILD_DIR} if [ -n "${KERNEL_DEFCONFIG}" ]; then
> > > 
> > >  		if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then
> > >  		
> > >  			cp ${WORKDIR}/${KERNEL_DEFCONFIG}
> > > 
> > > ${S}/${KERNEL_BUILD_DIR}/.config
> > > -			config_target="olddefconfig"
> > > -		else
> > > -			config_target="defconfig
> > > KBUILD_DEFCONFIG=${KERNEL_DEFCONFIG}" fi
> > > -	else
> > > -		config_target="defconfig"
> > > 
> > >  	fi
> > >  	
> > >  	# copy config fragments over to the kernel tree
> > >  	src_frags="${@ " ".join(config_fragments(d)) }"
> > > 
> > > -	out_frags="${@ " ".join(map(lambda frag:
> > > 'debian/fragments/'
> > > + frag, config_fragments(d))) }"
> > > -	if [ -n "${LINUX_VERSION_EXTENSION}" ]; then
> > > -		out_frags="${out_frags} debian/isar/version.cfg"
> > > -	fi
> > > 
> > >  	rm -rf ${S}/debian/fragments
> > >  	if [ -n "${src_frags}" ]; then
> > >  	
> > >  		mkdir -p ${S}/debian/fragments
> > >  		(cd ${WORKDIR} && cp ${src_frags}
> > > 
> > > ${S}/debian/fragments/) fi
> > > -
> > > -	sudo -E chroot --userspec=$(id -u):$(id -g)
> > > ${BUILDCHROOT_DIR} sh -c " \
> > > -		export ARCH=${KERNEL_ARCH} &&
> > > 
> > >           \
> > > 
> > > -		cd ${PP}/${PPS} &&
> > > 
> > >           \
> > > 
> > > -		make O=${KERNEL_BUILD_DIR} ${config_target} &&
> > > 
> > >           \
> > > 
> > > -		./scripts/kconfig/merge_config.sh
> > > 
> > >           \
> > > 
> > > -			-O ${KERNEL_BUILD_DIR}/
> > > 
> > >           \
> > > 
> > > -			${KERNEL_BUILD_DIR}/.config
> > > 
> > >           \
> > > 
> > > -			${out_frags}
> > > 
> > >           \
> > > 
> > > -        "
> > > 
> > >  }
> > >  
> > >  dpkg_runbuild_prepend() {  
> 
>