Re: [PATCH v2 1/5] linux-custom: Split up binaries from kernel headers to kbuild package

[Jan Kiszka <jan.kiszka@siemens.com>]

On 17.03.23 10:20, Koch, Stefan (DI PA DCP R&D 3) wrote:
> On Fri, 2023-03-17 at 09:18 +0100, Jan Kiszka wrote:
>> On 20.12.22 18:09, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>> This patch does introduce specific kernel kbuild packages that
>>> ship the "scripts" and "tools" binaries.
>>> The kernel headers fulfill this using symlinks to point
>>> to the "scripts" and "tools" of the kernel kbuild package.
>>>
>>> This is already known from debian kernel packages structure:
>>> - Generate a kernel headers package without binaries
>>> - Create kernel kbuild packages that
>>>   ship the "scripts" and "tools" binaries
>>> - Using symlinks to point to the "scripts"
>>>   and "tools" binaries
>>>
>>> Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
>>> ---
>>>  .../linux/files/debian/control.tmpl           |  9 +++-
>>>  .../linux/files/debian/isar/common.tmpl       |  2 +
>>>  .../linux/files/debian/isar/install.tmpl      | 51
>>> +++++++++++++++----
>>>  meta/recipes-kernel/linux/linux-custom.inc    |  4 ++
>>>  4 files changed, 54 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> b/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> index dd0b624..b19ca2c 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> @@ -26,7 +26,7 @@ Section: devel
>>>  Provides: linux-kernel-headers
>>>  Architecture: any
>>>  Description: Linux support headers for userspace development
>>> - This package provides userspaces headers from the Linux kernel. 
>>> These headers
>>> + This package provides userspaces headers from the Linux kernel.
>>> These headers
>>>   are used by the installed headers for GNU glibc and other system
>>> libraries.
>>>  
>>>  Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
>>> @@ -35,3 +35,10 @@ Architecture: any
>>>  Description: Linux kernel debugging symbols for @KR@
>>>   This package will come in handy if you need to debug the kernel.
>>> It provides
>>>   all the necessary debug symbols for the kernel and its modules.
>>> +
>>> +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
>>> +Architecture: any
>>> +Depends: ${perl:Depends}, ${shlib:Depends}
>>> +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
>>> tools for @KR@
>>> + This package provides kernel kbuild scripts and tools for @KR@
>>> + This is useful for people who need to build external modules
>>> diff --git a/meta/recipes-
>>> kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-
>>> kernel/linux/files/debian/isar/common.tmpl
>>> index 52ebebb..f4c0519 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
>>> @@ -9,6 +9,7 @@ ARCH=${KERNEL_ARCH}
>>>  KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
>>>  KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
>>>  KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
>>> +KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
>>>  
>>>  # Constants
>>>  KCONF=.config
>>> @@ -19,6 +20,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
>>>  deb_dbg_dir=${deb_img_dir}-dbg
>>>  deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
>>>  deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
>>> +deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
>>>  
>>>  # Array of packages to be generated
>>>  declare -A kern_pkgs
>>> diff --git a/meta/recipes-
>>> kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-
>>> kernel/linux/files/debian/isar/install.tmpl
>>> index 8a604e4..236b67c 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
>>> @@ -33,6 +33,7 @@ do_install() {
>>>      # Trace what we do here
>>>      set -x
>>>  
>>> +    # Run the install steps
>>>      install_image
>>>      if [ "${ARCH}" != "um" ]; then
>>>          install_config
>>> @@ -43,6 +44,10 @@ do_install() {
>>>      install_kmods
>>>      install_headers
>>>  
>>> +    # Cleanup and install kernel scripts and tools
>>> +    rm -rf ${deb_kern_kbuild_dir}
>>> +    install_kbuild ${deb_kern_kbuild_dir}
>>> +
>>>      # Stop tracing
>>>      set +x
>>>  }
>>> @@ -168,21 +173,15 @@ kernel_headers() {
>>>      mkdir -p ${destdir}
>>>      mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
>>>  
>>> -    (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -
>>> name '*.pl') >>${src_hdr_files}
>>> -    (cd ${S}; find arch/*/include include scripts -type f -o -type
>>> l) >>${src_hdr_files}
>>> +    (cd ${S}; find . -not -path './scripts/*' -a -not -path
>>> './tools/*' -a \( -name 'Makefile*' -o -name 'Kconfig*' -o -name
>>> '*.pl' \)) >>${src_hdr_files}
>>> +    (cd ${S}; find arch/*/include include -type f -o -type l)
>>>>> ${src_hdr_files}
>>>      (cd ${S}; find arch/${ARCH} -name module.lds -o -name
>>> Kbuild.platforms -o -name Platform) >>${src_hdr_files}
>>>      (cd ${S}; find $(find arch/${ARCH} -name include -o -name
>>> scripts -type d) -type f) >>${src_hdr_files}
>>>  
>>>      if [ -n "${CONFIG_MODULES}" ]; then
>>>          echo Module.symvers >> ${obj_hdr_files}
>>>      fi
>>> -    (cd ${O}; find arch/${ARCH}/include include scripts -type f)
>>>>> ${obj_hdr_files}
>>> -    if [ -n "${CONFIG_STACK_VALIDATION}" -o -n
>>> "${CONFIG_HAVE_OBJTOOL}" ]; then
>>> -        (cd ${O}; find tools/objtool -type f -executable)
>>>>> ${obj_hdr_files}
>>> -    fi
>>> -    if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
>>> -        (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
>>> gcc-common.h) >>${obj_hdr_files}
>>> -    fi
>>> +    (cd ${O}; find arch/${ARCH}/include include -type f)
>>>>> ${obj_hdr_files}
>>>  
>>>      # deploy files that were matched above
>>>      tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -
>>> xf -
>>> @@ -191,8 +190,11 @@ kernel_headers() {
>>>      # add the kernel config
>>>      cp ${O}/${KCONF} ${destdir}/.config
>>>  
>>> -    # handle kernel development tools
>>> -    kernel_tools
>>> +    # add symlink to scripts and tools directories
>>> +    ln -sf ../../lib/linux-kbuild-${krel}/scripts
>>> ${destdir}/scripts
>>> +    if [ -n "${CONFIG_STACK_VALIDATION}" -o -n
>>> "${CONFIG_HAVE_OBJTOOL}" ]; then
>>> +        ln -sf ../../lib/linux-kbuild-${krel}/tools
>>> ${destdir}/tools
>>
>> Is Debian using relative links for this as well? Or rather absolute
>> ones. We use a absolute one below.
> 
> Debian itself uses relative ones...
> 
> ls -l /usr/src/linux-headers-6.0.0-0.deb11.6-amd64/
> total 1660
> drwxr-xr-x 3 root root    4096 Mar  2 11:14 arch
> drwxr-xr-x 4 root root    4096 Mar  2 11:14 include
> -rw-r--r-- 1 root root      63 Dec 19 15:14 Makefile
> -rw-r--r-- 1 root root 1684046 Dec 19 15:14 Module.symvers
> lrwxrwxrwx 1 root root      34 Dec 19 15:14 scripts -> ../../lib/linux-
> kbuild-6.0/scripts
> lrwxrwxrwx 1 root root      32 Dec 19 15:14 tools -> ../../lib/linux-
> kbuild-6.0/tools
> 

Perfect, then we are good with the patch as-is.

Jan

>>
>>> +    fi
>>>  
>>>      # create symlinks
>>>      ln -sf /${kernel_headers_dir}
>>> ${deb_kern_hdr_dir}/lib/modules/${krel}/build
>>> @@ -206,4 +208,31 @@ install_headers() {
>>>      kernel_headers
>>>  }
>>>  
>>> +install_kbuild() {
>>> +    kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
>>> +    destdir=${1}/${kernel_kbuild_dir}
>>> +    src_kbuild_files=$(mktemp)
>>> +    obj_kbuild_files=$(mktemp)
>>> +
>>> +    mkdir -p ${destdir}
>>> +
>>> +    (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a
>>> \( -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \))
>>>>> ${src_kbuild_files}
>>> +    (cd ${S}; find scripts -type f -o -type l)
>>>>> ${src_kbuild_files}
>>> +
>>> +    (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
>>> +    if [ -n "${CONFIG_STACK_VALIDATION}" -o -n
>>> "${CONFIG_HAVE_OBJTOOL}" ]; then
>>> +        (cd ${O}; find tools/objtool -type f -executable)
>>>>> ${obj_kbuild_files}
>>> +    fi
>>> +    if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
>>> +        (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
>>> gcc-common.h) >>${obj_kbuild_files}
>>> +    fi
>>> +
>>> +    # deploy files that were matched above
>>> +    tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C
>>> ${destdir} -xf -
>>> +    tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C
>>> ${destdir} -xf -
>>> +
>>> +    # handle kernel development tools
>>> +    kernel_tools
>>> +}
>>> +
>>>  main install ${*}
>>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc
>>> b/meta/recipes-kernel/linux/linux-custom.inc
>>> index 96f0afc..dbda755 100644
>>> --- a/meta/recipes-kernel/linux/linux-custom.inc
>>> +++ b/meta/recipes-kernel/linux/linux-custom.inc
>>> @@ -96,6 +96,10 @@ python() {
>>>      kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True)
>>>      d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
>>>                           'linux-headers-' + kernel_name)
>>> +    headers_depends = "linux-kbuild-" + kernel_name
>>> +
>>> +    # Set dependency for kernel headers
>>> +    d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
>>>  }
>>>  
>>>  def get_kernel_arch(d):
>>
>> Except for the one question above, this commit looks good to me. But
>> maybe Cedric has some comments as well, given that he once wrote
>> significant parts of this.
>>
>> Jan
>>
> 

-- 
Siemens AG, Technology
Competence Center Embedded Linux