Re: [PATCH v2 1/5] linux-custom: Split up binaries from kernel headers to kbuild package

["Koch, Stefan" <stefan-koch@siemens.com>]

On Fri, 2023-03-17 at 09:18 +0100, Jan Kiszka wrote:
> On 20.12.22 18:09, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > This patch does introduce specific kernel kbuild packages that
> > ship the "scripts" and "tools" binaries.
> > The kernel headers fulfill this using symlinks to point
> > to the "scripts" and "tools" of the kernel kbuild package.
> > 
> > This is already known from debian kernel packages structure:
> > - Generate a kernel headers package without binaries
> > - Create kernel kbuild packages that
> >   ship the "scripts" and "tools" binaries
> > - Using symlinks to point to the "scripts"
> >   and "tools" binaries
> > 
> > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > ---
> >  .../linux/files/debian/control.tmpl           |  9 +++-
> >  .../linux/files/debian/isar/common.tmpl       |  2 +
> >  .../linux/files/debian/isar/install.tmpl      | 51
> > +++++++++++++++----
> >  meta/recipes-kernel/linux/linux-custom.inc    |  4 ++
> >  4 files changed, 54 insertions(+), 12 deletions(-)
> > 
> > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > index dd0b624..b19ca2c 100644
> > --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > @@ -26,7 +26,7 @@ Section: devel
> >  Provides: linux-kernel-headers
> >  Architecture: any
> >  Description: Linux support headers for userspace development
> > - This package provides userspaces headers from the Linux kernel. 
> > These headers
> > + This package provides userspaces headers from the Linux kernel.
> > These headers
> >   are used by the installed headers for GNU glibc and other system
> > libraries.
> >  
> >  Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> > @@ -35,3 +35,10 @@ Architecture: any
> >  Description: Linux kernel debugging symbols for @KR@
> >   This package will come in handy if you need to debug the kernel.
> > It provides
> >   all the necessary debug symbols for the kernel and its modules.
> > +
> > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> > +Architecture: any
> > +Depends: ${perl:Depends}, ${shlib:Depends}
> > +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > tools for @KR@
> > + This package provides kernel kbuild scripts and tools for @KR@
> > + This is useful for people who need to build external modules
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-
> > kernel/linux/files/debian/isar/common.tmpl
> > index 52ebebb..f4c0519 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > @@ -9,6 +9,7 @@ ARCH=${KERNEL_ARCH}
> >  KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
> >  KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
> >  KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> > +KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> >  
> >  # Constants
> >  KCONF=.config
> > @@ -19,6 +20,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
> >  deb_dbg_dir=${deb_img_dir}-dbg
> >  deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
> >  deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
> > +deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
> >  
> >  # Array of packages to be generated
> >  declare -A kern_pkgs
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-
> > kernel/linux/files/debian/isar/install.tmpl
> > index 8a604e4..236b67c 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > @@ -33,6 +33,7 @@ do_install() {
> >      # Trace what we do here
> >      set -x
> >  
> > +    # Run the install steps
> >      install_image
> >      if [ "${ARCH}" != "um" ]; then
> >          install_config
> > @@ -43,6 +44,10 @@ do_install() {
> >      install_kmods
> >      install_headers
> >  
> > +    # Cleanup and install kernel scripts and tools
> > +    rm -rf ${deb_kern_kbuild_dir}
> > +    install_kbuild ${deb_kern_kbuild_dir}
> > +
> >      # Stop tracing
> >      set +x
> >  }
> > @@ -168,21 +173,15 @@ kernel_headers() {
> >      mkdir -p ${destdir}
> >      mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
> >  
> > -    (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -
> > name '*.pl') >>${src_hdr_files}
> > -    (cd ${S}; find arch/*/include include scripts -type f -o -type
> > l) >>${src_hdr_files}
> > +    (cd ${S}; find . -not -path './scripts/*' -a -not -path
> > './tools/*' -a \( -name 'Makefile*' -o -name 'Kconfig*' -o -name
> > '*.pl' \)) >>${src_hdr_files}
> > +    (cd ${S}; find arch/*/include include -type f -o -type l)
> > >>${src_hdr_files}
> >      (cd ${S}; find arch/${ARCH} -name module.lds -o -name
> > Kbuild.platforms -o -name Platform) >>${src_hdr_files}
> >      (cd ${S}; find $(find arch/${ARCH} -name include -o -name
> > scripts -type d) -type f) >>${src_hdr_files}
> >  
> >      if [ -n "${CONFIG_MODULES}" ]; then
> >          echo Module.symvers >> ${obj_hdr_files}
> >      fi
> > -    (cd ${O}; find arch/${ARCH}/include include scripts -type f)
> > >>${obj_hdr_files}
> > -    if [ -n "${CONFIG_STACK_VALIDATION}" -o -n
> > "${CONFIG_HAVE_OBJTOOL}" ]; then
> > -        (cd ${O}; find tools/objtool -type f -executable)
> > >>${obj_hdr_files}
> > -    fi
> > -    if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> > -        (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> > gcc-common.h) >>${obj_hdr_files}
> > -    fi
> > +    (cd ${O}; find arch/${ARCH}/include include -type f)
> > >>${obj_hdr_files}
> >  
> >      # deploy files that were matched above
> >      tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -
> > xf -
> > @@ -191,8 +190,11 @@ kernel_headers() {
> >      # add the kernel config
> >      cp ${O}/${KCONF} ${destdir}/.config
> >  
> > -    # handle kernel development tools
> > -    kernel_tools
> > +    # add symlink to scripts and tools directories
> > +    ln -sf ../../lib/linux-kbuild-${krel}/scripts
> > ${destdir}/scripts
> > +    if [ -n "${CONFIG_STACK_VALIDATION}" -o -n
> > "${CONFIG_HAVE_OBJTOOL}" ]; then
> > +        ln -sf ../../lib/linux-kbuild-${krel}/tools
> > ${destdir}/tools
> 
> Is Debian using relative links for this as well? Or rather absolute
> ones. We use a absolute one below.

Debian itself uses relative ones...

ls -l /usr/src/linux-headers-6.0.0-0.deb11.6-amd64/
total 1660
drwxr-xr-x 3 root root    4096 Mar  2 11:14 arch
drwxr-xr-x 4 root root    4096 Mar  2 11:14 include
-rw-r--r-- 1 root root      63 Dec 19 15:14 Makefile
-rw-r--r-- 1 root root 1684046 Dec 19 15:14 Module.symvers
lrwxrwxrwx 1 root root      34 Dec 19 15:14 scripts -> ../../lib/linux-
kbuild-6.0/scripts
lrwxrwxrwx 1 root root      32 Dec 19 15:14 tools -> ../../lib/linux-
kbuild-6.0/tools

> 
> > +    fi
> >  
> >      # create symlinks
> >      ln -sf /${kernel_headers_dir}
> > ${deb_kern_hdr_dir}/lib/modules/${krel}/build
> > @@ -206,4 +208,31 @@ install_headers() {
> >      kernel_headers
> >  }
> >  
> > +install_kbuild() {
> > +    kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
> > +    destdir=${1}/${kernel_kbuild_dir}
> > +    src_kbuild_files=$(mktemp)
> > +    obj_kbuild_files=$(mktemp)
> > +
> > +    mkdir -p ${destdir}
> > +
> > +    (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a
> > \( -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \))
> > >>${src_kbuild_files}
> > +    (cd ${S}; find scripts -type f -o -type l)
> > >>${src_kbuild_files}
> > +
> > +    (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
> > +    if [ -n "${CONFIG_STACK_VALIDATION}" -o -n
> > "${CONFIG_HAVE_OBJTOOL}" ]; then
> > +        (cd ${O}; find tools/objtool -type f -executable)
> > >>${obj_kbuild_files}
> > +    fi
> > +    if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> > +        (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> > gcc-common.h) >>${obj_kbuild_files}
> > +    fi
> > +
> > +    # deploy files that were matched above
> > +    tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C
> > ${destdir} -xf -
> > +    tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C
> > ${destdir} -xf -
> > +
> > +    # handle kernel development tools
> > +    kernel_tools
> > +}
> > +
> >  main install ${*}
> > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > b/meta/recipes-kernel/linux/linux-custom.inc
> > index 96f0afc..dbda755 100644
> > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > @@ -96,6 +96,10 @@ python() {
> >      kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True)
> >      d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> >                           'linux-headers-' + kernel_name)
> > +    headers_depends = "linux-kbuild-" + kernel_name
> > +
> > +    # Set dependency for kernel headers
> > +    d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
> >  }
> >  
> >  def get_kernel_arch(d):
> 
> Except for the one question above, this commit looks good to me. But
> maybe Cedric has some comments as well, given that he once wrote
> significant parts of this.
> 
> Jan
> 

-- 
Stefan Koch
Siemens AG
www.siemens.com