Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages

["Koch, Stefan" <stefan-koch@siemens.com>]

On Thu, 2022-11-10 at 19:36 +0100, Jan Kiszka wrote:
> On 10.11.22 19:33, Jan Kiszka wrote:
> > On 10.11.22 18:49, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
> > > > On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > > > On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
> > > > > > On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > > > > > Hi
> > > > > > > 
> > > > > > > This patchset is a set of three patches:
> > > > > > > - Support overwriting configured schroot dir
> > > > > > > - Split up binaries from kernel headers to kbuild
> > > > > > > packages
> > > > > > > - Update custom_kernel docs for split up of kernel
> > > > > > > scripts and
> > > > > > > tools
> > > > > > > 
> > > > > > > The main use-case was to swap out the binaries
> > > > > > > from the kernel headers package.
> > > > > > > They will be swapped out into host and target
> > > > > > > specific kernel kbuild packages.
> > > > > > > 
> > > > > > > The main development goals were these:
> > > > > > > 
> > > > > > > 1. Solve already known isar custom kernel
> > > > > > > limitations from doc/custom_kernel.inc
> > > > > > > - kernel headers package does not support both native
> > > > > > >   and cross compilation of kernel modules when cross
> > > > > > > built
> > > > > > 
> > > > > > Right, that requires building packages for multiple
> > > > > > architectures.
> > > > > 
> > > > > Important feature... Maybe it should be added to the TODO
> > > > > list in
> > > > > the
> > > > > docs?
> > > > > 
> > > > > > This
> > > > > > is the feature we need, and it has nothing to do with the
> > > > > > kernel
> > > > > > packages per se.
> > > > > 
> > > > > The way that I assume debian will do building the kernel is
> > > > > that
> > > > > they
> > > > > use a generic debian defconfig for each architecture.
> > > > > E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-
> > > > > config-
> > > > > 5.10/config.arm64_none_arm64.xz
> > > > > 
> > > > > ISAR builds are often using specific defconfigs honored by
> > > > > ISAR's
> > > > > linux-custom for the target build.
> > > > > 
> > > > > Debian specific architecture builds (e. g. host and target,
> > > > > maybe
> > > > > with
> > > > > sbuild) for one target device, using target specific
> > > > > stripped-down
> > > > > defconfigs, may not working properly. The kernel build for a
> > > > > foreign
> > > > > architecture (e. g. host) may fail because of an incompatible
> > > > > defconfig.
> > > > > 
> > > > > Debian provide a generic very similar defconfig for different
> > > > > architectures, and they build the linux-image, linux-headers
> > > > > and
> > > > > linux-
> > > > > kbuild for each architecture:
> > > > > - So I assume the linux-kbuild:amd64 was build with linux-
> > > > > config-
> > > > > 5.10/config.amd64_none_amd64.xz
> > > > > - So I assume the linux-kbuild:arm64 was build with linux-
> > > > > config-
> > > > > 5.10/config.arm64_none_arm64.xz
> > > > > 
> > > > > The debian generic defconfig may differ in some arch specific
> > > > > points
> > > > > but you can use the linux-kbuild:amd64 for a cross-build of
> > > > > linux-
> > > > > image:arm64 because the resulting scripts and tools are from
> > > > > the
> > > > > identical set. So mixing linux-headers with different
> > > > > archictecture
> > > > > specific builds from linux-kbuild should work since the
> > > > > defconfigs
> > > > > have
> > > > > a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT
> > > > > thats
> > > > > needed for recordmcount utility as example).
> > > > 
> > > > Sorry, I don't understand what role the config plays here. Isar
> > > > is
> > > > used
> > > > to compile (usually) one specific kernel, Debian compiles a few
> > > > generic
> > > > ones. But the fact that Debian compile everything natively (for
> > > > production) and we do cross-compilation is not related to
> > > > different
> > > > configs.
> > > > 
> > > 
> > > I just wanted to point out for the future case when ISAR would
> > > support
> > > a generic way to build packages for multiple architectures: This
> > > might
> > > not be applicable to all kernel builds. E. g. because of the fact
> > > that
> > > the used defconfig for a "specific kernel" is not compatible for
> > > a
> > > multi architecture build, only for the architecture that the
> > > defconfig
> > > was made for.
> > > 
> > > Yes, it may work if you could specify different defconfigs for
> > > each
> > > architecture. One goal of this patch was to provide the kbuild
> > > scripts
> > > and tools just for host and target in a transparent way - without
> > > any
> > > changes to the build process of existing ISAR users.
> > 
> > Misunderstanding: I didn't meant to propose building the kernel-
> > image
> > package for multiple arch, only the then factored-out kbuild
> > packages
> > with the host tool (once for the build env, once for the target).
> > 

Maybe then some logic that detects cross-build could use for the target
build with "make scripts tools" only...

> > > 
> > > > The answer to the problem "I need kernel headers on the target"
> > > > right
> > > > now is in Isar to build natively the kernel and, thus, the
> > > > header
> > > > package. Not optimal, but we still need this case too rarely to
> > > > have
> > > > cared about it more so far.
> > > > 
> > > > > 
> > > > > > Jan
> > > > > > 
> > > > > > > 
> > > > > > > 2. Honor recommendations for future from
> > > > > > > doc/custom_kernel.inc
> > > > > > > - Generate kernel headers packages for both host and
> > > > > > > target
> > > > > > >   when using cross build
> > > > > > > 
> > > > > > > 3. Add extensions known from debian kernel packages
> > > > > > > structure
> > > > > > > - Generate a kernel headers package without binaries
> > > > > > > - Create specific kernel kbuild packages that
> > > > > > >   will ship the "scripts" and "tools" binaries
> > > > > > > - Use symlinks to point to the "scripts" and "tools"
> > > > > > > binaries
> > > > > > 
> > > > > > Keep in mind that we intend to remain compatible with the
> > > > > > layout
> > > > > > of
> > > > > > Debian. Self-built kernels should remain drop-in
> > > > > > replacements of
> > > > > > the
> > > > > > Debian kernel packages. And that is true for development
> > > > > > packages
> > > > > > as
> > > > > > well.
> > > > > > 
> > > > > > So, are your changes working against that, or are they
> > > > > > improving
> > > > > > existing incompatibility with the debian packages?
> > > > > 
> > > > > The approach from this patchset does build the kernel in the
> > > > > known
> > > > > ISAR
> > > > > specific way. Debian itself patches the kernel Makefiles to
> > > > > modularize
> > > > > the kernel build what ISAR doesn't do. With this patch the
> > > > > packages
> > > > > linux-kbuild and linux-kbuild-cross are the only new packages
> > > > > that
> > > > > the
> > > > > ISAR build will provide. That means ISAR will provide then
> > > > > linux-
> > > > > headers, linux-image and linux-kbuild. They match on the
> > > > > target
> > > > > with
> > > > > the package name of the respective debian packages. So the
> > > > > existing
> > > > > drop-in should not be affected.
> > > > > 
> > > > > In short this patch brings the ISAR linux-custom mechanism a
> > > > > bit
> > > > > nearer
> > > > > to the debian package structure. Since linux-headers behaves
> > > > > then
> > > > > more
> > > > > like the original debian linux-heades to ship the binaries
> > > > > within a
> > > > > separate package -> linux-kbuild.
> > > > > 
> > > > > The trick is to build only the scripts and tools with
> > > > > schroot-
> > > > > target
> > > > > (QEMU) in a non-cross way. Since the kernel does not support
> > > > > building
> > > > > the scripts and tools for target architecture when do a cross
> > > > > build
> > > > > without patching the kernel Makefiles. I think that patch was
> > > > > never
> > > > > merged into the kernel:
> > > > >  
> > > > >  
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fproject%2Flinux-kbuild%2Fpatch%2F1376046432-12588-1-git-send-email-broonie%40kernel.org%2F&data=05%7C01%7Cstefan-koch%40siemens.com%7C8ac82ead88ae49e7854308dac34a8659%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C638037022082153110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MgLc%2Bh9ygiMGt2sXXl2CT9rPscIaFFr1f4Y2s5xqGIs%3D&reserved=0
> > > > > 
> > > > 
> > > > The problem is that patch might have to not provide that use
> > > > case I
> > > > wrote about above. It is a rare use case, a non-distro use
> > > > case, but
> > > > I
> > > > wouldn't call it an invalid one. If we can address it by
> > > > splitting up
> > > > the Isar kernel build into different packages and then only
> > > > compile
> > > > the
> > > > binary bits once again, now for the target architecture, that
> > > > would
> > > > be a
> > > > nice solution.
> > > 
> > > That's what the patch do:
> > > - Move the binary scripts and tools from linux-headers to linux-
> > > kbuild
> > > as debian does it for their kernels
> > > - Compile only the scripts and tools for the target using
> > > schroot-
> > > target (QEMU) and package it as linux-kbuild
> > > - Package the binary scripts and tools from the full cross kernel
> > > build
> > > to linux-kbuild-cross using schroot-host
> > 
> > It's moving in a good direction - but why linux-kbuild-cross?
> > Nothing
> > depends on it in your patches, thus module builds will be broken.

Module build should work since linux-headers package depends on "linux-
kbuild-cross | linux-kbuild"

I have successfully cross build some module using the linux-module
recipe...

> > I
> > suspect we rather need
> > 
> > KERNEL_PKG_KERN_KBUILD=linux-kbuild-
> > ${KERNEL_NAME_PROVIDED}:<native>
> > 
> > and therefore (on amd64) linux-kbuild-${KERNEL_NAME_PROVIDED}:amd64
> > to
> > be built and then installed for module builds.
> > 

See my answer to [PATCH 2/3]...

> > Jan
> > 
> 
> And regarding structure of the series: please split up the creation
> of
> kbuild package from also building it for the target arch - if
> possible.

- Move the binary scripts and tools from linux-headers to linux-kbuild
as debian does it for their kernels
- Compile only the scripts and tools for the target using schroot-
target (QEMU) and package it as linux-kbuild

I would suggest this structure using two instead of one patch... Should
the target kbuild creation be toggleable? When yes, enabled or disabled
by default?

> 
> Jan
> 

-- 
Stefan Koch
Siemens AG
www.siemens.com