Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages

["Koch, Stefan" <stefan-koch@siemens.com>]

On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
> On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
> > > On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > > Hi
> > > > 
> > > > This patchset is a set of three patches:
> > > > - Support overwriting configured schroot dir
> > > > - Split up binaries from kernel headers to kbuild packages
> > > > - Update custom_kernel docs for split up of kernel scripts and
> > > > tools
> > > > 
> > > > The main use-case was to swap out the binaries
> > > > from the kernel headers package.
> > > > They will be swapped out into host and target
> > > > specific kernel kbuild packages.
> > > > 
> > > > The main development goals were these:
> > > > 
> > > > 1. Solve already known isar custom kernel
> > > > limitations from doc/custom_kernel.inc
> > > > - kernel headers package does not support both native
> > > >   and cross compilation of kernel modules when cross built
> > > 
> > > Right, that requires building packages for multiple
> > > architectures.
> > 
> > Important feature... Maybe it should be added to the TODO list in
> > the
> > docs?
> > 
> > > This
> > > is the feature we need, and it has nothing to do with the kernel
> > > packages per se.
> > 
> > The way that I assume debian will do building the kernel is that
> > they
> > use a generic debian defconfig for each architecture.
> > E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
> > 5.10/config.arm64_none_arm64.xz
> > 
> > ISAR builds are often using specific defconfigs honored by ISAR's
> > linux-custom for the target build.
> > 
> > Debian specific architecture builds (e. g. host and target, maybe
> > with
> > sbuild) for one target device, using target specific stripped-down
> > defconfigs, may not working properly. The kernel build for a
> > foreign
> > architecture (e. g. host) may fail because of an incompatible
> > defconfig.
> > 
> > Debian provide a generic very similar defconfig for different
> > architectures, and they build the linux-image, linux-headers and
> > linux-
> > kbuild for each architecture:
> > - So I assume the linux-kbuild:amd64 was build with linux-config-
> > 5.10/config.amd64_none_amd64.xz
> > - So I assume the linux-kbuild:arm64 was build with linux-config-
> > 5.10/config.arm64_none_arm64.xz
> > 
> > The debian generic defconfig may differ in some arch specific
> > points
> > but you can use the linux-kbuild:amd64 for a cross-build of linux-
> > image:arm64 because the resulting scripts and tools are from the
> > identical set. So mixing linux-headers with different archictecture
> > specific builds from linux-kbuild should work since the defconfigs
> > have
> > a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
> > needed for recordmcount utility as example).
> 
> Sorry, I don't understand what role the config plays here. Isar is
> used
> to compile (usually) one specific kernel, Debian compiles a few
> generic
> ones. But the fact that Debian compile everything natively (for
> production) and we do cross-compilation is not related to different
> configs.
> 

I just wanted to point out for the future case when ISAR would support
a generic way to build packages for multiple architectures: This might
not be applicable to all kernel builds. E. g. because of the fact that
the used defconfig for a "specific kernel" is not compatible for a
multi architecture build, only for the architecture that the defconfig
was made for.

Yes, it may work if you could specify different defconfigs for each
architecture. One goal of this patch was to provide the kbuild scripts
and tools just for host and target in a transparent way - without any
changes to the build process of existing ISAR users.

> The answer to the problem "I need kernel headers on the target" right
> now is in Isar to build natively the kernel and, thus, the header
> package. Not optimal, but we still need this case too rarely to have
> cared about it more so far.
> 
> > 
> > > Jan
> > > 
> > > > 
> > > > 2. Honor recommendations for future from doc/custom_kernel.inc
> > > > - Generate kernel headers packages for both host and target
> > > >   when using cross build
> > > > 
> > > > 3. Add extensions known from debian kernel packages structure
> > > > - Generate a kernel headers package without binaries
> > > > - Create specific kernel kbuild packages that
> > > >   will ship the "scripts" and "tools" binaries
> > > > - Use symlinks to point to the "scripts" and "tools" binaries
> > > 
> > > Keep in mind that we intend to remain compatible with the layout
> > > of
> > > Debian. Self-built kernels should remain drop-in replacements of
> > > the
> > > Debian kernel packages. And that is true for development packages
> > > as
> > > well.
> > > 
> > > So, are your changes working against that, or are they improving
> > > existing incompatibility with the debian packages?
> > 
> > The approach from this patchset does build the kernel in the known
> > ISAR
> > specific way. Debian itself patches the kernel Makefiles to
> > modularize
> > the kernel build what ISAR doesn't do. With this patch the packages
> > linux-kbuild and linux-kbuild-cross are the only new packages that
> > the
> > ISAR build will provide. That means ISAR will provide then linux-
> > headers, linux-image and linux-kbuild. They match on the target
> > with
> > the package name of the respective debian packages. So the existing
> > drop-in should not be affected.
> > 
> > In short this patch brings the ISAR linux-custom mechanism a bit
> > nearer
> > to the debian package structure. Since linux-headers behaves then
> > more
> > like the original debian linux-heades to ship the binaries within a
> > separate package -> linux-kbuild.
> > 
> > The trick is to build only the scripts and tools with schroot-
> > target
> > (QEMU) in a non-cross way. Since the kernel does not support
> > building
> > the scripts and tools for target architecture when do a cross build
> > without patching the kernel Makefiles. I think that patch was never
> > merged into the kernel:
> >  
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fproject%2Flinux-kbuild%2Fpatch%2F1376046432-12588-1-git-send-email-broonie%40kernel.org%2F&data=05%7C01%7Cstefan-koch%40siemens.com%7C5054591149e44251f36d08dac26b7b61%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C638036064125400893%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iPeO%2FkFjrLNSGQJ83fNsifWNI0wbPzFsWTIi%2B1fZ9PI%3D&reserved=0
> > 
> 
> The problem is that patch might have to not provide that use case I
> wrote about above. It is a rare use case, a non-distro use case, but
> I
> wouldn't call it an invalid one. If we can address it by splitting up
> the Isar kernel build into different packages and then only compile
> the
> binary bits once again, now for the target architecture, that would
> be a
> nice solution.

That's what the patch do:
- Move the binary scripts and tools from linux-headers to linux-kbuild
as debian does it for their kernels
- Compile only the scripts and tools for the target using schroot-
target (QEMU) and package it as linux-kbuild
- Package the binary scripts and tools from the full cross kernel build
to linux-kbuild-cross using schroot-host

> 
> On top, we should resolve the Isar issue that you can only build one
> package in one build for one architecture, not multiples. But I don't
> want special-casing the kernel build beyond that because the problem
> - I
> need dev tools both for my Isar build and later also for the target -
> is
> generic, even though still rare in your deployments.

This is a nice and important goal. Maybe ISAR is now nearer to this
since it has changed to use sbuild now.

> 
> Jan
> 

-- 
Stefan Koch
Siemens AG
www.siemens.com