Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages

[Jan Kiszka <jan.kiszka@siemens.com>]

On 10.11.22 18:49, Koch, Stefan (DI PA DCP R&D 3) wrote:
> On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
>> On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>> On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
>>>> On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>>>> Hi
>>>>>
>>>>> This patchset is a set of three patches:
>>>>> - Support overwriting configured schroot dir
>>>>> - Split up binaries from kernel headers to kbuild packages
>>>>> - Update custom_kernel docs for split up of kernel scripts and
>>>>> tools
>>>>>
>>>>> The main use-case was to swap out the binaries
>>>>> from the kernel headers package.
>>>>> They will be swapped out into host and target
>>>>> specific kernel kbuild packages.
>>>>>
>>>>> The main development goals were these:
>>>>>
>>>>> 1. Solve already known isar custom kernel
>>>>> limitations from doc/custom_kernel.inc
>>>>> - kernel headers package does not support both native
>>>>>   and cross compilation of kernel modules when cross built
>>>>
>>>> Right, that requires building packages for multiple
>>>> architectures.
>>>
>>> Important feature... Maybe it should be added to the TODO list in
>>> the
>>> docs?
>>>
>>>> This
>>>> is the feature we need, and it has nothing to do with the kernel
>>>> packages per se.
>>>
>>> The way that I assume debian will do building the kernel is that
>>> they
>>> use a generic debian defconfig for each architecture.
>>> E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
>>> 5.10/config.arm64_none_arm64.xz
>>>
>>> ISAR builds are often using specific defconfigs honored by ISAR's
>>> linux-custom for the target build.
>>>
>>> Debian specific architecture builds (e. g. host and target, maybe
>>> with
>>> sbuild) for one target device, using target specific stripped-down
>>> defconfigs, may not working properly. The kernel build for a
>>> foreign
>>> architecture (e. g. host) may fail because of an incompatible
>>> defconfig.
>>>
>>> Debian provide a generic very similar defconfig for different
>>> architectures, and they build the linux-image, linux-headers and
>>> linux-
>>> kbuild for each architecture:
>>> - So I assume the linux-kbuild:amd64 was build with linux-config-
>>> 5.10/config.amd64_none_amd64.xz
>>> - So I assume the linux-kbuild:arm64 was build with linux-config-
>>> 5.10/config.arm64_none_arm64.xz
>>>
>>> The debian generic defconfig may differ in some arch specific
>>> points
>>> but you can use the linux-kbuild:amd64 for a cross-build of linux-
>>> image:arm64 because the resulting scripts and tools are from the
>>> identical set. So mixing linux-headers with different archictecture
>>> specific builds from linux-kbuild should work since the defconfigs
>>> have
>>> a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
>>> needed for recordmcount utility as example).
>>
>> Sorry, I don't understand what role the config plays here. Isar is
>> used
>> to compile (usually) one specific kernel, Debian compiles a few
>> generic
>> ones. But the fact that Debian compile everything natively (for
>> production) and we do cross-compilation is not related to different
>> configs.
>>
> 
> I just wanted to point out for the future case when ISAR would support
> a generic way to build packages for multiple architectures: This might
> not be applicable to all kernel builds. E. g. because of the fact that
> the used defconfig for a "specific kernel" is not compatible for a
> multi architecture build, only for the architecture that the defconfig
> was made for.
> 
> Yes, it may work if you could specify different defconfigs for each
> architecture. One goal of this patch was to provide the kbuild scripts
> and tools just for host and target in a transparent way - without any
> changes to the build process of existing ISAR users.

Misunderstanding: I didn't meant to propose building the kernel-image
package for multiple arch, only the then factored-out kbuild packages
with the host tool (once for the build env, once for the target).

> 
>> The answer to the problem "I need kernel headers on the target" right
>> now is in Isar to build natively the kernel and, thus, the header
>> package. Not optimal, but we still need this case too rarely to have
>> cared about it more so far.
>>
>>>
>>>> Jan
>>>>
>>>>>
>>>>> 2. Honor recommendations for future from doc/custom_kernel.inc
>>>>> - Generate kernel headers packages for both host and target
>>>>>   when using cross build
>>>>>
>>>>> 3. Add extensions known from debian kernel packages structure
>>>>> - Generate a kernel headers package without binaries
>>>>> - Create specific kernel kbuild packages that
>>>>>   will ship the "scripts" and "tools" binaries
>>>>> - Use symlinks to point to the "scripts" and "tools" binaries
>>>>
>>>> Keep in mind that we intend to remain compatible with the layout
>>>> of
>>>> Debian. Self-built kernels should remain drop-in replacements of
>>>> the
>>>> Debian kernel packages. And that is true for development packages
>>>> as
>>>> well.
>>>>
>>>> So, are your changes working against that, or are they improving
>>>> existing incompatibility with the debian packages?
>>>
>>> The approach from this patchset does build the kernel in the known
>>> ISAR
>>> specific way. Debian itself patches the kernel Makefiles to
>>> modularize
>>> the kernel build what ISAR doesn't do. With this patch the packages
>>> linux-kbuild and linux-kbuild-cross are the only new packages that
>>> the
>>> ISAR build will provide. That means ISAR will provide then linux-
>>> headers, linux-image and linux-kbuild. They match on the target
>>> with
>>> the package name of the respective debian packages. So the existing
>>> drop-in should not be affected.
>>>
>>> In short this patch brings the ISAR linux-custom mechanism a bit
>>> nearer
>>> to the debian package structure. Since linux-headers behaves then
>>> more
>>> like the original debian linux-heades to ship the binaries within a
>>> separate package -> linux-kbuild.
>>>
>>> The trick is to build only the scripts and tools with schroot-
>>> target
>>> (QEMU) in a non-cross way. Since the kernel does not support
>>> building
>>> the scripts and tools for target architecture when do a cross build
>>> without patching the kernel Makefiles. I think that patch was never
>>> merged into the kernel:
>>>  
>>> https://patchwork.kernel.org/project/linux-kbuild/patch/1376046432-12588-1-git-send-email-broonie@kernel.org/
>>>
>>
>> The problem is that patch might have to not provide that use case I
>> wrote about above. It is a rare use case, a non-distro use case, but
>> I
>> wouldn't call it an invalid one. If we can address it by splitting up
>> the Isar kernel build into different packages and then only compile
>> the
>> binary bits once again, now for the target architecture, that would
>> be a
>> nice solution.
> 
> That's what the patch do:
> - Move the binary scripts and tools from linux-headers to linux-kbuild
> as debian does it for their kernels
> - Compile only the scripts and tools for the target using schroot-
> target (QEMU) and package it as linux-kbuild
> - Package the binary scripts and tools from the full cross kernel build
> to linux-kbuild-cross using schroot-host

It's moving in a good direction - but why linux-kbuild-cross? Nothing
depends on it in your patches, thus module builds will be broken. I
suspect we rather need

KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}:<native>

and therefore (on amd64) linux-kbuild-${KERNEL_NAME_PROVIDED}:amd64 to
be built and then installed for module builds.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux