Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages

[Jan Kiszka <jan.kiszka@siemens.com>]

On 10.11.22 19:33, Jan Kiszka wrote:
> On 10.11.22 18:49, Koch, Stefan (DI PA DCP R&D 3) wrote:
>> On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
>>> On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>>> On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
>>>>> On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>>>>> Hi
>>>>>>
>>>>>> This patchset is a set of three patches:
>>>>>> - Support overwriting configured schroot dir
>>>>>> - Split up binaries from kernel headers to kbuild packages
>>>>>> - Update custom_kernel docs for split up of kernel scripts and
>>>>>> tools
>>>>>>
>>>>>> The main use-case was to swap out the binaries
>>>>>> from the kernel headers package.
>>>>>> They will be swapped out into host and target
>>>>>> specific kernel kbuild packages.
>>>>>>
>>>>>> The main development goals were these:
>>>>>>
>>>>>> 1. Solve already known isar custom kernel
>>>>>> limitations from doc/custom_kernel.inc
>>>>>> - kernel headers package does not support both native
>>>>>>   and cross compilation of kernel modules when cross built
>>>>>
>>>>> Right, that requires building packages for multiple
>>>>> architectures.
>>>>
>>>> Important feature... Maybe it should be added to the TODO list in
>>>> the
>>>> docs?
>>>>
>>>>> This
>>>>> is the feature we need, and it has nothing to do with the kernel
>>>>> packages per se.
>>>>
>>>> The way that I assume debian will do building the kernel is that
>>>> they
>>>> use a generic debian defconfig for each architecture.
>>>> E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
>>>> 5.10/config.arm64_none_arm64.xz
>>>>
>>>> ISAR builds are often using specific defconfigs honored by ISAR's
>>>> linux-custom for the target build.
>>>>
>>>> Debian specific architecture builds (e. g. host and target, maybe
>>>> with
>>>> sbuild) for one target device, using target specific stripped-down
>>>> defconfigs, may not working properly. The kernel build for a
>>>> foreign
>>>> architecture (e. g. host) may fail because of an incompatible
>>>> defconfig.
>>>>
>>>> Debian provide a generic very similar defconfig for different
>>>> architectures, and they build the linux-image, linux-headers and
>>>> linux-
>>>> kbuild for each architecture:
>>>> - So I assume the linux-kbuild:amd64 was build with linux-config-
>>>> 5.10/config.amd64_none_amd64.xz
>>>> - So I assume the linux-kbuild:arm64 was build with linux-config-
>>>> 5.10/config.arm64_none_arm64.xz
>>>>
>>>> The debian generic defconfig may differ in some arch specific
>>>> points
>>>> but you can use the linux-kbuild:amd64 for a cross-build of linux-
>>>> image:arm64 because the resulting scripts and tools are from the
>>>> identical set. So mixing linux-headers with different archictecture
>>>> specific builds from linux-kbuild should work since the defconfigs
>>>> have
>>>> a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
>>>> needed for recordmcount utility as example).
>>>
>>> Sorry, I don't understand what role the config plays here. Isar is
>>> used
>>> to compile (usually) one specific kernel, Debian compiles a few
>>> generic
>>> ones. But the fact that Debian compile everything natively (for
>>> production) and we do cross-compilation is not related to different
>>> configs.
>>>
>>
>> I just wanted to point out for the future case when ISAR would support
>> a generic way to build packages for multiple architectures: This might
>> not be applicable to all kernel builds. E. g. because of the fact that
>> the used defconfig for a "specific kernel" is not compatible for a
>> multi architecture build, only for the architecture that the defconfig
>> was made for.
>>
>> Yes, it may work if you could specify different defconfigs for each
>> architecture. One goal of this patch was to provide the kbuild scripts
>> and tools just for host and target in a transparent way - without any
>> changes to the build process of existing ISAR users.
> 
> Misunderstanding: I didn't meant to propose building the kernel-image
> package for multiple arch, only the then factored-out kbuild packages
> with the host tool (once for the build env, once for the target).
> 
>>
>>> The answer to the problem "I need kernel headers on the target" right
>>> now is in Isar to build natively the kernel and, thus, the header
>>> package. Not optimal, but we still need this case too rarely to have
>>> cared about it more so far.
>>>
>>>>
>>>>> Jan
>>>>>
>>>>>>
>>>>>> 2. Honor recommendations for future from doc/custom_kernel.inc
>>>>>> - Generate kernel headers packages for both host and target
>>>>>>   when using cross build
>>>>>>
>>>>>> 3. Add extensions known from debian kernel packages structure
>>>>>> - Generate a kernel headers package without binaries
>>>>>> - Create specific kernel kbuild packages that
>>>>>>   will ship the "scripts" and "tools" binaries
>>>>>> - Use symlinks to point to the "scripts" and "tools" binaries
>>>>>
>>>>> Keep in mind that we intend to remain compatible with the layout
>>>>> of
>>>>> Debian. Self-built kernels should remain drop-in replacements of
>>>>> the
>>>>> Debian kernel packages. And that is true for development packages
>>>>> as
>>>>> well.
>>>>>
>>>>> So, are your changes working against that, or are they improving
>>>>> existing incompatibility with the debian packages?
>>>>
>>>> The approach from this patchset does build the kernel in the known
>>>> ISAR
>>>> specific way. Debian itself patches the kernel Makefiles to
>>>> modularize
>>>> the kernel build what ISAR doesn't do. With this patch the packages
>>>> linux-kbuild and linux-kbuild-cross are the only new packages that
>>>> the
>>>> ISAR build will provide. That means ISAR will provide then linux-
>>>> headers, linux-image and linux-kbuild. They match on the target
>>>> with
>>>> the package name of the respective debian packages. So the existing
>>>> drop-in should not be affected.
>>>>
>>>> In short this patch brings the ISAR linux-custom mechanism a bit
>>>> nearer
>>>> to the debian package structure. Since linux-headers behaves then
>>>> more
>>>> like the original debian linux-heades to ship the binaries within a
>>>> separate package -> linux-kbuild.
>>>>
>>>> The trick is to build only the scripts and tools with schroot-
>>>> target
>>>> (QEMU) in a non-cross way. Since the kernel does not support
>>>> building
>>>> the scripts and tools for target architecture when do a cross build
>>>> without patching the kernel Makefiles. I think that patch was never
>>>> merged into the kernel:
>>>>  
>>>> https://patchwork.kernel.org/project/linux-kbuild/patch/1376046432-12588-1-git-send-email-broonie@kernel.org/
>>>>
>>>
>>> The problem is that patch might have to not provide that use case I
>>> wrote about above. It is a rare use case, a non-distro use case, but
>>> I
>>> wouldn't call it an invalid one. If we can address it by splitting up
>>> the Isar kernel build into different packages and then only compile
>>> the
>>> binary bits once again, now for the target architecture, that would
>>> be a
>>> nice solution.
>>
>> That's what the patch do:
>> - Move the binary scripts and tools from linux-headers to linux-kbuild
>> as debian does it for their kernels
>> - Compile only the scripts and tools for the target using schroot-
>> target (QEMU) and package it as linux-kbuild
>> - Package the binary scripts and tools from the full cross kernel build
>> to linux-kbuild-cross using schroot-host
> 
> It's moving in a good direction - but why linux-kbuild-cross? Nothing
> depends on it in your patches, thus module builds will be broken. I
> suspect we rather need
> 
> KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}:<native>
> 
> and therefore (on amd64) linux-kbuild-${KERNEL_NAME_PROVIDED}:amd64 to
> be built and then installed for module builds.
> 
> Jan
> 

And regarding structure of the series: please split up the creation of
kbuild package from also building it for the target arch - if possible.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux