Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles

[Cedric Hombourger <Cedric_Hombourger@mentor.com>]

On 11/8/19 8:35 AM, Jan Kiszka wrote:
> On 08.11.19 08:22, Cedric Hombourger wrote:
>>
>> On 11/8/19 8:14 AM, Jan Kiszka wrote:
>>> On 08.11.19 08:07, Cedric Hombourger wrote:
>>>> The Linux kernel has great support for cross-compiling the kernel 
>>>> image
>>>> and modules. There is however no support/mechanism for 
>>>> cross-compiling the
>>>> build tools from its "scripts" directory. While HOSTCC may be set 
>>>> to use
>>>> our cross-compiler, the kernel build infrasture would then try to run
>>>> foreign-arch binaries such as fixdep. The easiest and least 
>>>> intrusive way
>>>> to support this is to enable execution of such binaries via 
>>>> binfmt/qemu
>>>> like we do in the buildchroot-target environment.
>>>>
>>>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>>>> ---
>>>>   meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++
>>>>   1 file changed, 6 insertions(+)
>>>>
>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb 
>>>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>>> index 408ad39..2e76acb 100644
>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \
>>>>       ${BUILDCHROOT_PREINSTALL_COMMON} \
>>>>       libc6:${DISTRO_ARCH} \
>>>>       crossbuild-essential-${DISTRO_ARCH}"
>>>> +
>>>> +buildchroot_install_files_append() {
>>>> +    if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then
>>>> +       sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' 
>>>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static'
>>>> +    fi
>>>> +}
>>>>
>>>
>>> You didn't address my comment on this one. I doubt it's the right 
>>> approach.
>>
>> More specifically?
>>
>> I do have something in the works but we *still* need to build a 
>> linux-kernel-headers package for the target. there are two approaches: 
>
> Currently, if you need the package for the target, you need to build 
> it natively. The cases of requiring such packages is so far rare 
> enough (none in all the many cases I saw so far) to deal with the 
> slowdown.
>
>> (1) use the Makefiles from the kernel and "simply" override HOSTCC 
>> but you then need qemu-static since the kernel will need to run some 
>> of its tools from "scripts/" (fixdep in particular) when it builds 
>> the other tools (e.g modpost) or (2) do like upstream manually 
>> cross-compile the tools.
>>
>> The downside of (2) is that you need to know what to compile (list of 
>> .c files) and how to compile them. That "knowledge" may vary from one 
>> kernel to another.
>>
>> I have local patches to produce a linux-kernel-headers-cross-amd64 
>> package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. 
>> next to libc headers) as well as kernel build tools for the build 
>> machine (typically amd64). This will allow us to cross-compile 
>> out-of-tree kernel modules without running any tools with a 
>> foreign-arch (for the avoidance of doubt, we still need 
>> qemu-user-static in buildchroot-host while building the kernel)
>>
>> So with all that said, you are more than welcome to suggest a better 
>> approach....
>
> Step 1 is to tag the currently generated header package in a way that 
> it won't be accidentally selected for target installation when it was 
> cross-built for a different host architecture. Step 2 could be 
> factoring out the linux-headers package build so that it can run with 
> ISAR_CROSS_COMPILE = "0" for the target architecture in the target 
> buildchroot - when that special case is actually needed.

If in step1, you are suggesting that our recipe generates packages for 
the following "Architecture": ${HOST_ARCH}, all, ${DISTRO_ARCH} then to 
the best of my knowledge that isn't possible! I think you can only 
generate packages for all, ${DISTRO_ARCH} from a dpkg-buildpackage run 
but again, let me know if you think otherwise (could not find anything 
in the Debian Standards document suggesting so but I may have overlooked)

If we agree that using debhelper and therefore our generic dpkg class is 
a good improvement (compared to kernel's builddeb script crafting 
packages on its own and having to specify dependencies manually), I 
would suggest we stick with the qemu-user-static approach for now (it is 
only needed for a fraction of seconds during the build so to me that is 
a non-issue).

As hinted, I am working on a linux-kernel-headers-cross package 
generation. I am currently considering running build.sh a second time 
but this time targeting HOST_ARCH and with a control file only carrying 
that -cross package (it would use the same kernel sources/tree as input) 
but I'd like to make this a second step since it needs more work and I a 
genuinely believe that the current solution is something we can live 
with for now

> Alternatively, though more for the long run, the tools cross-build of 
> upstream could be improved.
yes that would be good (if you are referring to the ability to do 
something like make scripts O=build-host-tools 
HOST_CROSS_COMPILE=${DEB_HOST_GNU_TYPE}- in the kernel)
>
> Jan
>