Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles

[Jan Kiszka <jan.kiszka@siemens.com>]

On 08.11.19 08:59, Cedric Hombourger wrote:
> 
> On 11/8/19 8:35 AM, Jan Kiszka wrote:
>> On 08.11.19 08:22, Cedric Hombourger wrote:
>>>
>>> On 11/8/19 8:14 AM, Jan Kiszka wrote:
>>>> On 08.11.19 08:07, Cedric Hombourger wrote:
>>>>> The Linux kernel has great support for cross-compiling the kernel 
>>>>> image
>>>>> and modules. There is however no support/mechanism for 
>>>>> cross-compiling the
>>>>> build tools from its "scripts" directory. While HOSTCC may be set 
>>>>> to use
>>>>> our cross-compiler, the kernel build infrasture would then try to run
>>>>> foreign-arch binaries such as fixdep. The easiest and least 
>>>>> intrusive way
>>>>> to support this is to enable execution of such binaries via 
>>>>> binfmt/qemu
>>>>> like we do in the buildchroot-target environment.
>>>>>
>>>>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>>>>> ---
>>>>>   meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++
>>>>>   1 file changed, 6 insertions(+)
>>>>>
>>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb 
>>>>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>>>> index 408ad39..2e76acb 100644
>>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>>>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \
>>>>>       ${BUILDCHROOT_PREINSTALL_COMMON} \
>>>>>       libc6:${DISTRO_ARCH} \
>>>>>       crossbuild-essential-${DISTRO_ARCH}"
>>>>> +
>>>>> +buildchroot_install_files_append() {
>>>>> +    if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then
>>>>> +       sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' 
>>>>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static'
>>>>> +    fi
>>>>> +}
>>>>>
>>>>
>>>> You didn't address my comment on this one. I doubt it's the right 
>>>> approach.
>>>
>>> More specifically?
>>>
>>> I do have something in the works but we *still* need to build a 
>>> linux-kernel-headers package for the target. there are two approaches: 
>>
>> Currently, if you need the package for the target, you need to build 
>> it natively. The cases of requiring such packages is so far rare 
>> enough (none in all the many cases I saw so far) to deal with the 
>> slowdown.
>>
>>> (1) use the Makefiles from the kernel and "simply" override HOSTCC 
>>> but you then need qemu-static since the kernel will need to run some 
>>> of its tools from "scripts/" (fixdep in particular) when it builds 
>>> the other tools (e.g modpost) or (2) do like upstream manually 
>>> cross-compile the tools.
>>>
>>> The downside of (2) is that you need to know what to compile (list of 
>>> .c files) and how to compile them. That "knowledge" may vary from one 
>>> kernel to another.
>>>
>>> I have local patches to produce a linux-kernel-headers-cross-amd64 
>>> package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. 
>>> next to libc headers) as well as kernel build tools for the build 
>>> machine (typically amd64). This will allow us to cross-compile 
>>> out-of-tree kernel modules without running any tools with a 
>>> foreign-arch (for the avoidance of doubt, we still need 
>>> qemu-user-static in buildchroot-host while building the kernel)
>>>
>>> So with all that said, you are more than welcome to suggest a better 
>>> approach....
>>
>> Step 1 is to tag the currently generated header package in a way that 
>> it won't be accidentally selected for target installation when it was 
>> cross-built for a different host architecture. Step 2 could be 
>> factoring out the linux-headers package build so that it can run with 
>> ISAR_CROSS_COMPILE = "0" for the target architecture in the target 
>> buildchroot - when that special case is actually needed.
> 
> If in step1, you are suggesting that our recipe generates packages for 
> the following "Architecture": ${HOST_ARCH}, all, ${DISTRO_ARCH} then to 
> the best of my knowledge that isn't possible! I think you can only 
> generate packages for all, ${DISTRO_ARCH} from a dpkg-buildpackage run 
> but again, let me know if you think otherwise (could not find anything 
> in the Debian Standards document suggesting so but I may have overlooked)

Hmm, the problem is that we only build our packages with -a$target_arch. 
We would actually have to build them twice when doing cross build, once 
for the target and once for the host. Then the right architecture would 
be chosen when installing a self-built build dependency into the 
buildchroot of the next package, I would assume.

> 
> If we agree that using debhelper and therefore our generic dpkg class is 
> a good improvement (compared to kernel's builddeb script crafting 
> packages on its own and having to specify dependencies manually), I 
> would suggest we stick with the qemu-user-static approach for now (it is 
> only needed for a fraction of seconds during the build so to me that is 
> a non-issue).

I'm concerned that adding qemu-user-static generally will paper over 
other issues. The buildchroot-host is not supposed to handle target 
binaries. We would be off now for this single package.

> 
> As hinted, I am working on a linux-kernel-headers-cross package 
> generation. I am currently considering running build.sh a second time 
> but this time targeting HOST_ARCH and with a control file only carrying 
> that -cross package (it would use the same kernel sources/tree as input) 
> but I'd like to make this a second step since it needs more work and I a 
> genuinely believe that the current solution is something we can live 
> with for now

As I said, we have no urgent need for the current solution because you 
can generate a target-compatible header package by not using cross-build 
(AKA using  production build). You could support that path by 
introducing a mode where linux-custom only builds the linux-headers package.

So I would rather use the time and teach Isar in general about building 
correct host-vs-target devel packages, including a second build run when 
cross-building is active. We had similar issues earlier, e.g. in the 
openssl area. So better fix them conceptually, rather than patching on 
thing only.

> 
>> Alternatively, though more for the long run, the tools cross-build of 
>> upstream could be improved.
> yes that would be good (if you are referring to the ability to do 
> something like make scripts O=build-host-tools 
> HOST_CROSS_COMPILE=${DEB_HOST_GNU_TYPE}- in the kernel)

Yes, something in this direction.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux