From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6755821036247187456 X-Received: by 2002:a19:5505:: with SMTP id n5mr1214562lfe.141.1573201703155; Fri, 08 Nov 2019 00:28:23 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:8806:: with SMTP id x6ls2167630ljh.11.gmail; Fri, 08 Nov 2019 00:28:22 -0800 (PST) X-Google-Smtp-Source: APXvYqzwyi3WX3wBqBnMhOxsgMqyktuDzaRDrrtoHmEVGn6IZbHuWrfhOwpNtGn0rGgaFP+togti X-Received: by 2002:a2e:300d:: with SMTP id w13mr4586455ljw.117.1573201702608; Fri, 08 Nov 2019 00:28:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1573201702; cv=none; d=google.com; s=arc-20160816; b=ocFUIzpnPxQ98GlK4voSdUYoKHhgJysw6ix1BzygfGe++HS/ZF5OBodh6Rvx5NQtqw xs97Zply9qCvH7ZnKZXM5jXvZM5Iid6hANJZ1DCJCzOu4c3MmFZlhC1dmA3X4+Zo4inG tw5y3eKQ7f4hWbim6BWwFbl3kyJ+7+ITOPWFAPb+D2phka+PSYSLytTCpN4An41Y0l8n K6JGPVdYRNteN/Lpgj9JvTQHXNpc5PiHOZJhvf9Z4YXV7H08VdmAOrKrPdf9GpAHgmrb 7Gw0FOU13JNbp1WveMYTw1AbpiZsj5TBZLYKFcIAyukmBKePsE9U8WPdDmUKK0v/RQcb uYaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=EHQqms/D7ulmbRrj9I/AsZa8s39H3lorrVl85PJ0Mlw=; b=mr8x5HuNWAjU4tqVZIw7WR3Q7KkFqtKepoRdbnKANqlyJ7WwkOuiFES7jMQqo/QIvz Pn5GFdwVSKyBH+M02eu/H2p7tJ0RyzBMIdfBJSh9DQWt4e7r1DXShWCv7X9pi9eTKP4t ZuOfoBneuapiRLak1javAqFb9F1+ZPyGEtkHquTkY4/0d65kDe9DA3L85SjG8NTYBIJN dUfG3O2CDOQdy8yBMM4oiFys3G92T1qnksbOtohISY5l2yPDgpjCAUUfrTbxTegNrb4K FsB39sHFKsr8hEl9WwgL1h3gibDyFPxTZeUOxr45+hfwWAusm6I7md8OAhdXIFeKygkK 3jVg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id c25si296901lji.2.2019.11.08.00.28.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Nov 2019 00:28:22 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id xA88SLWd032650 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 8 Nov 2019 09:28:21 +0100 Received: from [167.87.14.235] ([167.87.14.235]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id xA88SKuX016571; Fri, 8 Nov 2019 09:28:21 +0100 Subject: Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles To: Cedric Hombourger , isar-users@googlegroups.com References: <1573118604-909-1-git-send-email-Cedric_Hombourger@mentor.com> <1573196839-1143-1-git-send-email-Cedric_Hombourger@mentor.com> <1573196839-1143-7-git-send-email-Cedric_Hombourger@mentor.com> <28a6ef09-18f0-8e82-c464-2d2d530718b2@siemens.com> <84c9fbd7-aef5-2e06-ee38-5e9d44dc609a@mentor.com> <869f9760-f4ed-879f-655f-c03abd34c999@siemens.com> From: Jan Kiszka Message-ID: Date: Fri, 8 Nov 2019 09:28:20 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: LoGKLDrgMmIm On 08.11.19 08:59, Cedric Hombourger wrote: > > On 11/8/19 8:35 AM, Jan Kiszka wrote: >> On 08.11.19 08:22, Cedric Hombourger wrote: >>> >>> On 11/8/19 8:14 AM, Jan Kiszka wrote: >>>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>>> The Linux kernel has great support for cross-compiling the kernel >>>>> image >>>>> and modules. There is however no support/mechanism for >>>>> cross-compiling the >>>>> build tools from its "scripts" directory. While HOSTCC may be set >>>>> to use >>>>> our cross-compiler, the kernel build infrasture would then try to run >>>>> foreign-arch binaries such as fixdep. The easiest and least >>>>> intrusive way >>>>> to support this is to enable execution of such binaries via >>>>> binfmt/qemu >>>>> like we do in the buildchroot-target environment. >>>>> >>>>> Signed-off-by: Cedric Hombourger >>>>> --- >>>>>   meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >>>>>   1 file changed, 6 insertions(+) >>>>> >>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> index 408ad39..2e76acb 100644 >>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >>>>>       ${BUILDCHROOT_PREINSTALL_COMMON} \ >>>>>       libc6:${DISTRO_ARCH} \ >>>>>       crossbuild-essential-${DISTRO_ARCH}" >>>>> + >>>>> +buildchroot_install_files_append() { >>>>> +    if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >>>>> +       sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >>>>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >>>>> +    fi >>>>> +} >>>>> >>>> >>>> You didn't address my comment on this one. I doubt it's the right >>>> approach. >>> >>> More specifically? >>> >>> I do have something in the works but we *still* need to build a >>> linux-kernel-headers package for the target. there are two approaches: >> >> Currently, if you need the package for the target, you need to build >> it natively. The cases of requiring such packages is so far rare >> enough (none in all the many cases I saw so far) to deal with the >> slowdown. >> >>> (1) use the Makefiles from the kernel and "simply" override HOSTCC >>> but you then need qemu-static since the kernel will need to run some >>> of its tools from "scripts/" (fixdep in particular) when it builds >>> the other tools (e.g modpost) or (2) do like upstream manually >>> cross-compile the tools. >>> >>> The downside of (2) is that you need to know what to compile (list of >>> .c files) and how to compile them. That "knowledge" may vary from one >>> kernel to another. >>> >>> I have local patches to produce a linux-kernel-headers-cross-amd64 >>> package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. >>> next to libc headers) as well as kernel build tools for the build >>> machine (typically amd64). This will allow us to cross-compile >>> out-of-tree kernel modules without running any tools with a >>> foreign-arch (for the avoidance of doubt, we still need >>> qemu-user-static in buildchroot-host while building the kernel) >>> >>> So with all that said, you are more than welcome to suggest a better >>> approach.... >> >> Step 1 is to tag the currently generated header package in a way that >> it won't be accidentally selected for target installation when it was >> cross-built for a different host architecture. Step 2 could be >> factoring out the linux-headers package build so that it can run with >> ISAR_CROSS_COMPILE = "0" for the target architecture in the target >> buildchroot - when that special case is actually needed. > > If in step1, you are suggesting that our recipe generates packages for > the following "Architecture": ${HOST_ARCH}, all, ${DISTRO_ARCH} then to > the best of my knowledge that isn't possible! I think you can only > generate packages for all, ${DISTRO_ARCH} from a dpkg-buildpackage run > but again, let me know if you think otherwise (could not find anything > in the Debian Standards document suggesting so but I may have overlooked) Hmm, the problem is that we only build our packages with -a$target_arch. We would actually have to build them twice when doing cross build, once for the target and once for the host. Then the right architecture would be chosen when installing a self-built build dependency into the buildchroot of the next package, I would assume. > > If we agree that using debhelper and therefore our generic dpkg class is > a good improvement (compared to kernel's builddeb script crafting > packages on its own and having to specify dependencies manually), I > would suggest we stick with the qemu-user-static approach for now (it is > only needed for a fraction of seconds during the build so to me that is > a non-issue). I'm concerned that adding qemu-user-static generally will paper over other issues. The buildchroot-host is not supposed to handle target binaries. We would be off now for this single package. > > As hinted, I am working on a linux-kernel-headers-cross package > generation. I am currently considering running build.sh a second time > but this time targeting HOST_ARCH and with a control file only carrying > that -cross package (it would use the same kernel sources/tree as input) > but I'd like to make this a second step since it needs more work and I a > genuinely believe that the current solution is something we can live > with for now As I said, we have no urgent need for the current solution because you can generate a target-compatible header package by not using cross-build (AKA using production build). You could support that path by introducing a mode where linux-custom only builds the linux-headers package. So I would rather use the time and teach Isar in general about building correct host-vs-target devel packages, including a second build run when cross-building is active. We had similar issues earlier, e.g. in the openssl area. So better fix them conceptually, rather than patching on thing only. > >> Alternatively, though more for the long run, the tools cross-build of >> upstream could be improved. > yes that would be good (if you are referring to the ability to do > something like make scripts O=build-host-tools > HOST_CROSS_COMPILE=${DEB_HOST_GNU_TYPE}- in the kernel) Yes, something in this direction. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux