Cross-building and debugging

Cross-building and debugging

[andrea.marson.dave]

[-- Attachment #1.1: Type: text/plain, Size: 346 bytes --]

Hi everybody,

IIUC, when using Isar it is implicitly assumed that the user applications 
are natively built on the target.
If, for any reason, this is not possibile, one has to cross-build the 
applications.
Which approach do you recommend to set up the cross-build environment, in 
order to make it work smoothly with Isar?

Thank you,
Andrea


[-- Attachment #1.2: Type: text/html, Size: 391 bytes --]

Re: Cross-building and debugging

[Andreas Reichel]

On Mon, Oct 30, 2017 at 01:34:19AM -0700, andrea.marson.dave@gmail.com wrote:
> Hi everybody,
> 
> IIUC, when using Isar it is implicitly assumed that the user applications 
> are natively built on the target.

Hi,

since people are mostly on vacation today, I offer an answer according
to my latest information:

Applications are built natively within the build-chroot. But not "on"
the target, but for the target, using the target arch.
> If, for any reason, this is not possibile, one has to cross-build the 
> applications.

This is a wrong assumption. One can enter a cross-chroot and inside this
cross-chroot, using binfmt support, target-arch tools can be executed
"quasi natively" since the kernel passes execution to a qemu emulator.
> Which approach do you recommend to set up the cross-build environment, in 
> order to make it work smoothly with Isar?
>
I am not an active Isar user and lack details, but I thougt this might
help in meantime.

Kind regards,
Andreas
 
> Thank you,
> Andrea
> 
> -- 
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To post to this group, send email to isar-users@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/0b55db7d-e602-4f4b-b6d5-bc1c0a0e18e8%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

Re: Cross-building and debugging

[Henning Schild]

Am Mon, 30 Oct 2017 01:34:19 -0700
schrieb <andrea.marson.dave@gmail.com>:

> Hi everybody,
> 
> IIUC, when using Isar it is implicitly assumed that the user
> applications are natively built on the target.
> If, for any reason, this is not possibile, one has to cross-build the 
> applications.

Why would that not be possible? I guess you just do not have the
patience to wait for the slow qemu-wrapped compiler.

If you ask me now, cross compile support should not be added to Isar,
because it is against what Debian is doing. But maybe someone will
convince me otherwise some day.

> Which approach do you recommend to set up the cross-build
> environment, in order to make it work smoothly with Isar?

There are a few patches out there that enable cross builds, but none of
that stuff was proposed for upstream yet.

https://github.com/ilbers/isar/commits/lenormf/develop-l20170602-dpkg-cross

https://groups.google.com/d/msgid/isar-users/8fe13268-9bfa-4b24-897a-133c9530c188%40googlegroups.com

If the only upside of such complexity is gain in speed i would probably
reject patches that try to introduce something like that. With the
recently merged cache features one could just keep the .debs and the
compile-time would only be a problem once and maybe for CI.

Today you could use dpkg-raw and just package your binaries, that maybe
came from another build-system. Or you could write a recipe that adds
cross-compile tasks to dpkg-raw.

>> inherit dpkg-raw
>>
>> do_compile() {
>>	make CROSS_COMPILE=/foobar/
>>	...
>> }
>>
>> addtask compile after do_unpack before do_install

Henning

> Thank you,
> Andrea
> 

Re: Cross-building and debugging

[Baurzhan Ismagulov]

Hello Andrea,

On Mon, Oct 30, 2017 at 01:34:19AM -0700, andrea.marson.dave@gmail.com wrote:
> IIUC, when using Isar it is implicitly assumed that the user applications 
> are natively built on the target.

The current implementation builds user applications natively in buildchroot
using qemu on the host. This was a customer requirement to avoid having to
tweak Debian packages which one may happen to want to rebuild and which don't
support cross-building.

That said, Isar's predecessor is based on cross-building, which is state of the
art in Emdebian and has its advantages in certain use cases.


> If, for any reason, this is not possibile, one has to cross-build the 
> applications.
> Which approach do you recommend to set up the cross-build environment, in 
> order to make it work smoothly with Isar?

Basically, it worked like this:

1. Install cross-toolchain (e.g., gcc-arm-linux-gnueabihf) into buildchroot.

2. Modify the package's debian/rules for cross-building (e.g., pass the right
   --host=arm-linux-gnueabihf to ./configure).

3. Cross-build (e.g., call dpkg-buildpackage with --arch=arm or similar).

I don't remember the details, so the quotations above are not literal.

There is a branch
https://github.com/ilbers/isar/tree/lenormf/develop-l20170602-dpkg-cross which
should support cross-building. We plan to refine it and propose for
integration into master.

Till then, you might use the current qemu approach or look at the branch above
(will probably require some hacking). If you can share your use case, maybe we
could suggest something. E.g., GLIB doesn't cross-compile out of the box due to
configure failing to run test programs for the target; supplying the test
results to configure pre-canned in a file makes it happy.


With kind regards,
Baurzhan.

Re: Cross-building and debugging

[Baurzhan Ismagulov]

On Mon, Oct 30, 2017 at 02:43:05PM +0100, Henning Schild wrote:
> If you ask me now, cross compile support should not be added to Isar,
> because it is against what Debian is doing.

Debian happens not to cross-build due to various reasons. That said,
cross-building isn't officially prohibited. In fact, Emdebian, debian-cross,
and debian-bootstrap are using it. There were mass bug filings that resulted
IIRC in a couple of hundred packages accepting cross-building patches. I'm also
very curious what debian-mobile comes up with.

You are right that without testing, those changes would bitrot. The efforts
above are not a part of the official Debian infrastructure yet, but there are
discussions about that.


> If the only upside of such complexity is gain in speed i would probably
> reject patches that try to introduce something like that.

The efforts of e.g. debian-bootstrap is to be able to reproducibly bootstrap
any arch from zero. This is an important ability for any distro, desktop or
embedded.

The implementation involves installing the compiler and passing the right
options to configure and dpkg-buildpackage, which we do already for native
building. So, additional complexity is marginal. Debugging may be one level
more complex, but cross-building is officially supported in Debian tools. Isar
itself doesn't impose one or another way of building, so the user would choose
whichever one is suitable for him -- individually for debug or release builds
of the same working copy.

Build times are critical in times where developers are scarce and their time is
expensive. The slower the development loop, the more is the developer blocked.
And what is not tested doesn't work. This is one of the reasons Isar explicitly
cares for performance -- as the last but not least aspect.


> With the
> recently merged cache features one could just keep the .debs and the
> compile-time would only be a problem once and maybe for CI.

The initial "cache" name turned out to be unfortunate and has caused much
confusion. The idea behind the feature is to provide mechanism for deploying to
apt repos. The policy -- i.e., how the users are going to use it -- is up to
them. The intended use cases are:

1. Hardware vendors providing layers for Isar with their source and binary
   packages (kernel modules, libraries, tools, etc.).

2. Product builders managing binary artifacts as a part of their CM, especially
   safety-oriented processes.

And sharing the binary artifacts between the developers to speed up build times
is a nice add-on.


> Today you could use dpkg-raw and just package your binaries, that maybe
> came from another build-system.

In the long run, I'd like to see that extended to a more Debian way, Isar
generating debian/rules on the fly.


With kind regards,
Baurzhan.