Re: building custom kernel fails

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-04-19 16:40, Jon Nettleton wrote:
> 
> On Thu, Apr 19, 2018 at 4:35 PM Alexander Smirnov <asmirnov@ilbers.de
> <mailto:asmirnov@ilbers.de>> wrote:
> 
>     On 04/19/2018 05:26 PM, Jon Nettleton wrote:
>     >
>     > On Thu, Apr 19, 2018 at 3:47 PM Henning Schild
>     > <henning.schild@siemens.com <mailto:henning.schild@siemens.com>
>     <mailto:henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>>> wrote:
>     >
>     >     Am Thu, 19 Apr 2018 13:27:36 +0000
>     >     schrieb Jon Nettleton <jon@solid-run.com
>     <mailto:jon@solid-run.com> <mailto:jon@solid-run.com
>     <mailto:jon@solid-run.com>>>:
>     >
>     >      > On Thu, Apr 19, 2018 at 12:38 PM Henning Schild
>     >      > <henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>
>     <mailto:henning.schild@siemens.com <mailto:henning.schild@siemens.com>>>
>     >     wrote:
>     >      >
>     >      > > Am Thu, 19 Apr 2018 09:33:28 +0000
>     >      > > schrieb Jon Nettleton <jon@solid-run.com
>     <mailto:jon@solid-run.com>
>     >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>>:
>     >      > >
>     >      > > > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
>     >      > > > <henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>
>     >     <mailto:henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>>> wrote:
>     >      > > >
>     >      > > > > Am Thu, 19 Apr 2018 00:32:21 -0700
>     >      > > > > schrieb <jon@solid-run.com <mailto:jon@solid-run.com>
>     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>>:
>     >      > > > >
>     >      > > > > > I am now building a custom debian package from our
>     kernel
>     >      > > > > > sources, the kernel compile goes fine.  The deb-pkg is
>     >      > > > > > failing because the number of files passed to xarg
>     is too
>     >      > > > > > long.
>     >      > > > > >
>     >      > > > > > Error message is.
>     >      > > > > >
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | Using default
>     distribution
>     >      > > > > > of 'unstable' in the changelog
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | Install
>     lsb-release or set
>     >      > > > > > $KDEB_CHANGELOG_DIST explicitly
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs:
>     md5sum:
>     >      > > > > > Argument list too long
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - |
>     >      > > > > > scripts/package/Makefile:90: recipe for target
>     'deb-pkg'
>     >      > > > > > failed 2018-04-19 07:11:17 - INFO     - | make[1]: ***
>     >      > > > > > [deb-pkg] Error 126 2018-04-19 07:11:17 - INFO     - |
>     >      > > > > > Makefile:1347: recipe for target 'deb-pkg' failed
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg]
>     >     Error 2
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2
>     >     from a
>     >      > > > > > shell command.
>     >      > > > > >
>     >      > > > > > looks like on large kernel builds the file lists
>     will need to
>     >      > > > > > be broken up.
>     >      > > > > >
>     >      > > > >
>     >      > > > > I have seen that before. It happens in buildchroot
>     when using
>     >      > > > > qemu-arm-static binfmt magic. Something in that chain
>     seems to
>     >      > > > > cause problems for very long argument lists.
>     >      > > > > In my case it was a "make clean" and i removed the
>     files from
>     >      > > > > outside the buildchroot. But since it now happened
>     again it is
>     >      > > > > probably something worth looking into.
>     >      > > > >
>     >      > > > > Henning
>     >      > > > >
>     >      > > >
>     >      > > > Oh interesting.  I wonder if this has something to do
>     with the
>     >      > > > binfmt support in CentOS vs Debian.  I will look into
>     it and
>     >      > > > report back.
>     >      > >
>     >      > > I saw it on Gentoo building in a debian docker container.
>     So i
>     >     guess
>     >      > > the problem might be related to the binfmt wrapping in
>     general, not
>     >      > > distro specific.
>     >      > >
>     >      > > Henning
>     >      > >
>     >      > >
>     >      > It looks like that was the problem.  I just went through
>     the pain of
>     >      > building qemu 2.9.1 for Centos7...and all the dependencies
>     that were
>     >      > needed by it.   That has fixed the custom kernel build issues.
>     >      > Thanks for all the pointers.
>     >      >
>     >      > I would say for now if anyone asks you can consider Centos
>     7 a no
>     >      > go.  If there is interest I may add a build to Copr for
>     other users.
>     >      > Fedora 26 and newer should be fine.
>     >
>     >     Isar calls "random" sudos all over the place. Someone actually
>     allowing
>     >     this on a productive system surprises me ... to say the least.
>     >
>     >     My advise it to use a VM or container to run Isar in, and in
>     that case
>     >     going for Debian9 is easy. One example, used by many of us, is
>     >     https://hub.docker.com/r/kasproject/kas-isar/
>     >
>     >     Henning
>     >
>     >
>     > Yes that is what I am  using to build with.  I can only imagine
>     that the
>     > newer build of qemu brings in a configuration to the host os that is
>     > required for things to build properly.
>     >
> 
>     My host OS is Fedora for many years, so I use chroot to work with
>     Debian
>     environment (for Isar as well):
> 
>     $ sudo debootstrap jessie debian8-isar/ http://ftp.debian.org/debian
>     $ sudo chroot debian8-isar
> 
>     This also could be an option.
> 
>     Alex
> 
> 
> yes very true, thanks.  I am specifically interested in using docker
> containers to build images so I can bring up a build farm for customers
> that want to build production images.  My main builder is x86_64 however
> I am looking forward to comparing the performance of that machine vs our
> "new" ARM64 server boards.

We are looking into very similar questions. We have a build
infrastructure ready based on gitlab-ci (you can find the yml file in
Isar, in fact), and we would also be happy to recommend public CI like
Travis or Shippable for Isar builds.

As pointed out in the other branch of this thread, there are some
challenges preventing unprivileged builds right now. That is at least
the need to mount images or paths inside the container and the need to
configure binfmt according to Debian expectations.

If you have a CI system where you can spawn a VM on each build and that
VM can run a (privileged) Docker container, you could move forward
already despite of this. Would be our plan B here, but it's not set up
with three clicks either.

>  Of course I can run run debootstrap within
> the container first, but since CIP is using Kas I also want to use that
> infrastructure.

Using kas quickly pays off when it come to managing sources and versions
of your dependencies, or providing your users a way to put a layer on
top of your released configuration.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux