building custom kernel fails

building custom kernel fails

[jon]

[-- Attachment #1.1: Type: text/plain, Size: 944 bytes --]

I am now building a custom debian package from our kernel sources, the 
kernel compile goes fine.  The deb-pkg is failing because the number of 
files passed to xarg is too long.

Error message is.

2018-04-19 07:11:17 - INFO     - | Using default distribution of 'unstable' 
in the changelog
2018-04-19 07:11:17 - INFO     - | Install lsb-release or set 
$KDEB_CHANGELOG_DIST explicitly
2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum: Argument list 
too long
2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90: recipe for 
target 'deb-pkg' failed
2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target 
'deb-pkg' failed
2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell 
command.

looks like on large kernel builds the file lists will need to be broken up.

[-- Attachment #1.2: Type: text/html, Size: 1158 bytes --]

Re: building custom kernel fails

[Claudius Heine]

Hi Jon,

On 2018-04-19 09:32, jon@solid-run.com wrote:
> I am now building a custom debian package from our kernel sources, the
> kernel compile goes fine.  The deb-pkg is failing because the number of
> files passed to xarg is too long.
> 
> Error message is.
> 
> 2018-04-19 07:11:17 - INFO     - | Using default distribution of 'unstable'
> in the changelog
> 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> $KDEB_CHANGELOG_DIST explicitly
> 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum: Argument list
> too long
> 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90: recipe for
> target 'deb-pkg' failed
> 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
> 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target
> 'deb-pkg' failed
> 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell
> command.
> 
> looks like on large kernel builds the file lists will need to be broken up.
> 

That looks like its a kernel source issue. You might need to patch the 
Makefile to maybe use a loop.

Good luck!
Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: building custom kernel fails

[Jan Kiszka]

On 2018-04-19 09:32, jon@solid-run.com wrote:
> I am now building a custom debian package from our kernel sources, the
> kernel compile goes fine.  The deb-pkg is failing because the number of
> files passed to xarg is too long.
> 
> Error message is.
> 
> 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> 'unstable' in the changelog
> 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> $KDEB_CHANGELOG_DIST explicitly
> 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum: Argument list
> too long
> 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90: recipe
> for target 'deb-pkg' failed
> 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
> 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target
> 'deb-pkg' failed
> 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell
> command.
> 
> looks like on large kernel builds the file lists will need to be broken up.

Where exactly is this argument list passed? What exactly is your setup
(host, target distro, kernel version etc.)?

BTW, you can find a working custom kernel build here e.g.:
https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
There is an ARM64 version of that as well, just waiting for cross-build
support to be available.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 1928 bytes --]

On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:

> On 2018-04-19 09:32, jon@solid-run.com wrote:
> > I am now building a custom debian package from our kernel sources, the
> > kernel compile goes fine.  The deb-pkg is failing because the number of
> > files passed to xarg is too long.
> >
> > Error message is.
> >
> > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> > 'unstable' in the changelog
> > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> > $KDEB_CHANGELOG_DIST explicitly
> > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum: Argument list
> > too long
> > 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90: recipe
> > for target 'deb-pkg' failed
> > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
> > 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target
> > 'deb-pkg' failed
> > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell
> > command.
> >
> > looks like on large kernel builds the file lists will need to be broken
> up.
>
> Where exactly is this argument list passed? What exactly is your setup
> (host, target distro, kernel version etc.)?
>
> BTW, you can find a working custom kernel build here e.g.:
>
> https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
> There is an ARM64 version of that as well, just waiting for cross-build
> support to be available.
>
> Jan
>
>
>  Yes I have used your recipe as a reference.  It is a 4.9.y lts kernel
with the NXP BSP merged for our hardware. https://www.solid-run.com

I am attempting to use the default make deb-pkg in that LTS kernel.  I am
running this in a kas-isar container on a CentOS 7 host.  My guess is
ulimit my possibly be getting in the way.  I will do a minimal kernel
config build and then move up from there.

Jon

[-- Attachment #2: Type: text/html, Size: 2674 bytes --]

Re: building custom kernel fails

[Jan Kiszka]

On 2018-04-19 09:47, Jon Nettleton wrote:
> 
> On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka <jan.kiszka@siemens.com
> <mailto:jan.kiszka@siemens.com>> wrote:
> 
>     On 2018-04-19 09:32, jon@solid-run.com <mailto:jon@solid-run.com> wrote:
>     > I am now building a custom debian package from our kernel sources, the
>     > kernel compile goes fine.  The deb-pkg is failing because the
>     number of
>     > files passed to xarg is too long.
>     >
>     > Error message is.
>     >
>     > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
>     > 'unstable' in the changelog
>     > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
>     > $KDEB_CHANGELOG_DIST explicitly
>     > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
>     Argument list
>     > too long
>     > 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90: recipe
>     > for target 'deb-pkg' failed
>     > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
>     > 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target
>     > 'deb-pkg' failed
>     > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
>     > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell
>     > command.
>     >
>     > looks like on large kernel builds the file lists will need to be
>     broken up.
> 
>     Where exactly is this argument list passed? What exactly is your setup
>     (host, target distro, kernel version etc.)?
> 
>     BTW, you can find a working custom kernel build here e.g.:
>     https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
>     There is an ARM64 version of that as well, just waiting for cross-build
>     support to be available.
> 
>     Jan
> 
> 
>  Yes I have used your recipe as a reference.  It is a 4.9.y lts kernel
> with the NXP BSP merged for our hardware. https://www.solid-run.com
> 
> I am attempting to use the default make deb-pkg in that LTS kernel.  I
> am running this in a kas-isar container on a CentOS 7 host.  My guess is
> ulimit my possibly be getting in the way.  I will do a minimal kernel
> config build and then move up from there.

Are you using our recipes-kernel/linux/linux-custom.inc or your own rules?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 2461 bytes --]

On Thu, Apr 19, 2018 at 9:52 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:

> On 2018-04-19 09:47, Jon Nettleton wrote:
> >
> > On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka <jan.kiszka@siemens.com
> > <mailto:jan.kiszka@siemens.com>> wrote:
> >
> >     On 2018-04-19 09:32, jon@solid-run.com <mailto:jon@solid-run.com>
> wrote:
> >     > I am now building a custom debian package from our kernel sources,
> the
> >     > kernel compile goes fine.  The deb-pkg is failing because the
> >     number of
> >     > files passed to xarg is too long.
> >     >
> >     > Error message is.
> >     >
> >     > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> >     > 'unstable' in the changelog
> >     > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> >     > $KDEB_CHANGELOG_DIST explicitly
> >     > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> >     Argument list
> >     > too long
> >     > 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90:
> recipe
> >     > for target 'deb-pkg' failed
> >     > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
> >     > 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target
> >     > 'deb-pkg' failed
> >     > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> >     > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a
> shell
> >     > command.
> >     >
> >     > looks like on large kernel builds the file lists will need to be
> >     broken up.
> >
> >     Where exactly is this argument list passed? What exactly is your
> setup
> >     (host, target distro, kernel version etc.)?
> >
> >     BTW, you can find a working custom kernel build here e.g.:
> >
> https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
> >     There is an ARM64 version of that as well, just waiting for
> cross-build
> >     support to be available.
> >
> >     Jan
> >
> >
> >  Yes I have used your recipe as a reference.  It is a 4.9.y lts kernel
> > with the NXP BSP merged for our hardware. https://www.solid-run.com
> >
> > I am attempting to use the default make deb-pkg in that LTS kernel.  I
> > am running this in a kas-isar container on a CentOS 7 host.  My guess is
> > ulimit my possibly be getting in the way.  I will do a minimal kernel
> > config build and then move up from there.
>
> Are you using our recipes-kernel/linux/linux-custom.inc or your own rules?
>
> Jan
>
>
Correct

[-- Attachment #2: Type: text/html, Size: 3699 bytes --]

Re: building custom kernel fails

[Jan Kiszka]

On 2018-04-19 09:56, Jon Nettleton wrote:
> On Thu, Apr 19, 2018 at 9:52 AM Jan Kiszka <jan.kiszka@siemens.com
> <mailto:jan.kiszka@siemens.com>> wrote:
> 
>     On 2018-04-19 09:47, Jon Nettleton wrote:
>     >
>     > On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka <jan.kiszka@siemens.com
>     <mailto:jan.kiszka@siemens.com>
>     > <mailto:jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>>>
>     wrote:
>     >
>     >     On 2018-04-19 09:32, jon@solid-run.com
>     <mailto:jon@solid-run.com> <mailto:jon@solid-run.com
>     <mailto:jon@solid-run.com>> wrote:
>     >     > I am now building a custom debian package from our kernel
>     sources, the
>     >     > kernel compile goes fine.  The deb-pkg is failing because the
>     >     number of
>     >     > files passed to xarg is too long.
>     >     >
>     >     > Error message is.
>     >     >
>     >     > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
>     >     > 'unstable' in the changelog
>     >     > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
>     >     > $KDEB_CHANGELOG_DIST explicitly
>     >     > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
>     >     Argument list
>     >     > too long
>     >     > 2018-04-19 07:11:17 - INFO     - |
>     scripts/package/Makefile:90: recipe
>     >     > for target 'deb-pkg' failed
>     >     > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg]
>     Error 126
>     >     > 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for
>     target
>     >     > 'deb-pkg' failed
>     >     > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
>     >     > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from
>     a shell
>     >     > command.
>     >     >
>     >     > looks like on large kernel builds the file lists will need to be
>     >     broken up.
>     >
>     >     Where exactly is this argument list passed? What exactly is
>     your setup
>     >     (host, target distro, kernel version etc.)?
>     >
>     >     BTW, you can find a working custom kernel build here e.g.:
>     >   
>      https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
>     >     There is an ARM64 version of that as well, just waiting for
>     cross-build
>     >     support to be available.
>     >
>     >     Jan
>     >
>     >
>     >  Yes I have used your recipe as a reference.  It is a 4.9.y lts kernel
>     > with the NXP BSP merged for our hardware. https://www.solid-run.com
>     >
>     > I am attempting to use the default make deb-pkg in that LTS kernel.  I
>     > am running this in a kas-isar container on a CentOS 7 host.  My
>     guess is
>     > ulimit my possibly be getting in the way.  I will do a minimal kernel
>     > config build and then move up from there.
> 
>     Are you using our recipes-kernel/linux/linux-custom.inc or your own
>     rules?
> 
>     Jan
> 
> 
> Correct

Err, there was an "or" in my question...

If your "correct" applied to option 2, I would recommend option 1.

Jan

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 3365 bytes --]

On Thu, Apr 19, 2018 at 9:57 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:

> On 2018-04-19 09:56, Jon Nettleton wrote:
> > On Thu, Apr 19, 2018 at 9:52 AM Jan Kiszka <jan.kiszka@siemens.com
> > <mailto:jan.kiszka@siemens.com>> wrote:
> >
> >     On 2018-04-19 09:47, Jon Nettleton wrote:
> >     >
> >     > On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka <jan.kiszka@siemens.com
> >     <mailto:jan.kiszka@siemens.com>
> >     > <mailto:jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>>>
> >     wrote:
> >     >
> >     >     On 2018-04-19 09:32, jon@solid-run.com
> >     <mailto:jon@solid-run.com> <mailto:jon@solid-run.com
> >     <mailto:jon@solid-run.com>> wrote:
> >     >     > I am now building a custom debian package from our kernel
> >     sources, the
> >     >     > kernel compile goes fine.  The deb-pkg is failing because the
> >     >     number of
> >     >     > files passed to xarg is too long.
> >     >     >
> >     >     > Error message is.
> >     >     >
> >     >     > 2018-04-19 07:11:17 - INFO     - | Using default
> distribution of
> >     >     > 'unstable' in the changelog
> >     >     > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> >     >     > $KDEB_CHANGELOG_DIST explicitly
> >     >     > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> >     >     Argument list
> >     >     > too long
> >     >     > 2018-04-19 07:11:17 - INFO     - |
> >     scripts/package/Makefile:90: recipe
> >     >     > for target 'deb-pkg' failed
> >     >     > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg]
> >     Error 126
> >     >     > 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for
> >     target
> >     >     > 'deb-pkg' failed
> >     >     > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error
> 2
> >     >     > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from
> >     a shell
> >     >     > command.
> >     >     >
> >     >     > looks like on large kernel builds the file lists will need
> to be
> >     >     broken up.
> >     >
> >     >     Where exactly is this argument list passed? What exactly is
> >     your setup
> >     >     (host, target distro, kernel version etc.)?
> >     >
> >     >     BTW, you can find a working custom kernel build here e.g.:
> >     >
> >
> https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
> >     >     There is an ARM64 version of that as well, just waiting for
> >     cross-build
> >     >     support to be available.
> >     >
> >     >     Jan
> >     >
> >     >
> >     >  Yes I have used your recipe as a reference.  It is a 4.9.y lts
> kernel
> >     > with the NXP BSP merged for our hardware.
> https://www.solid-run.com
> >     >
> >     > I am attempting to use the default make deb-pkg in that LTS
> kernel.  I
> >     > am running this in a kas-isar container on a CentOS 7 host.  My
> >     guess is
> >     > ulimit my possibly be getting in the way.  I will do a minimal
> kernel
> >     > config build and then move up from there.
> >
> >     Are you using our recipes-kernel/linux/linux-custom.inc or your own
> >     rules?
> >
> >     Jan
> >
> >
> > Correct
>
> Err, there was an "or" in my question...
>
> If your "correct" applied to option 2, I would recommend option 1.
>
> Jan
>

Sorry was a bit distracted.  I am using your rules, option 1.

[-- Attachment #2: Type: text/html, Size: 5510 bytes --]

Re: building custom kernel fails

[Jan Kiszka]

On 2018-04-19 10:05, Jon Nettleton wrote:
> On Thu, Apr 19, 2018 at 9:57 AM Jan Kiszka <jan.kiszka@siemens.com
> <mailto:jan.kiszka@siemens.com>> wrote:
> 
>     On 2018-04-19 09:56, Jon Nettleton wrote:
>     > On Thu, Apr 19, 2018 at 9:52 AM Jan Kiszka <jan.kiszka@siemens.com
>     <mailto:jan.kiszka@siemens.com>
>     > <mailto:jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>>>
>     wrote:
>     >
>     >     On 2018-04-19 09:47, Jon Nettleton wrote:
>     >     >
>     >     > On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka
>     <jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>
>     >     <mailto:jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>>
>     >     > <mailto:jan.kiszka@siemens.com
>     <mailto:jan.kiszka@siemens.com> <mailto:jan.kiszka@siemens.com
>     <mailto:jan.kiszka@siemens.com>>>>
>     >     wrote:
>     >     >
>     >     >     On 2018-04-19 09:32, jon@solid-run.com
>     <mailto:jon@solid-run.com>
>     >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>
>     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>
>     >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>> wrote:
>     >     >     > I am now building a custom debian package from our kernel
>     >     sources, the
>     >     >     > kernel compile goes fine.  The deb-pkg is failing
>     because the
>     >     >     number of
>     >     >     > files passed to xarg is too long.
>     >     >     >
>     >     >     > Error message is.
>     >     >     >
>     >     >     > 2018-04-19 07:11:17 - INFO     - | Using default
>     distribution of
>     >     >     > 'unstable' in the changelog
>     >     >     > 2018-04-19 07:11:17 - INFO     - | Install lsb-release
>     or set
>     >     >     > $KDEB_CHANGELOG_DIST explicitly
>     >     >     > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
>     >     >     Argument list
>     >     >     > too long
>     >     >     > 2018-04-19 07:11:17 - INFO     - |
>     >     scripts/package/Makefile:90: recipe
>     >     >     > for target 'deb-pkg' failed
>     >     >     > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg]
>     >     Error 126
>     >     >     > 2018-04-19 07:11:17 - INFO     - | Makefile:1347:
>     recipe for
>     >     target
>     >     >     > 'deb-pkg' failed
>     >     >     > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg]
>     Error 2
>     >     >     > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code
>     2 from
>     >     a shell
>     >     >     > command.
>     >     >     >
>     >     >     > looks like on large kernel builds the file lists will
>     need to be
>     >     >     broken up.
>     >     >
>     >     >     Where exactly is this argument list passed? What exactly is
>     >     your setup
>     >     >     (host, target distro, kernel version etc.)?
>     >     >
>     >     >     BTW, you can find a working custom kernel build here e.g.:
>     >     >   
>     >   
>       https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
>     >     >     There is an ARM64 version of that as well, just waiting for
>     >     cross-build
>     >     >     support to be available.
>     >     >
>     >     >     Jan
>     >     >
>     >     >
>     >     >  Yes I have used your recipe as a reference.  It is a 4.9.y
>     lts kernel
>     >     > with the NXP BSP merged for our hardware.
>     https://www.solid-run.com
>     >     >
>     >     > I am attempting to use the default make deb-pkg in that LTS
>     kernel.  I
>     >     > am running this in a kas-isar container on a CentOS 7 host.  My
>     >     guess is
>     >     > ulimit my possibly be getting in the way.  I will do a
>     minimal kernel
>     >     > config build and then move up from there.
>     >
>     >     Are you using our recipes-kernel/linux/linux-custom.inc or
>     your own
>     >     rules?
>     >
>     >     Jan
>     >
>     >
>     > Correct
> 
>     Err, there was an "or" in my question...
> 
>     If your "correct" applied to option 2, I would recommend option 1.
> 
>     Jan
> 
> 
> Sorry was a bit distracted.  I am using your rules, option 1. 

As we are also building older kernels with that rules (4.4-cip namely),
I suspect the problem either comes from the config or BSP-specific
changes. Hope, NXP was not touching stuff again they shouldn't change...

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 5134 bytes --]

On Thu, Apr 19, 2018 at 10:12 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:

> On 2018-04-19 10:05, Jon Nettleton wrote:
> > On Thu, Apr 19, 2018 at 9:57 AM Jan Kiszka <jan.kiszka@siemens.com
> > <mailto:jan.kiszka@siemens.com>> wrote:
> >
> >     On 2018-04-19 09:56, Jon Nettleton wrote:
> >     > On Thu, Apr 19, 2018 at 9:52 AM Jan Kiszka <jan.kiszka@siemens.com
> >     <mailto:jan.kiszka@siemens.com>
> >     > <mailto:jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>>>
> >     wrote:
> >     >
> >     >     On 2018-04-19 09:47, Jon Nettleton wrote:
> >     >     >
> >     >     > On Thu, Apr 19, 2018 at 9:40 AM Jan Kiszka
> >     <jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com>
> >     >     <mailto:jan.kiszka@siemens.com <mailto:jan.kiszka@siemens.com
> >>
> >     >     > <mailto:jan.kiszka@siemens.com
> >     <mailto:jan.kiszka@siemens.com> <mailto:jan.kiszka@siemens.com
> >     <mailto:jan.kiszka@siemens.com>>>>
> >     >     wrote:
> >     >     >
> >     >     >     On 2018-04-19 09:32, jon@solid-run.com
> >     <mailto:jon@solid-run.com>
> >     >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>
> >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>
> >     >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>> wrote:
> >     >     >     > I am now building a custom debian package from our
> kernel
> >     >     sources, the
> >     >     >     > kernel compile goes fine.  The deb-pkg is failing
> >     because the
> >     >     >     number of
> >     >     >     > files passed to xarg is too long.
> >     >     >     >
> >     >     >     > Error message is.
> >     >     >     >
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | Using default
> >     distribution of
> >     >     >     > 'unstable' in the changelog
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | Install lsb-release
> >     or set
> >     >     >     > $KDEB_CHANGELOG_DIST explicitly
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs:
> md5sum:
> >     >     >     Argument list
> >     >     >     > too long
> >     >     >     > 2018-04-19 07:11:17 - INFO     - |
> >     >     scripts/package/Makefile:90: recipe
> >     >     >     > for target 'deb-pkg' failed
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | make[1]: ***
> [deb-pkg]
> >     >     Error 126
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | Makefile:1347:
> >     recipe for
> >     >     target
> >     >     >     > 'deb-pkg' failed
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg]
> >     Error 2
> >     >     >     > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code
> >     2 from
> >     >     a shell
> >     >     >     > command.
> >     >     >     >
> >     >     >     > looks like on large kernel builds the file lists will
> >     need to be
> >     >     >     broken up.
> >     >     >
> >     >     >     Where exactly is this argument list passed? What exactly
> is
> >     >     your setup
> >     >     >     (host, target distro, kernel version etc.)?
> >     >     >
> >     >     >     BTW, you can find a working custom kernel build here
> e.g.:
> >     >     >
> >     >
> >
> https://github.com/siemens/jailhouse-images/tree/master/recipes-kernel/linux
> >     >     >     There is an ARM64 version of that as well, just waiting
> for
> >     >     cross-build
> >     >     >     support to be available.
> >     >     >
> >     >     >     Jan
> >     >     >
> >     >     >
> >     >     >  Yes I have used your recipe as a reference.  It is a 4.9.y
> >     lts kernel
> >     >     > with the NXP BSP merged for our hardware.
> >     https://www.solid-run.com
> >     >     >
> >     >     > I am attempting to use the default make deb-pkg in that LTS
> >     kernel.  I
> >     >     > am running this in a kas-isar container on a CentOS 7 host.
> My
> >     >     guess is
> >     >     > ulimit my possibly be getting in the way.  I will do a
> >     minimal kernel
> >     >     > config build and then move up from there.
> >     >
> >     >     Are you using our recipes-kernel/linux/linux-custom.inc or
> >     your own
> >     >     rules?
> >     >
> >     >     Jan
> >     >
> >     >
> >     > Correct
> >
> >     Err, there was an "or" in my question...
> >
> >     If your "correct" applied to option 2, I would recommend option 1.
> >
> >     Jan
> >
> >
> > Sorry was a bit distracted.  I am using your rules, option 1.
>
> As we are also building older kernels with that rules (4.4-cip namely),
> I suspect the problem either comes from the config or BSP-specific
> changes. Hope, NXP was not touching stuff again they shouldn't change...
>
> Jan


>

We have no problems building packages from within a fakeroot build method.
This seems to be more of a specific environment problem.  I know that
CentOS isn't officially supported, but that is my server platform of
choice.  That is why building from within a debianized container is a great
solution moving forward.

I notice that the kernel is being built and packaged with the root user?
Is there a specific reason for this?

Jon

[-- Attachment #2: Type: text/html, Size: 8950 bytes --]

Re: building custom kernel fails

[Jan Kiszka]

On 2018-04-19 10:18, Jon Nettleton wrote:
> I notice that the kernel is being built and packaged with the root
> user?  Is there a specific reason for this?

There are multiple, in fact. Look for these threads, e.g.:

- https://groups.google.com/d/msgid/isar-users/1496e693-490f-16d6-0957-c9281ed7dd3e%40ilbers.de
- https://groups.google.com/d/msgid/isar-users/20170802132413.GA25215%40iiotirae

Jan

Re: building custom kernel fails

[Henning Schild]

Am Thu, 19 Apr 2018 00:32:21 -0700
schrieb <jon@solid-run.com>:

> I am now building a custom debian package from our kernel sources,
> the kernel compile goes fine.  The deb-pkg is failing because the
> number of files passed to xarg is too long.
> 
> Error message is.
> 
> 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> 'unstable' in the changelog
> 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set 
> $KDEB_CHANGELOG_DIST explicitly
> 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum: Argument
> list too long
> 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90:
> recipe for target 'deb-pkg' failed
> 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
> 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target 
> 'deb-pkg' failed
> 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell 
> command.
> 
> looks like on large kernel builds the file lists will need to be
> broken up.
> 

I have seen that before. It happens in buildchroot when using
qemu-arm-static binfmt magic. Something in that chain seems to cause
problems for very long argument lists.
In my case it was a "make clean" and i removed the files from outside
the buildchroot. But since it now happened again it is probably
something worth looking into.

Henning

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 1689 bytes --]

On Thu, Apr 19, 2018 at 11:19 AM Henning Schild <henning.schild@siemens.com>
wrote:

> Am Thu, 19 Apr 2018 00:32:21 -0700
> schrieb <jon@solid-run.com>:
>
> > I am now building a custom debian package from our kernel sources,
> > the kernel compile goes fine.  The deb-pkg is failing because the
> > number of files passed to xarg is too long.
> >
> > Error message is.
> >
> > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> > 'unstable' in the changelog
> > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> > $KDEB_CHANGELOG_DIST explicitly
> > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum: Argument
> > list too long
> > 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90:
> > recipe for target 'deb-pkg' failed
> > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error 126
> > 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for target
> > 'deb-pkg' failed
> > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a shell
> > command.
> >
> > looks like on large kernel builds the file lists will need to be
> > broken up.
> >
>
> I have seen that before. It happens in buildchroot when using
> qemu-arm-static binfmt magic. Something in that chain seems to cause
> problems for very long argument lists.
> In my case it was a "make clean" and i removed the files from outside
> the buildchroot. But since it now happened again it is probably
> something worth looking into.
>
> Henning
>

Oh interesting.  I wonder if this has something to do with the binfmt
support in CentOS vs Debian.  I will look into it and report back.

Jon

[-- Attachment #2: Type: text/html, Size: 2297 bytes --]

Re: building custom kernel fails

[Henning Schild]

Am Thu, 19 Apr 2018 09:33:28 +0000
schrieb Jon Nettleton <jon@solid-run.com>:

> On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
> <henning.schild@siemens.com> wrote:
> 
> > Am Thu, 19 Apr 2018 00:32:21 -0700
> > schrieb <jon@solid-run.com>:
> >  
> > > I am now building a custom debian package from our kernel sources,
> > > the kernel compile goes fine.  The deb-pkg is failing because the
> > > number of files passed to xarg is too long.
> > >
> > > Error message is.
> > >
> > > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> > > 'unstable' in the changelog
> > > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> > > $KDEB_CHANGELOG_DIST explicitly
> > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> > > Argument list too long
> > > 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90:
> > > recipe for target 'deb-pkg' failed
> > > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error
> > > 126 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for
> > > target 'deb-pkg' failed
> > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a
> > > shell command.
> > >
> > > looks like on large kernel builds the file lists will need to be
> > > broken up.
> > >  
> >
> > I have seen that before. It happens in buildchroot when using
> > qemu-arm-static binfmt magic. Something in that chain seems to cause
> > problems for very long argument lists.
> > In my case it was a "make clean" and i removed the files from
> > outside the buildchroot. But since it now happened again it is
> > probably something worth looking into.
> >
> > Henning
> >  
> 
> Oh interesting.  I wonder if this has something to do with the binfmt
> support in CentOS vs Debian.  I will look into it and report back.

I saw it on Gentoo building in a debian docker container. So i guess
the problem might be related to the binfmt wrapping in general, not
distro specific.

Henning

> Jon

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 2610 bytes --]

On Thu, Apr 19, 2018 at 12:38 PM Henning Schild <henning.schild@siemens.com>
wrote:

> Am Thu, 19 Apr 2018 09:33:28 +0000
> schrieb Jon Nettleton <jon@solid-run.com>:
>
> > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
> > <henning.schild@siemens.com> wrote:
> >
> > > Am Thu, 19 Apr 2018 00:32:21 -0700
> > > schrieb <jon@solid-run.com>:
> > >
> > > > I am now building a custom debian package from our kernel sources,
> > > > the kernel compile goes fine.  The deb-pkg is failing because the
> > > > number of files passed to xarg is too long.
> > > >
> > > > Error message is.
> > > >
> > > > 2018-04-19 07:11:17 - INFO     - | Using default distribution of
> > > > 'unstable' in the changelog
> > > > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> > > > $KDEB_CHANGELOG_DIST explicitly
> > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> > > > Argument list too long
> > > > 2018-04-19 07:11:17 - INFO     - | scripts/package/Makefile:90:
> > > > recipe for target 'deb-pkg' failed
> > > > 2018-04-19 07:11:17 - INFO     - | make[1]: *** [deb-pkg] Error
> > > > 126 2018-04-19 07:11:17 - INFO     - | Makefile:1347: recipe for
> > > > target 'deb-pkg' failed
> > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a
> > > > shell command.
> > > >
> > > > looks like on large kernel builds the file lists will need to be
> > > > broken up.
> > > >
> > >
> > > I have seen that before. It happens in buildchroot when using
> > > qemu-arm-static binfmt magic. Something in that chain seems to cause
> > > problems for very long argument lists.
> > > In my case it was a "make clean" and i removed the files from
> > > outside the buildchroot. But since it now happened again it is
> > > probably something worth looking into.
> > >
> > > Henning
> > >
> >
> > Oh interesting.  I wonder if this has something to do with the binfmt
> > support in CentOS vs Debian.  I will look into it and report back.
>
> I saw it on Gentoo building in a debian docker container. So i guess
> the problem might be related to the binfmt wrapping in general, not
> distro specific.
>
> Henning
>
>
It looks like that was the problem.  I just went through the pain of
building qemu 2.9.1 for Centos7...and all the dependencies that were needed
by it.   That has fixed the custom kernel build issues.  Thanks for all the
pointers.

I would say for now if anyone asks you can consider Centos 7 a no go.  If
there is interest I may add a build to Copr for other users.  Fedora 26 and
newer should be fine.

Jon

[-- Attachment #2: Type: text/html, Size: 3673 bytes --]

Re: building custom kernel fails

[Henning Schild]

Am Thu, 19 Apr 2018 13:27:36 +0000
schrieb Jon Nettleton <jon@solid-run.com>:

> On Thu, Apr 19, 2018 at 12:38 PM Henning Schild
> <henning.schild@siemens.com> wrote:
> 
> > Am Thu, 19 Apr 2018 09:33:28 +0000
> > schrieb Jon Nettleton <jon@solid-run.com>:
> >  
> > > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
> > > <henning.schild@siemens.com> wrote:
> > >  
> > > > Am Thu, 19 Apr 2018 00:32:21 -0700
> > > > schrieb <jon@solid-run.com>:
> > > >  
> > > > > I am now building a custom debian package from our kernel
> > > > > sources, the kernel compile goes fine.  The deb-pkg is
> > > > > failing because the number of files passed to xarg is too
> > > > > long.
> > > > >
> > > > > Error message is.
> > > > >
> > > > > 2018-04-19 07:11:17 - INFO     - | Using default distribution
> > > > > of 'unstable' in the changelog
> > > > > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> > > > > $KDEB_CHANGELOG_DIST explicitly
> > > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> > > > > Argument list too long
> > > > > 2018-04-19 07:11:17 - INFO     - |
> > > > > scripts/package/Makefile:90: recipe for target 'deb-pkg'
> > > > > failed 2018-04-19 07:11:17 - INFO     - | make[1]: ***
> > > > > [deb-pkg] Error 126 2018-04-19 07:11:17 - INFO     - |
> > > > > Makefile:1347: recipe for target 'deb-pkg' failed
> > > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> > > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a
> > > > > shell command.
> > > > >
> > > > > looks like on large kernel builds the file lists will need to
> > > > > be broken up.
> > > > >  
> > > >
> > > > I have seen that before. It happens in buildchroot when using
> > > > qemu-arm-static binfmt magic. Something in that chain seems to
> > > > cause problems for very long argument lists.
> > > > In my case it was a "make clean" and i removed the files from
> > > > outside the buildchroot. But since it now happened again it is
> > > > probably something worth looking into.
> > > >
> > > > Henning
> > > >  
> > >
> > > Oh interesting.  I wonder if this has something to do with the
> > > binfmt support in CentOS vs Debian.  I will look into it and
> > > report back.  
> >
> > I saw it on Gentoo building in a debian docker container. So i guess
> > the problem might be related to the binfmt wrapping in general, not
> > distro specific.
> >
> > Henning
> >
> >  
> It looks like that was the problem.  I just went through the pain of
> building qemu 2.9.1 for Centos7...and all the dependencies that were
> needed by it.   That has fixed the custom kernel build issues.
> Thanks for all the pointers.
> 
> I would say for now if anyone asks you can consider Centos 7 a no
> go.  If there is interest I may add a build to Copr for other users.
> Fedora 26 and newer should be fine.

Isar calls "random" sudos all over the place. Someone actually allowing
this on a productive system surprises me ... to say the least.

My advise it to use a VM or container to run Isar in, and in that case
going for Debian9 is easy. One example, used by many of us, is 
https://hub.docker.com/r/kasproject/kas-isar/

Henning

> Jon

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 3580 bytes --]

On Thu, Apr 19, 2018 at 3:47 PM Henning Schild <henning.schild@siemens.com>
wrote:

> Am Thu, 19 Apr 2018 13:27:36 +0000
> schrieb Jon Nettleton <jon@solid-run.com>:
>
> > On Thu, Apr 19, 2018 at 12:38 PM Henning Schild
> > <henning.schild@siemens.com> wrote:
> >
> > > Am Thu, 19 Apr 2018 09:33:28 +0000
> > > schrieb Jon Nettleton <jon@solid-run.com>:
> > >
> > > > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
> > > > <henning.schild@siemens.com> wrote:
> > > >
> > > > > Am Thu, 19 Apr 2018 00:32:21 -0700
> > > > > schrieb <jon@solid-run.com>:
> > > > >
> > > > > > I am now building a custom debian package from our kernel
> > > > > > sources, the kernel compile goes fine.  The deb-pkg is
> > > > > > failing because the number of files passed to xarg is too
> > > > > > long.
> > > > > >
> > > > > > Error message is.
> > > > > >
> > > > > > 2018-04-19 07:11:17 - INFO     - | Using default distribution
> > > > > > of 'unstable' in the changelog
> > > > > > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
> > > > > > $KDEB_CHANGELOG_DIST explicitly
> > > > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> > > > > > Argument list too long
> > > > > > 2018-04-19 07:11:17 - INFO     - |
> > > > > > scripts/package/Makefile:90: recipe for target 'deb-pkg'
> > > > > > failed 2018-04-19 07:11:17 - INFO     - | make[1]: ***
> > > > > > [deb-pkg] Error 126 2018-04-19 07:11:17 - INFO     - |
> > > > > > Makefile:1347: recipe for target 'deb-pkg' failed
> > > > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg] Error 2
> > > > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2 from a
> > > > > > shell command.
> > > > > >
> > > > > > looks like on large kernel builds the file lists will need to
> > > > > > be broken up.
> > > > > >
> > > > >
> > > > > I have seen that before. It happens in buildchroot when using
> > > > > qemu-arm-static binfmt magic. Something in that chain seems to
> > > > > cause problems for very long argument lists.
> > > > > In my case it was a "make clean" and i removed the files from
> > > > > outside the buildchroot. But since it now happened again it is
> > > > > probably something worth looking into.
> > > > >
> > > > > Henning
> > > > >
> > > >
> > > > Oh interesting.  I wonder if this has something to do with the
> > > > binfmt support in CentOS vs Debian.  I will look into it and
> > > > report back.
> > >
> > > I saw it on Gentoo building in a debian docker container. So i guess
> > > the problem might be related to the binfmt wrapping in general, not
> > > distro specific.
> > >
> > > Henning
> > >
> > >
> > It looks like that was the problem.  I just went through the pain of
> > building qemu 2.9.1 for Centos7...and all the dependencies that were
> > needed by it.   That has fixed the custom kernel build issues.
> > Thanks for all the pointers.
> >
> > I would say for now if anyone asks you can consider Centos 7 a no
> > go.  If there is interest I may add a build to Copr for other users.
> > Fedora 26 and newer should be fine.
>
> Isar calls "random" sudos all over the place. Someone actually allowing
> this on a productive system surprises me ... to say the least.
>
> My advise it to use a VM or container to run Isar in, and in that case
> going for Debian9 is easy. One example, used by many of us, is
> https://hub.docker.com/r/kasproject/kas-isar/
>
> Henning
>
>
Yes that is what I am  using to build with.  I can only imagine that the
newer build of qemu brings in a configuration to the host os that is
required for things to build properly.

Jon

[-- Attachment #2: Type: text/html, Size: 5328 bytes --]

Re: building custom kernel fails

[Alexander Smirnov]

On 04/19/2018 05:26 PM, Jon Nettleton wrote:
> 
> On Thu, Apr 19, 2018 at 3:47 PM Henning Schild 
> <henning.schild@siemens.com <mailto:henning.schild@siemens.com>> wrote:
> 
>     Am Thu, 19 Apr 2018 13:27:36 +0000
>     schrieb Jon Nettleton <jon@solid-run.com <mailto:jon@solid-run.com>>:
> 
>      > On Thu, Apr 19, 2018 at 12:38 PM Henning Schild
>      > <henning.schild@siemens.com <mailto:henning.schild@siemens.com>>
>     wrote:
>      >
>      > > Am Thu, 19 Apr 2018 09:33:28 +0000
>      > > schrieb Jon Nettleton <jon@solid-run.com
>     <mailto:jon@solid-run.com>>:
>      > >
>      > > > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
>      > > > <henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>> wrote:
>      > > >
>      > > > > Am Thu, 19 Apr 2018 00:32:21 -0700
>      > > > > schrieb <jon@solid-run.com <mailto:jon@solid-run.com>>:
>      > > > >
>      > > > > > I am now building a custom debian package from our kernel
>      > > > > > sources, the kernel compile goes fine.  The deb-pkg is
>      > > > > > failing because the number of files passed to xarg is too
>      > > > > > long.
>      > > > > >
>      > > > > > Error message is.
>      > > > > >
>      > > > > > 2018-04-19 07:11:17 - INFO     - | Using default distribution
>      > > > > > of 'unstable' in the changelog
>      > > > > > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or set
>      > > > > > $KDEB_CHANGELOG_DIST explicitly
>      > > > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
>      > > > > > Argument list too long
>      > > > > > 2018-04-19 07:11:17 - INFO     - |
>      > > > > > scripts/package/Makefile:90: recipe for target 'deb-pkg'
>      > > > > > failed 2018-04-19 07:11:17 - INFO     - | make[1]: ***
>      > > > > > [deb-pkg] Error 126 2018-04-19 07:11:17 - INFO     - |
>      > > > > > Makefile:1347: recipe for target 'deb-pkg' failed
>      > > > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg]
>     Error 2
>      > > > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2
>     from a
>      > > > > > shell command.
>      > > > > >
>      > > > > > looks like on large kernel builds the file lists will need to
>      > > > > > be broken up.
>      > > > > >
>      > > > >
>      > > > > I have seen that before. It happens in buildchroot when using
>      > > > > qemu-arm-static binfmt magic. Something in that chain seems to
>      > > > > cause problems for very long argument lists.
>      > > > > In my case it was a "make clean" and i removed the files from
>      > > > > outside the buildchroot. But since it now happened again it is
>      > > > > probably something worth looking into.
>      > > > >
>      > > > > Henning
>      > > > >
>      > > >
>      > > > Oh interesting.  I wonder if this has something to do with the
>      > > > binfmt support in CentOS vs Debian.  I will look into it and
>      > > > report back.
>      > >
>      > > I saw it on Gentoo building in a debian docker container. So i
>     guess
>      > > the problem might be related to the binfmt wrapping in general, not
>      > > distro specific.
>      > >
>      > > Henning
>      > >
>      > >
>      > It looks like that was the problem.  I just went through the pain of
>      > building qemu 2.9.1 for Centos7...and all the dependencies that were
>      > needed by it.   That has fixed the custom kernel build issues.
>      > Thanks for all the pointers.
>      >
>      > I would say for now if anyone asks you can consider Centos 7 a no
>      > go.  If there is interest I may add a build to Copr for other users.
>      > Fedora 26 and newer should be fine.
> 
>     Isar calls "random" sudos all over the place. Someone actually allowing
>     this on a productive system surprises me ... to say the least.
> 
>     My advise it to use a VM or container to run Isar in, and in that case
>     going for Debian9 is easy. One example, used by many of us, is
>     https://hub.docker.com/r/kasproject/kas-isar/
> 
>     Henning
> 
> 
> Yes that is what I am  using to build with.  I can only imagine that the 
> newer build of qemu brings in a configuration to the host os that is 
> required for things to build properly.
> 

My host OS is Fedora for many years, so I use chroot to work with Debian 
environment (for Isar as well):

$ sudo debootstrap jessie debian8-isar/ http://ftp.debian.org/debian
$ sudo chroot debian8-isar

This also could be an option.

Alex

> Jon
> 
> -- 
> You received this message because you are subscribed to the Google 
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to isar-users+unsubscribe@googlegroups.com 
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To post to this group, send email to isar-users@googlegroups.com 
> <mailto:isar-users@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/isar-users/CABdtJHt9xX6HxrdKMPEDnJ5k6hNqgy_DpB-%2BJq1ZQ7UQCiVcPQ%40mail.gmail.com 
> <https://groups.google.com/d/msgid/isar-users/CABdtJHt9xX6HxrdKMPEDnJ5k6hNqgy_DpB-%2BJq1ZQ7UQCiVcPQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

Re: building custom kernel fails

[Jon Nettleton]

[-- Attachment #1: Type: text/plain, Size: 5239 bytes --]

On Thu, Apr 19, 2018 at 4:35 PM Alexander Smirnov <asmirnov@ilbers.de>
wrote:

> On 04/19/2018 05:26 PM, Jon Nettleton wrote:
> >
> > On Thu, Apr 19, 2018 at 3:47 PM Henning Schild
> > <henning.schild@siemens.com <mailto:henning.schild@siemens.com>> wrote:
> >
> >     Am Thu, 19 Apr 2018 13:27:36 +0000
> >     schrieb Jon Nettleton <jon@solid-run.com <mailto:jon@solid-run.com
> >>:
> >
> >      > On Thu, Apr 19, 2018 at 12:38 PM Henning Schild
> >      > <henning.schild@siemens.com <mailto:henning.schild@siemens.com>>
> >     wrote:
> >      >
> >      > > Am Thu, 19 Apr 2018 09:33:28 +0000
> >      > > schrieb Jon Nettleton <jon@solid-run.com
> >     <mailto:jon@solid-run.com>>:
> >      > >
> >      > > > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
> >      > > > <henning.schild@siemens.com
> >     <mailto:henning.schild@siemens.com>> wrote:
> >      > > >
> >      > > > > Am Thu, 19 Apr 2018 00:32:21 -0700
> >      > > > > schrieb <jon@solid-run.com <mailto:jon@solid-run.com>>:
> >      > > > >
> >      > > > > > I am now building a custom debian package from our kernel
> >      > > > > > sources, the kernel compile goes fine.  The deb-pkg is
> >      > > > > > failing because the number of files passed to xarg is too
> >      > > > > > long.
> >      > > > > >
> >      > > > > > Error message is.
> >      > > > > >
> >      > > > > > 2018-04-19 07:11:17 - INFO     - | Using default
> distribution
> >      > > > > > of 'unstable' in the changelog
> >      > > > > > 2018-04-19 07:11:17 - INFO     - | Install lsb-release or
> set
> >      > > > > > $KDEB_CHANGELOG_DIST explicitly
> >      > > > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs: md5sum:
> >      > > > > > Argument list too long
> >      > > > > > 2018-04-19 07:11:17 - INFO     - |
> >      > > > > > scripts/package/Makefile:90: recipe for target 'deb-pkg'
> >      > > > > > failed 2018-04-19 07:11:17 - INFO     - | make[1]: ***
> >      > > > > > [deb-pkg] Error 126 2018-04-19 07:11:17 - INFO     - |
> >      > > > > > Makefile:1347: recipe for target 'deb-pkg' failed
> >      > > > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg]
> >     Error 2
> >      > > > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2
> >     from a
> >      > > > > > shell command.
> >      > > > > >
> >      > > > > > looks like on large kernel builds the file lists will
> need to
> >      > > > > > be broken up.
> >      > > > > >
> >      > > > >
> >      > > > > I have seen that before. It happens in buildchroot when
> using
> >      > > > > qemu-arm-static binfmt magic. Something in that chain seems
> to
> >      > > > > cause problems for very long argument lists.
> >      > > > > In my case it was a "make clean" and i removed the files
> from
> >      > > > > outside the buildchroot. But since it now happened again it
> is
> >      > > > > probably something worth looking into.
> >      > > > >
> >      > > > > Henning
> >      > > > >
> >      > > >
> >      > > > Oh interesting.  I wonder if this has something to do with the
> >      > > > binfmt support in CentOS vs Debian.  I will look into it and
> >      > > > report back.
> >      > >
> >      > > I saw it on Gentoo building in a debian docker container. So i
> >     guess
> >      > > the problem might be related to the binfmt wrapping in general,
> not
> >      > > distro specific.
> >      > >
> >      > > Henning
> >      > >
> >      > >
> >      > It looks like that was the problem.  I just went through the pain
> of
> >      > building qemu 2.9.1 for Centos7...and all the dependencies that
> were
> >      > needed by it.   That has fixed the custom kernel build issues.
> >      > Thanks for all the pointers.
> >      >
> >      > I would say for now if anyone asks you can consider Centos 7 a no
> >      > go.  If there is interest I may add a build to Copr for other
> users.
> >      > Fedora 26 and newer should be fine.
> >
> >     Isar calls "random" sudos all over the place. Someone actually
> allowing
> >     this on a productive system surprises me ... to say the least.
> >
> >     My advise it to use a VM or container to run Isar in, and in that
> case
> >     going for Debian9 is easy. One example, used by many of us, is
> >     https://hub.docker.com/r/kasproject/kas-isar/
> >
> >     Henning
> >
> >
> > Yes that is what I am  using to build with.  I can only imagine that the
> > newer build of qemu brings in a configuration to the host os that is
> > required for things to build properly.
> >
>
> My host OS is Fedora for many years, so I use chroot to work with Debian
> environment (for Isar as well):
>
> $ sudo debootstrap jessie debian8-isar/ http://ftp.debian.org/debian
> $ sudo chroot debian8-isar
>
> This also could be an option.
>
> Alex
>
>
yes very true, thanks.  I am specifically interested in using docker
containers to build images so I can bring up a build farm for customers
that want to build production images.  My main builder is x86_64 however I
am looking forward to comparing the performance of that machine vs our
"new" ARM64 server boards.  Of course I can run run debootstrap within the
container first, but since CIP is using Kas I also want to use that
infrastructure.

-Jon

[-- Attachment #2: Type: text/html, Size: 8100 bytes --]

Re: building custom kernel fails

[Jan Kiszka]

On 2018-04-19 16:40, Jon Nettleton wrote:
> 
> On Thu, Apr 19, 2018 at 4:35 PM Alexander Smirnov <asmirnov@ilbers.de
> <mailto:asmirnov@ilbers.de>> wrote:
> 
>     On 04/19/2018 05:26 PM, Jon Nettleton wrote:
>     >
>     > On Thu, Apr 19, 2018 at 3:47 PM Henning Schild
>     > <henning.schild@siemens.com <mailto:henning.schild@siemens.com>
>     <mailto:henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>>> wrote:
>     >
>     >     Am Thu, 19 Apr 2018 13:27:36 +0000
>     >     schrieb Jon Nettleton <jon@solid-run.com
>     <mailto:jon@solid-run.com> <mailto:jon@solid-run.com
>     <mailto:jon@solid-run.com>>>:
>     >
>     >      > On Thu, Apr 19, 2018 at 12:38 PM Henning Schild
>     >      > <henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>
>     <mailto:henning.schild@siemens.com <mailto:henning.schild@siemens.com>>>
>     >     wrote:
>     >      >
>     >      > > Am Thu, 19 Apr 2018 09:33:28 +0000
>     >      > > schrieb Jon Nettleton <jon@solid-run.com
>     <mailto:jon@solid-run.com>
>     >     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>>:
>     >      > >
>     >      > > > On Thu, Apr 19, 2018 at 11:19 AM Henning Schild
>     >      > > > <henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>
>     >     <mailto:henning.schild@siemens.com
>     <mailto:henning.schild@siemens.com>>> wrote:
>     >      > > >
>     >      > > > > Am Thu, 19 Apr 2018 00:32:21 -0700
>     >      > > > > schrieb <jon@solid-run.com <mailto:jon@solid-run.com>
>     <mailto:jon@solid-run.com <mailto:jon@solid-run.com>>>:
>     >      > > > >
>     >      > > > > > I am now building a custom debian package from our
>     kernel
>     >      > > > > > sources, the kernel compile goes fine.  The deb-pkg is
>     >      > > > > > failing because the number of files passed to xarg
>     is too
>     >      > > > > > long.
>     >      > > > > >
>     >      > > > > > Error message is.
>     >      > > > > >
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | Using default
>     distribution
>     >      > > > > > of 'unstable' in the changelog
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | Install
>     lsb-release or set
>     >      > > > > > $KDEB_CHANGELOG_DIST explicitly
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | /usr/bin/xargs:
>     md5sum:
>     >      > > > > > Argument list too long
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - |
>     >      > > > > > scripts/package/Makefile:90: recipe for target
>     'deb-pkg'
>     >      > > > > > failed 2018-04-19 07:11:17 - INFO     - | make[1]: ***
>     >      > > > > > [deb-pkg] Error 126 2018-04-19 07:11:17 - INFO     - |
>     >      > > > > > Makefile:1347: recipe for target 'deb-pkg' failed
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | make: *** [deb-pkg]
>     >     Error 2
>     >      > > > > > 2018-04-19 07:11:17 - INFO     - | WARNING: exit code 2
>     >     from a
>     >      > > > > > shell command.
>     >      > > > > >
>     >      > > > > > looks like on large kernel builds the file lists
>     will need to
>     >      > > > > > be broken up.
>     >      > > > > >
>     >      > > > >
>     >      > > > > I have seen that before. It happens in buildchroot
>     when using
>     >      > > > > qemu-arm-static binfmt magic. Something in that chain
>     seems to
>     >      > > > > cause problems for very long argument lists.
>     >      > > > > In my case it was a "make clean" and i removed the
>     files from
>     >      > > > > outside the buildchroot. But since it now happened
>     again it is
>     >      > > > > probably something worth looking into.
>     >      > > > >
>     >      > > > > Henning
>     >      > > > >
>     >      > > >
>     >      > > > Oh interesting.  I wonder if this has something to do
>     with the
>     >      > > > binfmt support in CentOS vs Debian.  I will look into
>     it and
>     >      > > > report back.
>     >      > >
>     >      > > I saw it on Gentoo building in a debian docker container.
>     So i
>     >     guess
>     >      > > the problem might be related to the binfmt wrapping in
>     general, not
>     >      > > distro specific.
>     >      > >
>     >      > > Henning
>     >      > >
>     >      > >
>     >      > It looks like that was the problem.  I just went through
>     the pain of
>     >      > building qemu 2.9.1 for Centos7...and all the dependencies
>     that were
>     >      > needed by it.   That has fixed the custom kernel build issues.
>     >      > Thanks for all the pointers.
>     >      >
>     >      > I would say for now if anyone asks you can consider Centos
>     7 a no
>     >      > go.  If there is interest I may add a build to Copr for
>     other users.
>     >      > Fedora 26 and newer should be fine.
>     >
>     >     Isar calls "random" sudos all over the place. Someone actually
>     allowing
>     >     this on a productive system surprises me ... to say the least.
>     >
>     >     My advise it to use a VM or container to run Isar in, and in
>     that case
>     >     going for Debian9 is easy. One example, used by many of us, is
>     >     https://hub.docker.com/r/kasproject/kas-isar/
>     >
>     >     Henning
>     >
>     >
>     > Yes that is what I am  using to build with.  I can only imagine
>     that the
>     > newer build of qemu brings in a configuration to the host os that is
>     > required for things to build properly.
>     >
> 
>     My host OS is Fedora for many years, so I use chroot to work with
>     Debian
>     environment (for Isar as well):
> 
>     $ sudo debootstrap jessie debian8-isar/ http://ftp.debian.org/debian
>     $ sudo chroot debian8-isar
> 
>     This also could be an option.
> 
>     Alex
> 
> 
> yes very true, thanks.  I am specifically interested in using docker
> containers to build images so I can bring up a build farm for customers
> that want to build production images.  My main builder is x86_64 however
> I am looking forward to comparing the performance of that machine vs our
> "new" ARM64 server boards.

We are looking into very similar questions. We have a build
infrastructure ready based on gitlab-ci (you can find the yml file in
Isar, in fact), and we would also be happy to recommend public CI like
Travis or Shippable for Isar builds.

As pointed out in the other branch of this thread, there are some
challenges preventing unprivileged builds right now. That is at least
the need to mount images or paths inside the container and the need to
configure binfmt according to Debian expectations.

If you have a CI system where you can spawn a VM on each build and that
VM can run a (privileged) Docker container, you could move forward
already despite of this. Would be our plan B here, but it's not set up
with three clicks either.

>  Of course I can run run debootstrap within
> the container first, but since CIP is using Kas I also want to use that
> infrastructure.

Using kas quickly pays off when it come to managing sources and versions
of your dependencies, or providing your users a way to put a layer on
top of your released configuration.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux