multi MACHINE build for same DISTRO_ARCH

multi MACHINE build for same DISTRO_ARCH

[Benedikt Niedermayr]

Hi guys,

before I send some patches to you I would like to discuss this a bit.

The problem is to run multi machine builds for the same DISTRO_ARCH.

We have for example a BSP layer which supports imx6, imx7 and imx8 
architectures, and we like to be able to run

those builds within a single bitbake command:


bitbake multiconfig:imx6-stretch:isar-image-base \

              multiconfig:imx7-stretch:isar-image-base \

              multiconfig:imx8-stretch:isar-image-base


The build runs fine if DISTRO_ARCH differs between targets. But it does 
not run successfully if only MACHINE differs and DISTRO_ARCH is the same.


I think running multi machine build within one bitbake build folder 
makes a lot of sense, especially when it comes to build time and CI.

I don't like it to create a new build folder for every MACHINE, although 
everything (rootfs, buildchroot, etc.) stays almost the same among 
DISTRO_ARCHs.



Is anybody familiar with this behavior?



Thanks,

Benni

Re: multi MACHINE build for same DISTRO_ARCH

[Jan Kiszka]

On 01.07.19 16:41, 'Benedikt Niedermayr' via isar-users wrote:
> Hi guys,
> 
> before I send some patches to you I would like to discuss this a bit.
> 
> The problem is to run multi machine builds for the same DISTRO_ARCH.
> 
> We have for example a BSP layer which supports imx6, imx7 and imx8 
> architectures, and we like to be able to run
> 
> those builds within a single bitbake command:
> 
> 
> bitbake multiconfig:imx6-stretch:isar-image-base \
> 
>               multiconfig:imx7-stretch:isar-image-base \
> 
>               multiconfig:imx8-stretch:isar-image-base
> 
> 
> The build runs fine if DISTRO_ARCH differs between targets. But it does not run 
> successfully if only MACHINE differs and DISTRO_ARCH is the same.
> 
> 
> I think running multi machine build within one bitbake build folder makes a lot 
> of sense, especially when it comes to build time and CI.
> 
> I don't like it to create a new build folder for every MACHINE, although 
> everything (rootfs, buildchroot, etc.) stays almost the same among DISTRO_ARCHs.
> 
> 
> 
> Is anybody familiar with this behavior?
> 

You may want to study https://github.com/siemens/jailhouse-images for a working 
pattern. May not be the only possible one, but already that required some 
careful balancing.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: multi MACHINE build for same DISTRO_ARCH

[Benedikt Niedermayr]

Am 01.07.19 um 16:51 schrieb Jan Kiszka:
> On 01.07.19 16:41, 'Benedikt Niedermayr' via isar-users wrote:
>> Hi guys,
>>
>> before I send some patches to you I would like to discuss this a bit.
>>
>> The problem is to run multi machine builds for the same DISTRO_ARCH.
>>
>> We have for example a BSP layer which supports imx6, imx7 and imx8 
>> architectures, and we like to be able to run
>>
>> those builds within a single bitbake command:
>>
>>
>> bitbake multiconfig:imx6-stretch:isar-image-base \
>>
>>               multiconfig:imx7-stretch:isar-image-base \
>>
>>               multiconfig:imx8-stretch:isar-image-base
>>
>>
>> The build runs fine if DISTRO_ARCH differs between targets. But it 
>> does not run successfully if only MACHINE differs and DISTRO_ARCH is 
>> the same.
>>
>>
>> I think running multi machine build within one bitbake build folder 
>> makes a lot of sense, especially when it comes to build time and CI.
>>
>> I don't like it to create a new build folder for every MACHINE, 
>> although everything (rootfs, buildchroot, etc.) stays almost the same 
>> among DISTRO_ARCHs.
>>
>>
>>
>> Is anybody familiar with this behavior?
>>
>
> You may want to study https://github.com/siemens/jailhouse-images for 
> a working pattern. May not be the only possible one, but already that 
> required some careful balancing.
>
> Jan
>

Hi Jan,

thanks.


I will take a look at it and reply ASAP.

Re: multi MACHINE build for same DISTRO_ARCH

[Henning Schild]

Hi,

the key is to get the split between machine and distro configs right.
And to make sure all your recipes create generic enough packages that
can be shared or differ in name/version where not.

The great sharing potential, reducing build time and size, but also
means a high risk of false sharing if you make a mistake. I
personally avoid multiconfig builds for that reason.

Henning

Am Mon, 1 Jul 2019 16:41:36 +0200
schrieb 'Benedikt Niedermayr' via isar-users
<isar-users@googlegroups.com>:

> Hi guys,
> 
> before I send some patches to you I would like to discuss this a bit.
> 
> The problem is to run multi machine builds for the same DISTRO_ARCH.
> 
> We have for example a BSP layer which supports imx6, imx7 and imx8 
> architectures, and we like to be able to run
> 
> those builds within a single bitbake command:
> 
> 
> bitbake multiconfig:imx6-stretch:isar-image-base \
> 
>               multiconfig:imx7-stretch:isar-image-base \
> 
>               multiconfig:imx8-stretch:isar-image-base
> 
> 
> The build runs fine if DISTRO_ARCH differs between targets. But it
> does not run successfully if only MACHINE differs and DISTRO_ARCH is
> the same.
> 
> 
> I think running multi machine build within one bitbake build folder 
> makes a lot of sense, especially when it comes to build time and CI.
> 
> I don't like it to create a new build folder for every MACHINE,
> although everything (rootfs, buildchroot, etc.) stays almost the same
> among DISTRO_ARCHs.
> 
> 
> 
> Is anybody familiar with this behavior?
> 
> 
> 
> Thanks,
> 
> Benni
> 
> 
> 
> 

Re: multi MACHINE build for same DISTRO_ARCH

[Baurzhan Ismagulov]

Hello Benedikt,

On Mon, Jul 01, 2019 at 04:41:36PM +0200, 'Benedikt Niedermayr' via isar-users wrote:
> bitbake multiconfig:imx6-stretch:isar-image-base \
> � � � � � �� multiconfig:imx7-stretch:isar-image-base \
> � � � � � �� multiconfig:imx8-stretch:isar-image-base
> 
> The build runs fine if DISTRO_ARCH differs between targets. But it does not
> run successfully if only MACHINE differs and DISTRO_ARCH is the same.

Where does it fail?

Which Isar revision?


> I think running multi machine build within one bitbake build folder makes a
> lot of sense, especially when it comes to build time and CI.

Yes, we are interested in that use case.


> Is anybody familiar with this behavior?

We used to have issues with qemuarm-stretch and de0-nano-soc-stretch at least
in cross-build, but it didn't occur in recent revisions (enabled in CI in
56cc472).


With kind regards,
Baurzhan.

Re: multi MACHINE build for same DISTRO_ARCH

[benbrenson89]

[-- Attachment #1.1: Type: text/plain, Size: 592 bytes --]

> Where does it fail? 
It fails on different points within the build. In my custom layer 
configuration it fails when building the u-boot-script recipe.

IMHO the problem appears due to running those recipes in parallel (for each 
multiconfig) and  some races appear
when mounting/unmounting the WORKDIR into the rootfs.

The next problem may appear if created packages of one MACHINE will 
overwritten by the same package of another MACHINE.
Packages must contain a machine specific suffix in order to fix that. 


> Which Isar revision? 
I ran those builds with the current next branch.




[-- Attachment #1.2: Type: text/html, Size: 823 bytes --]

Re: multi MACHINE build for same DISTRO_ARCH

[Jan Kiszka]

On 10.07.19 20:52, benbrenson89 via isar-users wrote:
>> Where does it fail?
> It fails on different points within the build. In my custom layer configuration
> it fails when building the u-boot-script recipe.
> 
> IMHO the problem appears due to running those recipes in parallel (for each
> multiconfig) and  some races appear
> when mounting/unmounting the WORKDIR into the rootfs.
> 

Vanilla u-boot-script is only built once per target arch. All normal specifics
are in the u-boot env. Or in /etc/default/u-boot-script that can be adjusted by
a machine-specific customization package. If you do the adjustment via
u-boot-script directly, you need to make that package in your overlay
machine-specific:

PN .= "-${MACHINE}"

> The next problem may appear if created packages of one MACHINE will overwritten
> by the same package of another MACHINE.
> Packages must contain a machine specific suffix in order to fix that.
> 

See above if you are deviating from the default.

Again, please study jailhouse-images carefully. It contains working pattern and
- to my best knowledge - has no multiconfig issues.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: multi MACHINE build for same DISTRO_ARCH

[Henning Schild]

Am Wed, 10 Jul 2019 21:09:21 +0200
schrieb "[ext] Jan Kiszka" <jan.kiszka@siemens.com>:

> On 10.07.19 20:52, benbrenson89 via isar-users wrote:
> >> Where does it fail?  
> > It fails on different points within the build. In my custom layer
> > configuration it fails when building the u-boot-script recipe.
> > 
> > IMHO the problem appears due to running those recipes in parallel
> > (for each multiconfig) and  some races appear
> > when mounting/unmounting the WORKDIR into the rootfs.
> >   
> 
> Vanilla u-boot-script is only built once per target arch. All normal
> specifics are in the u-boot env. Or in /etc/default/u-boot-script
> that can be adjusted by a machine-specific customization package. If
> you do the adjustment via u-boot-script directly, you need to make
> that package in your overlay machine-specific:
> 
> PN .= "-${MACHINE}"
> 
> > The next problem may appear if created packages of one MACHINE will
> > overwritten by the same package of another MACHINE.
> > Packages must contain a machine specific suffix in order to fix
> > that. 
>
> See above if you are deviating from the default.

And only for those that actually can not be shared. If you do that for
all your packages you might as well not use multiconfig because your
sharing goes down to nothing but buildchroot.

Henning

> Again, please study jailhouse-images carefully. It contains working
> pattern and
> - to my best knowledge - has no multiconfig issues.
> 
> Jan
>