[PATCH] install crossbuild-essential into base schroot

[PATCH] install crossbuild-essential into base schroot

[Felix Moessbauer]

This patch always installs the crossbuild-essential-<distro>
package into the lower dir of the host sbuild schroot.

This avoids having to install this package in the upper-dir
when cross-compiling, and is identical to build-essential when natively
compiling.
By that, the parallel cross-build of packages is significantly faster.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
index aa82846b..f7c7d11b 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
@@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
 
 require sbuild-chroot.inc
 
+# always add the crossbuild-essentials package which is
+# a noop for the host distro, but significantly reduces
+# the download + install time when cross-compiling
+SBUILD_CHROOT_PREINSTALL_COMMON += "crossbuild-essential-${DISTRO_ARCH}"
+
 ROOTFS_ARCH = "${HOST_ARCH}"
 ROOTFS_DISTRO = "${HOST_DISTRO}"
-- 
2.30.2

Re: [PATCH] install crossbuild-essential into base schroot

[Jan Kiszka]

On 03.08.22 18:44, Felix Moessbauer wrote:
> This patch always installs the crossbuild-essential-<distro>

<distro-arch>

> package into the lower dir of the host sbuild schroot.
> 
> This avoids having to install this package in the upper-dir
> when cross-compiling, and is identical to build-essential when natively
> compiling.
> By that, the parallel cross-build of packages is significantly faster.
>

I was wondering if that should be made dependent on ISAR_CROSS_COMPILE,
even though users could turn it off globally and only enable it for
individual packages?

Jan

> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> index aa82846b..f7c7d11b 100644
> --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> @@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
>  
>  require sbuild-chroot.inc
>  
> +# always add the crossbuild-essentials package which is
> +# a noop for the host distro, but significantly reduces
> +# the download + install time when cross-compiling
> +SBUILD_CHROOT_PREINSTALL_COMMON += "crossbuild-essential-${DISTRO_ARCH}"
> +
>  ROOTFS_ARCH = "${HOST_ARCH}"
>  ROOTFS_DISTRO = "${HOST_DISTRO}"


-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [PATCH] install crossbuild-essential into base schroot

[Uladzimir Bely]

In the email from Thursday, 4 August 2022 09:07:08 +03 user Jan Kiszka wrote:
> On 03.08.22 18:44, Felix Moessbauer wrote:
> > This patch always installs the crossbuild-essential-<distro>
> 
> <distro-arch>
> 
> > package into the lower dir of the host sbuild schroot.
> > 
> > This avoids having to install this package in the upper-dir
> > when cross-compiling, and is identical to build-essential when natively
> > compiling.
> > By that, the parallel cross-build of packages is significantly faster.
> 
> I was wondering if that should be made dependent on ISAR_CROSS_COMPILE,
> even though users could turn it off globally and only enable it for
> individual packages?
> 
> Jan
> 

This might be even more conditional.

Depending on compat arch option state, we might actually need crossbuild-
essential-${COMPAT_DISTRO_ARCH} instead. Even in "native" builds, even for 
sbuild-chroot-target...

> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> > 
> >  meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb index
> > aa82846b..f7c7d11b 100644
> > --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > @@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
> > 
> >  require sbuild-chroot.inc
> > 
> > +# always add the crossbuild-essentials package which is
> > +# a noop for the host distro, but significantly reduces
> > +# the download + install time when cross-compiling
> > +SBUILD_CHROOT_PREINSTALL_COMMON += "crossbuild-essential-${DISTRO_ARCH}"
> > +
> > 
> >  ROOTFS_ARCH = "${HOST_ARCH}"
> >  ROOTFS_DISTRO = "${HOST_DISTRO}"


-- 
Uladzimir Bely

RE: [PATCH] install crossbuild-essential into base schroot

[Moessbauer, Felix]

> -----Original Message-----
> From: Uladzimir Bely <ubely@ilbers.de>
> Sent: Thursday, August 4, 2022 8:23 AM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>;
> isar-users@googlegroups.com
> Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>
> Subject: Re: [PATCH] install crossbuild-essential into base schroot
> 
> In the email from Thursday, 4 August 2022 09:07:08 +03 user Jan Kiszka wrote:
> > On 03.08.22 18:44, Felix Moessbauer wrote:
> > > This patch always installs the crossbuild-essential-<distro>
> >
> > <distro-arch>
> >
> > > package into the lower dir of the host sbuild schroot.
> > >
> > > This avoids having to install this package in the upper-dir when
> > > cross-compiling, and is identical to build-essential when natively
> > > compiling.
> > > By that, the parallel cross-build of packages is significantly faster.
> >
> > I was wondering if that should be made dependent on
> > ISAR_CROSS_COMPILE, even though users could turn it off globally and
> > only enable it for individual packages?

I tried that before, but this leads to non-deterministic build data, as the ISAR_CROSS_COMPILE can be switched on / off on a per-receipt basis, but the lower-dir of the schroot is shared across all recipes.

> >
> > Jan
> >
> 
> This might be even more conditional.
> 
> Depending on compat arch option state, we might actually need crossbuild-
> essential-${COMPAT_DISTRO_ARCH} instead. Even in "native" builds, even
> for sbuild-chroot-target...

In the end, this is a tradeoff. Having sbuild default-dependencies in the base schroot does not affect correctness of the build (they will be installed later on anyways).
It simply reduces the amount of work that has to be done per recipe.

This patch improves the situation for cross-builds.
Adding support for COMPAT_DISTRO_ARCH is a different story.

Felix

> 
> > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > > ---
> > >
> > >  meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
> > >  1 file changed, 5 insertions(+)
> > >
> > > diff --git
> > > a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb index
> > > aa82846b..f7c7d11b 100644
> > > --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > @@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
> > >
> > >  require sbuild-chroot.inc
> > >
> > > +# always add the crossbuild-essentials package which is # a noop
> > > +for the host distro, but significantly reduces # the download +
> > > +install time when cross-compiling
> SBUILD_CHROOT_PREINSTALL_COMMON
> > > ++= "crossbuild-essential-${DISTRO_ARCH}"
> > > +
> > >
> > >  ROOTFS_ARCH = "${HOST_ARCH}"
> > >  ROOTFS_DISTRO = "${HOST_DISTRO}"
> 
> 
> --
> Uladzimir Bely
> 
> 

Re: [PATCH] install crossbuild-essential into base schroot

[Anton Mikanovich]

03.08.2022 19:44, Felix Moessbauer wrote:
> This patch always installs the crossbuild-essential-<distro>
> package into the lower dir of the host sbuild schroot.
>
> This avoids having to install this package in the upper-dir
> when cross-compiling, and is identical to build-essential when natively
> compiling.
> By that, the parallel cross-build of packages is significantly faster.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>   meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
>   1 file changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> index aa82846b..f7c7d11b 100644
> --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> @@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
>   
>   require sbuild-chroot.inc
>   
> +# always add the crossbuild-essentials package which is
> +# a noop for the host distro, but significantly reduces
> +# the download + install time when cross-compiling
> +SBUILD_CHROOT_PREINSTALL_COMMON += "crossbuild-essential-${DISTRO_ARCH}"
> +
>   ROOTFS_ARCH = "${HOST_ARCH}"
>   ROOTFS_DISTRO = "${HOST_DISTRO}"

This will probably not work on riscv64 and any other targets with self-build
crossbuild-essential.

Re: [PATCH] install crossbuild-essential into base schroot

[Jan Kiszka]

On 04.08.22 08:22, Uladzimir Bely wrote:
> In the email from Thursday, 4 August 2022 09:07:08 +03 user Jan Kiszka wrote:
>> On 03.08.22 18:44, Felix Moessbauer wrote:
>>> This patch always installs the crossbuild-essential-<distro>
>>
>> <distro-arch>
>>
>>> package into the lower dir of the host sbuild schroot.
>>>
>>> This avoids having to install this package in the upper-dir
>>> when cross-compiling, and is identical to build-essential when natively
>>> compiling.
>>> By that, the parallel cross-build of packages is significantly faster.
>>
>> I was wondering if that should be made dependent on ISAR_CROSS_COMPILE,
>> even though users could turn it off globally and only enable it for
>> individual packages?
>>
>> Jan
>>
> 
> This might be even more conditional.
> 
> Depending on compat arch option state, we might actually need crossbuild-
> essential-${COMPAT_DISTRO_ARCH} instead. Even in "native" builds, even for 
> sbuild-chroot-target...
> 

Indeed, good point. Though all that remains optimization, and we would
simply continue to install those deps on a package-by-package case.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

RE: [PATCH] install crossbuild-essential into base schroot

[Moessbauer, Felix]

> -----Original Message-----
> From: Anton Mikanovich <amikan@ilbers.de>
> Sent: Thursday, August 4, 2022 9:36 AM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>;
> isar-users@googlegroups.com
> Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Uladzimir Bely
> <ubely@ilbers.de>
> Subject: Re: [PATCH] install crossbuild-essential into base schroot
> 
> 03.08.2022 19:44, Felix Moessbauer wrote:
> > This patch always installs the crossbuild-essential-<distro> package
> > into the lower dir of the host sbuild schroot.
> >
> > This avoids having to install this package in the upper-dir when
> > cross-compiling, and is identical to build-essential when natively
> > compiling.
> > By that, the parallel cross-build of packages is significantly faster.
> >
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >   meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
> >   1 file changed, 5 insertions(+)
> >
> > diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > index aa82846b..f7c7d11b 100644
> > --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > @@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
> >
> >   require sbuild-chroot.inc
> >
> > +# always add the crossbuild-essentials package which is # a noop for
> > +the host distro, but significantly reduces # the download + install
> > +time when cross-compiling SBUILD_CHROOT_PREINSTALL_COMMON +=
> > +"crossbuild-essential-${DISTRO_ARCH}"
> > +
> >   ROOTFS_ARCH = "${HOST_ARCH}"
> >   ROOTFS_DISTRO = "${HOST_DISTRO}"
> 
> This will probably not work on riscv64 and any other targets with self-build
> crossbuild-essential.

The CI did not report any issues with this patch, but maybe an riscv64 test is missing...
Either we exclude these community-ports of Debian, or we simply just apply the patches for the architectures we support and we know it works (like armhf, arm64).

Even if this is just a performance thing, in layers with dozens of packages the overhead of just installing the compilers is huge.
It also bloats the upper-dir of the schroot leading to significant disk consumption.

Jan recently reported that the build time raised a lot when switching an arm64 layer to from ISAR v0.9 to ISAR-next (sbuild).
Very likely most of that came from the crossbuild-essential packages.

Felix

Re: [PATCH] install crossbuild-essential into base schroot

[Uladzimir Bely]

[-- Attachment #1: Type: text/plain, Size: 3601 bytes --]

In the email from Thursday, 4 August 2022 10:56:40 +03 user Moessbauer, Felix wrote:
> > -----Original Message-----
> > From: Anton Mikanovich <amikan@ilbers.de>
> > Sent: Thursday, August 4, 2022 9:36 AM
> > To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>;
> > isar-users@googlegroups.com
> > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Uladzimir Bely
> > <ubely@ilbers.de>
> > Subject: Re: [PATCH] install crossbuild-essential into base schroot
> > 
> > 03.08.2022 19:44, Felix Moessbauer wrote:
> > 
> > > This patch always installs the crossbuild-essential-<distro> package
> > > into the lower dir of the host sbuild schroot.
> > >
> > >
> > >
> > > This avoids having to install this package in the upper-dir when
> > > cross-compiling, and is identical to build-essential when natively
> > > compiling.
> > > By that, the parallel cross-build of packages is significantly faster.
> > >
> > >
> > >
> > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > > ---
> > > 
> > >   meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb | 5 +++++
> > >   1 file changed, 5 insertions(+)
> > >
> > >
> > >
> > > diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > index aa82846b..f7c7d11b 100644
> > > --- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > +++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb
> > > @@ -9,5 +9,10 @@ SBUILD_VARIANT = "host"
> > >
> > >
> > >
> > >   require sbuild-chroot.inc
> > >
> > >
> > >
> > > +# always add the crossbuild-essentials package which is # a noop for
> > > +the host distro, but significantly reduces # the download + install
> > > +time when cross-compiling SBUILD_CHROOT_PREINSTALL_COMMON +=
> > > +"crossbuild-essential-${DISTRO_ARCH}"
> > > +
> > > 
> > >   ROOTFS_ARCH = "${HOST_ARCH}"
> > >   ROOTFS_DISTRO = "${HOST_DISTRO}"
> > 
> > 
> > This will probably not work on riscv64 and any other targets with
> > self-build
 crossbuild-essential.
> 
> 
> The CI did not report any issues with this patch, but maybe an riscv64 test
> is missing...
 Either we exclude these community-ports of Debian, or we
> simply just apply the patches for the architectures we support and we know
> it works (like armhf, arm64). 
> Even if this is just a performance thing, in layers with dozens of packages
> the overhead of just installing the compilers is huge.
 It also bloats the
> upper-dir of the schroot leading to significant disk consumption. 
> Jan recently reported that the build time raised a lot when switching an
> arm64 layer to from ISAR v0.9 to ISAR-next (sbuild).
 Very likely most of
> that came from the crossbuild-essential packages. 
> Felix

We could also simply control it from local.conf at downstream level:

SBUILD_CHROOT_PREINSTALL += "${@ 'crossbuild-essential-${DISTRO_ARCH}' if 
d.getVar('SBUILD_VARIANT') == 'host' else ''}"

This still just needs a small change in SBUILD_CHROOT_PREINSTALL definition in sbuild-
chroot.inc (replace '?=' with '+=')

builder@9f6a3f775537:/build/bc$ bitbake -e mc:qemuarm-bullseye:sbuild-chroot-target | grep 
"SBUILD_CHROOT_PREINSTALL=" 
SBUILD_CHROOT_PREINSTALL="fakeroot build-essential debhelper 
ccache devscripts equivs" 
builder@9f6a3f775537:/build/bc$ bitbake -e mc:qemuarm-bullseye:sbuild-chroot-host | grep 
"SBUILD_CHROOT_PREINSTALL="       
SBUILD_CHROOT_PREINSTALL="crossbuild-essential-armhf fakeroot build-
essential debhelper ccache devscripts equivs  "
This approach looks more flexible for me.

-- 
Uladzimir Bely


[-- Attachment #2: Type: text/html, Size: 10951 bytes --]

Re: [PATCH] install crossbuild-essential into base schroot

[Anton Mikanovich]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]

04.08.2022 10:56, Moessbauer, Felix wrote:
> The CI did not report any issues with this patch, but maybe an riscv64 test is missing...
> Either we exclude these community-ports of Debian, or we simply just apply the patches for the architectures we support and we know it works (like armhf, arm64).
>
> Even if this is just a performance thing, in layers with dozens of packages the overhead of just installing the compilers is huge.
> It also bloats the upper-dir of the schroot leading to significant disk consumption.
>
> Jan recently reported that the build time raised a lot when switching an arm64 layer to from ISAR v0.9 to ISAR-next (sbuild).
> Very likely most of that came from the crossbuild-essential packages.
>
> Felix

Hello Felix,

I've just checked mc:qemuriscv64-sid-ports:isar-image-base target 
manually and
it fails with:

| E: Unable to locate package crossbuild-essential-riscv64

on do_rootfs_install task of sbuild-chroot-host recipe, which was 
predictable.
It was not failed on CI because of KFAIL on Sid-Ports targets (they are not
stable enough).

Overall preinstalling is good idea and will improve building speed, but we
should implement it in the right way.

[-- Attachment #2: Type: text/html, Size: 1956 bytes --]