Re: [RFC PATCH] native.bbclass

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

On Tue, 2023-01-17 at 19:03 +0100, Adriaan Schmidt wrote:
> Hi all,
> 
> as mentioned by Jan, I have another approach of building
> native/host/compat
> arch packages, inspired by OE's native class.
> 
> The way it works is:
> - A recipe can inherit native.bbclass (or, if we like we could even
> add it
>   to all recipes via the dpkg class).
> - For recipes that inherit native.bbclass, bitbake generates a new
> bitbake
>   target with suffix `-native`. Recipes can DEPEND on those `-native`
> packages.
>   My use case that made me develop this is goreleaser, a build tool
>   for which I have a bitbake recipe. If I want to crosscompile, I
> need it
>   for the host architecture. By DEPENDing on `goreleaser-native`, I
> can
>   then add `goreleaser:native` to the build dependencies in my
> control file
>   (the `:native` makes dpkg-buildpackage choose the right
> architecture).
> - When building the recipe, the only change is that we set
> PACKAGE_ARCH=HOST_ARCH.
>   In addition we have the override `class-native`, which is set when
> building
>   the native variant of the recipe and lets us make conditional
> changes to
>   what is built.
> 
> Some caveats/notes:
> - Both "normal" target arch and `-native` builds generate ARCH=all
> packages
>   and source packages. So there is potential duplication. In theory
> those
>   packages should be identical, but we currently don't have any way
> of knowing.

Yes, that is indeed problematic. But we already have this problem today
when doing manual duplication to get multi-arch support.

>   This is in part due to using shared isar-apt to transfer packages
> between
>   jobs, where the last job to push a package always wins.
>   My earlier RFC on removing (that use of) isar-apt and explicitly
>   transferring dependent packages might help here. In that case we
> have full
>   control of what packages are provided, and can detect such
> duplicates.
> - I'm changing the default overrides to include PACKAGE_ARCH instead
> of
>   DISTRO_ARCH. Maybe this is how it always should have been?

Hmm... I'm not 100% sure if that is correct. The PACKAGE_ARCH is what
is passed to --host= of sbuild. This describes the toolchain that is
used to generate the package and somehow corresponds to DISTRO_ARCH
(for target packages and HOST_ARCH for host tools). The architecture of
the package that is built is specified by DPKG_ARCH. IMHO the naming
scheme here is quite confusing.

> - This is currently only for "native", but I think we can simply
> duplicate
>   the pattern to also support compat arch packages.

We have an internal layer that heavily uses multi-arch and builds
library packages for at least two architectures at the same time.
Currently this is done via a manual duplication of the recipes (using a
shared .inc), but this is error prone and tedious.
Having this in ISAR would significantly reduce the maintenance effort
there.

> - Using PN in recipes can become tricky. For example, I sometimes see
> the
>   pattern of setting SRC_URI="git://github.com/${PN}/${PN}.git".
>   This would break when building the native variant, as this will
> then
>   have a different PN. Personally, I don't think this is a big
> problem.
>   Using ${PN} in SRC_URI may not be a good idea anyways, and if we
> really
>   need a variable, there is BPN, which does not have the -native
> suffix.

We even have that pattern in the hello.bb example. It's pretty
prominent.

Felix

> - We currently don't have a use case for this in meta-isar. But maybe
> we
>   can come up with a good example.
> 
> I'd like to try if this approach works for the other use cases we
> have.
> 
> Adriaan
> 
> PS: If you'd like some more examples of the BBCLASSEXTEND feature,
> have
>     a look at the SDK generation, which has been refactored in that
> way.
> 
> ---
>  meta/classes/native.bbclass | 13 +++++++++++++
>  meta/conf/bitbake.conf      |  5 +++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes/native.bbclass
> 
> diff --git a/meta/classes/native.bbclass
> b/meta/classes/native.bbclass
> new file mode 100644
> index 00000000..896c8a06
> --- /dev/null
> +++ b/meta/classes/native.bbclass
> @@ -0,0 +1,13 @@
> +BBCLASSEXTEND = "native"
> +BPN = "${PN}"
> +
> +python native_virtclass_handler() {
> +    pn = e.data.getVar('PN')
> +    if pn.endswith('-native'):
> +        e.data.setVar('BPN', pn[:-len('-native')])
> +        e.data.appendVar('OVERRIDES', ':class-native')
> +}
> +addhandler native_virtclass_handler
> +native_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
> +
> +PACKAGE_ARCH_class-native = "${HOST_ARCH}"
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 98412e02..ef962fc7 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -67,8 +67,8 @@ KERNEL_FILE_mipsel ?= "vmlinux"
>  KERNEL_FILE_riscv64 ?= "vmlinux"
>  KERNEL_FILE_arm64 ?= "vmlinux"
>  
> -OVERRIDES =
> "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO
> _CODENAME}:forcevariable"
> -FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
> +OVERRIDES =
> "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTR
> O_CODENAME}:forcevariable"
> +FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
>  COMPAT_OVERRIDE = "${@'compat-arch' if
> d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
>  
>  # Setting default QEMU_ARCH variables for different DISTRO_ARCH:
> @@ -81,6 +81,7 @@ QEMU_ARCH_riscv64 = "riscv64"
>  
>  # Codename of the repository created by the caching class
>  DEBDISTRONAME ?= "isar"
> +NATIVELSBSTRING ?= "isarnative"
>  
>  # Strings used in sstate signature files
>  TARGET_VENDOR = ""
> -- 
> 2.30.2
>