Re: [RFC PATCH] native.bbclass

[Jan Kiszka <jan.kiszka@siemens.com>]

On 17.01.23 19:03, Adriaan Schmidt wrote:
> Hi all,
> 
> as mentioned by Jan, I have another approach of building native/host/compat
> arch packages, inspired by OE's native class.
> 
> The way it works is:
> - A recipe can inherit native.bbclass (or, if we like we could even add it
>   to all recipes via the dpkg class).

I think we should not start spreading these inherits across random
recipes when there are no costs and risks (as I believe) in adding that
to dpkg directly, likely even dpkg-base.

> - For recipes that inherit native.bbclass, bitbake generates a new bitbake
>   target with suffix `-native`. Recipes can DEPEND on those `-native` packages.
>   My use case that made me develop this is goreleaser, a build tool
>   for which I have a bitbake recipe. If I want to crosscompile, I need it
>   for the host architecture. By DEPENDing on `goreleaser-native`, I can
>   then add `goreleaser:native` to the build dependencies in my control file
>   (the `:native` makes dpkg-buildpackage choose the right architecture).
> - When building the recipe, the only change is that we set PACKAGE_ARCH=HOST_ARCH.
>   In addition we have the override `class-native`, which is set when building
>   the native variant of the recipe and lets us make conditional changes to
>   what is built.

So, if I decide to pre-populate an image with libfoo not only for the
target arch but maybe also others (thinking of compat here), I would
have to do

IMAGE_INSTALL += "meta-package"

and meta-package would have

DEBIAN_DEPENDS += "libfoo, libfoo:${COMPAT_DISTRO_ARCH}"
DEPENDS += "libfoo libfoo-compat"

Unless we find a way to teach IMAGE_INSTALL resolving "...:<some-arch>"
into the right -native/-compat DEPENDS.

Obviously simpler is to have some 32-bit-only legacy app that states

PACKAGE_ARCH = "${COMPAT_DISTRO_ARCH}"
DEBIAN_DEPENDS = "libfoo:${COMPAT_DISTRO_ARCH}"
DEPENDS = "libfoo-compat"

libfoo itself would no longer have to worry about becoming compat as well.

> 
> Some caveats/notes:
> - Both "normal" target arch and `-native` builds generate ARCH=all packages
>   and source packages. So there is potential duplication. In theory those
>   packages should be identical, but we currently don't have any way of knowing.
>   This is in part due to using shared isar-apt to transfer packages between
>   jobs, where the last job to push a package always wins.
>   My earlier RFC on removing (that use of) isar-apt and explicitly
>   transferring dependent packages might help here. In that case we have full
>   control of what packages are provided, and can detect such duplicates.

We have a second duplication issue, and that is around the debian source
package. Again, all targets should normally generate a bit-identical
version of that so that only identical versions will be found when
collecting them for potential redistribution. I was assuming Isar would
already build a deb-src repo for isar-apt, but it this feature seems to
be missing. Once we add it, the topic above will become relevant for it
as well.

> - I'm changing the default overrides to include PACKAGE_ARCH instead of
>   DISTRO_ARCH. Maybe this is how it always should have been?
> - This is currently only for "native", but I think we can simply duplicate
>   the pattern to also support compat arch packages.

This should be done in the same run, we already need it as urgently as
-native.

> - Using PN in recipes can become tricky. For example, I sometimes see the
>   pattern of setting SRC_URI="git://github.com/${PN}/${PN}.git".
>   This would break when building the native variant, as this will then
>   have a different PN. Personally, I don't think this is a big problem.
>   Using ${PN} in SRC_URI may not be a good idea anyways, and if we really
>   need a variable, there is BPN, which does not have the -native suffix.

I agree.

> - We currently don't have a use case for this in meta-isar. But maybe we
>   can come up with a good example.

Typical use cases are self-built build tools for other self-built
packages. Or cleaning up the kbuild issue that also Stefan Koch was
working on. In fact, kbuild is an existing in-tree -native use case that
we hacked so far to generate only -native packages. When converting
that, we could add a test case that uses the to-be-generated
kbuild:target package on the generated image, e.g. to install a dkms
kernel module live.

Jan

> 
> I'd like to try if this approach works for the other use cases we have.
> 
> Adriaan
> 
> PS: If you'd like some more examples of the BBCLASSEXTEND feature, have
>     a look at the SDK generation, which has been refactored in that way.
> 
> ---
>  meta/classes/native.bbclass | 13 +++++++++++++
>  meta/conf/bitbake.conf      |  5 +++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes/native.bbclass
> 
> diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
> new file mode 100644
> index 00000000..896c8a06
> --- /dev/null
> +++ b/meta/classes/native.bbclass
> @@ -0,0 +1,13 @@
> +BBCLASSEXTEND = "native"
> +BPN = "${PN}"
> +
> +python native_virtclass_handler() {
> +    pn = e.data.getVar('PN')
> +    if pn.endswith('-native'):
> +        e.data.setVar('BPN', pn[:-len('-native')])
> +        e.data.appendVar('OVERRIDES', ':class-native')
> +}
> +addhandler native_virtclass_handler
> +native_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
> +
> +PACKAGE_ARCH_class-native = "${HOST_ARCH}"
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 98412e02..ef962fc7 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -67,8 +67,8 @@ KERNEL_FILE_mipsel ?= "vmlinux"
>  KERNEL_FILE_riscv64 ?= "vmlinux"
>  KERNEL_FILE_arm64 ?= "vmlinux"
>  
> -OVERRIDES = "${DISTRO_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
> -FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
> +OVERRIDES = "${PACKAGE_ARCH}:${COMPAT_OVERRIDE}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
> +FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
>  COMPAT_OVERRIDE = "${@'compat-arch' if d.getVar('ISAR_ENABLE_COMPAT_ARCH') == '1' else ''}"
>  
>  # Setting default QEMU_ARCH variables for different DISTRO_ARCH:
> @@ -81,6 +81,7 @@ QEMU_ARCH_riscv64 = "riscv64"
>  
>  # Codename of the repository created by the caching class
>  DEBDISTRONAME ?= "isar"
> +NATIVELSBSTRING ?= "isarnative"
>  
>  # Strings used in sstate signature files
>  TARGET_VENDOR = ""

-- 
Siemens AG, Technology
Competence Center Embedded Linux