Re: [PATCH v2 0/9] HOST_ARCH, -native, riscv64, and all the rest

[Uladzimir Bely <ubely@ilbers.de>]

On Thu, 2023-10-12 at 06:51 +0200, Jan Kiszka wrote:
> On 11.10.23 12:51, 'Jan Kiszka' via isar-users wrote:
> > On 11.10.23 12:49, Uladzimir Bely wrote:
> > > On Fri, 2023-10-06 at 17:34 +0200, Jan Kiszka wrote:
> > > > Changes in v2:
> > > >  - drop merged revert
> > > >  - rename BUILD_HOST_ARCH to BUILD_ARCH
> > > >  - add fix for explicit -native build in non-cross setups
> > > >  - add missing debian-sid.conf
> > > > 
> > > > This tries to resolve the native build issues of the
> > > > visionfive2,
> > > > fixing
> > > > some remaining corner case of -native usage:
> > > > 
> > > > If package A has B-native as build dependency but is then
> > > > compiled
> > > > without cross-building, using qemu-user instead, it will
> > > > incorrectly
> > > > request the build of B:, rather than B:. This
> > > > is resolved automatically now, and the solution is used for the
> > > > visionfive2.
> > > > 
> > > > For that board to build again, we also need to switch from sid-
> > > > ports
> > > > to
> > > > sid, for riscv64 in general.
> > > > 
> > > > Furthermore, this reworks the crossvars logic to also account
> > > > for the
> > > > corner case that a package is still requested as native
> > > > although
> > > > ISAR_CROSS_COMPILE is off. While this is likely irrelvant in
> > > > practice,
> > > > the result is a more consistent logic of the crossvars class.
> > > > 
> > > > Jan
> > > > 
> > > > Jan Kiszka (9):
> > > >   Rename BUILD_HOST_ARCH to BUILD_ARCH
> > > >   dpkg: Drop redundant PACKAGE_ARCH initialization
> > > >   crossvars: Adjust logic to account for -native package builds
> > > > in
> > > >     non-cross setups
> > > >   multiarch: Downgrade native dependencies when building in
> > > > emulated
> > > >     environment
> > > >   meta-isar: jh7110-u-boot-spl-tool: Rely on native annotation
> > > >   meta-isar: jh7110-u-boot-spl-image: Fix native build
> > > >   meta-isar: u-boot-starfive-visionfive2: Simplify build
> > > > dependencies
> > > >   linux-custom: Drop unused template variable
> > > >   Move riscv64 from sid-ports to regular sid
> > > > 
> > > >  doc/user_manual.md                            |  4 ++--
> > > >  kas/distro/Kconfig                            |  6 +++---
> > > >  ...{debian-sid-ports.yaml => debian-sid.yaml} |  2 +-
> > > >  kas/machine/Kconfig                           |  6 +++---
> > > >  meta-isar/conf/mc.conf                        |  6 +++---
> > > >  ...64-sid-ports.conf => qemuriscv64-sid.conf} |  2 +-
> > > >  ...0-sid-ports.conf => sifive-fu540-sid.conf} |  2 +-
> > > >  ...rts.conf => starfive-visionfive2-sid.conf} |  2 +-
> > > >  .../jh7110-u-boot-spl-image_0.1.bb            |  6 +++---
> > > >  .../jh7110-u-boot-spl-tool_0.1.bb             |  3 ---
> > > >  ...-starfive-visionfive2_2021.10+VF2-3.0.4.bb |  4 +---
> > > >  meta/classes/crossvars.bbclass                | 19 ++++++++++-
> > > > ------
> > > >  meta/classes/dpkg-base.bbclass                |  2 +-
> > > >  meta/classes/dpkg.bbclass                     |  4 +---
> > > >  meta/classes/multiarch.bbclass                | 12 +++++++++++
> > > >  meta/conf/bitbake.conf                        |  2 +-
> > > >  meta/conf/distro/debian-sid.conf              | 21
> > > > +++++++++++++++++++
> > > >  meta/recipes-kernel/linux/linux-custom.inc    |  1 -
> > > >  testsuite/citest.py                           |  6 +++---
> > > >  19 files changed, 70 insertions(+), 40 deletions(-)
> > > >  rename kas/distro/{debian-sid-ports.yaml => debian-sid.yaml}
> > > > (78%)
> > > >  rename meta-isar/conf/multiconfig/{qemuriscv64-sid-ports.conf
> > > > =>
> > > > qemuriscv64-sid.conf} (85%)
> > > >  rename meta-isar/conf/multiconfig/{sifive-fu540-sid-ports.conf
> > > > =>
> > > > sifive-fu540-sid.conf} (76%)
> > > >  rename meta-isar/conf/multiconfig/{starfive-visionfive2-sid-
> > > > ports.conf => starfive-visionfive2-sid.conf} (78%)
> > > >  create mode 100644 meta/conf/distro/debian-sid.conf
> > > > 
> > > 
> > > While testing the patchset, I noticed that packages with "-
> > > compat"
> > > suffix now pull and use "sbuild-chroot-target" instead of
> > > "sbuild-
> > > chroot-host".
> > > 
> > > Conditions:
> > > ISAR_CROSS_COMPILE = "1"
> > > ISAR_ENABLE_COMPAT_ARCH = "1"
> > > 
> > > Build command: `bitbake -v mc:qemuarm64-bookworm:hello-compat`
> > > 
> > > On 'next':
> > > 
> > > ```
> > > build$ ls -1 tmp/work/debian-bookworm-arm64/
> > > base-apt
> > > hello-compat
> > > isar-apt
> > > isar-bootstrap-host
> > > sbuild-chroot-host
> > > ```
> > > 
> > > With patchset:
> > > 
> > > ```
> > > build$ ls -1 tmp/work/debian-bookworm-arm64/
> > > base-apt
> > > hello-compat
> > > isar-apt
> > > isar-bootstrap-target
> > > sbuild-chroot-target
> > > ```
> > > 
> > > If talking about CI, it passes except repro tests.
> > > 
> > > Due to this issue, some 'deb' files from raspios repositories go
> > > to
> > > "debian" download subdirectory, and during the second (base-apt
> > > cached)
> > > build they go to Debian's `base-apt` and it leads to the conflict
> > > when
> > > resolving dependencies.
> > > 
> > > I'm stil debugging this, but this part of information may be
> > > useful.
> > 
> > Likely an issue of patch 3. The rest should still work when pulling
> > this
> > out, though.
> > 
> 
> Yep, patch 3, and this should fix it:
> 
> diff --git a/meta/classes/crossvars.bbclass
> b/meta/classes/crossvars.bbclass
> index 87e45417..a7834b99 100644
> --- a/meta/classes/crossvars.bbclass
> +++ b/meta/classes/crossvars.bbclass
> @@ -16,12 +16,13 @@ python __anonymous() {
>      flavor_suffix = ('-' + flavor) if flavor else ''
> 
>      distro_arch = d.getVar('DISTRO_ARCH')
> +    compat_arch = d.getVar('COMPAT_DISTRO_ARCH')
>      host_arch = d.getVar('HOST_ARCH')
>      package_arch = d.getVar('PACKAGE_ARCH')
> 
>      if distro_arch != host_arch and \
>          (package_arch == host_arch or \
> -         (package_arch == distro_arch and mode == "1")):
> +         (package_arch in [distro_arch, compat_arch] and mode ==
> "1")):
>          d.setVar('BUILD_ARCH', host_arch)
>          schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>          sbuild_dep = "sbuild-chroot-host" + flavor_suffix +
> ":do_build"
> 
> Should I resent the whole series with this update?
> 

Yes, please.

In my internal experiments I came up to something similar:

diff --git a/meta/classes/crossvars.bbclass
b/meta/classes/crossvars.bbclass
index 87e45417..501dfef2 100644
--- a/meta/classes/crossvars.bbclass
+++ b/meta/classes/crossvars.bbclass
@@ -18,9 +18,11 @@ python __anonymous() {
     distro_arch = d.getVar('DISTRO_ARCH')
     host_arch = d.getVar('HOST_ARCH')
     package_arch = d.getVar('PACKAGE_ARCH')
+    compat_distro_arch = d.getVar('COMPAT_DISTRO_ARCH')
 
     if distro_arch != host_arch and \
         (package_arch == host_arch or \
+         (package_arch == compat_distro_arch and mode == "1") or \
          (package_arch == distro_arch and mode == "1")):
         d.setVar('BUILD_ARCH', host_arch)
         schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)

.. and now testing if it fixes the issue.

> Jan
>