[PATCH v2] base: Fix HOST_ARCH for native builds

[PATCH v2] base: Fix HOST_ARCH for native builds

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

HOST_ARCH must be DISTRO_ARCH when we are not cross-building. Otherwise,
recipes that set PACKAGE_ARCH to it will fail in native builds.

Use late expansion for ISAR_CROSS_COMPILE to avoid recursion issues.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

That "expand=False" seems to work fine, so let's move forward with it.

 meta/classes/base.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 88004120..8ca089a2 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -49,13 +49,15 @@ def oe_import(d):
 # We need the oe module name space early (before INHERITs get added)
 OE_IMPORTED := "${@oe_import(d)}"
 
-def get_deb_host_arch():
+def get_deb_host_arch(d):
     import subprocess
+    if d.getVar("ISAR_CROSS_COMPILE", expand=False) != "1":
+        return d.getVar("DISTRO_ARCH")
     host_arch = subprocess.check_output(
         ["dpkg", "--print-architecture"]
     ).decode('utf-8').strip()
     return host_arch
-HOST_ARCH ??= "${@get_deb_host_arch()}"
+HOST_ARCH ??= "${@get_deb_host_arch(d)}"
 HOST_DISTRO ??= "${DISTRO}"
 
 die() {
-- 
2.35.3

Re: [PATCH v2] base: Fix HOST_ARCH for native builds

[Uladzimir Bely]

On Mon, 2023-09-25 at 13:41 +0200, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> HOST_ARCH must be DISTRO_ARCH when we are not cross-building.
> Otherwise,
> recipes that set PACKAGE_ARCH to it will fail in native builds.
> 
> Use late expansion for ISAR_CROSS_COMPILE to avoid recursion issues.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> That "expand=False" seems to work fine, so let's move forward with
> it.
> 
>  meta/classes/base.bbclass | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 88004120..8ca089a2 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -49,13 +49,15 @@ def oe_import(d):
>  # We need the oe module name space early (before INHERITs get added)
>  OE_IMPORTED := "${@oe_import(d)}"
>  
> -def get_deb_host_arch():
> +def get_deb_host_arch(d):
>      import subprocess
> +    if d.getVar("ISAR_CROSS_COMPILE", expand=False) != "1":
> +        return d.getVar("DISTRO_ARCH")
>      host_arch = subprocess.check_output(
>          ["dpkg", "--print-architecture"]
>      ).decode('utf-8').strip()
>      return host_arch
> -HOST_ARCH ??= "${@get_deb_host_arch()}"
> +HOST_ARCH ??= "${@get_deb_host_arch(d)}"
>  HOST_DISTRO ??= "${DISTRO}"
>  
>  die() {

For some reason this doesn't properly work for some targets in case of
cross-compile mode.

Without patch (default local.conf with ISAR_CROSS_COMPILE set):
```
bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
HOST_ARCH="amd64"
bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
"^HOST_ARCH="
HOST_ARCH="amd64"
bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep "^HOST_ARCH="
HOST_ARCH="amd64"
bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep "^HOST_ARCH="
HOST_ARCH="amd64"
```

With patch:
```
bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
HOST_ARCH="amd64"
bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
"^HOST_ARCH="
HOST_ARCH="amd64"
bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep "^HOST_ARCH="
HOST_ARCH="armhf"
bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep "^HOST_ARCH="
HOST_ARCH="armhf"
```

For some reasons this affects only "armhf" targets...

As a result, in case of rpi-arm-v7 ${DL_DIR}/deb/debian-bullseye/ has
wrongly imported non-debian packages and second (cached from base-apt)
build does not work since it can't resolve the dependencies.

Re: [PATCH v2] base: Fix HOST_ARCH for native builds

[Uladzimir Bely]

On Fri, 2023-09-29 at 19:36 +0300, Uladzimir Bely wrote:
> On Mon, 2023-09-25 at 13:41 +0200, Jan Kiszka wrote:
> > From: Jan Kiszka <jan.kiszka@siemens.com>
> > 
> > HOST_ARCH must be DISTRO_ARCH when we are not cross-building.
> > Otherwise,
> > recipes that set PACKAGE_ARCH to it will fail in native builds.
> > 
> > Use late expansion for ISAR_CROSS_COMPILE to avoid recursion
> > issues.
> > 
> > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> > ---
> > 
> > That "expand=False" seems to work fine, so let's move forward with
> > it.
> > 
> >  meta/classes/base.bbclass | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> > index 88004120..8ca089a2 100644
> > --- a/meta/classes/base.bbclass
> > +++ b/meta/classes/base.bbclass
> > @@ -49,13 +49,15 @@ def oe_import(d):
> >  # We need the oe module name space early (before INHERITs get
> > added)
> >  OE_IMPORTED := "${@oe_import(d)}"
> >  
> > -def get_deb_host_arch():
> > +def get_deb_host_arch(d):
> >      import subprocess
> > +    if d.getVar("ISAR_CROSS_COMPILE", expand=False) != "1":
> > +        return d.getVar("DISTRO_ARCH")
> >      host_arch = subprocess.check_output(
> >          ["dpkg", "--print-architecture"]
> >      ).decode('utf-8').strip()
> >      return host_arch
> > -HOST_ARCH ??= "${@get_deb_host_arch()}"
> > +HOST_ARCH ??= "${@get_deb_host_arch(d)}"
> >  HOST_DISTRO ??= "${DISTRO}"
> >  
> >  die() {
> 
> For some reason this doesn't properly work for some targets in case
> of
> cross-compile mode.
> 
> Without patch (default local.conf with ISAR_CROSS_COMPILE set):
> ```
> bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
> HOST_ARCH="amd64"
> bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
> "^HOST_ARCH="
> HOST_ARCH="amd64"
> bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep
> "^HOST_ARCH="
> HOST_ARCH="amd64"
> bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep
> "^HOST_ARCH="
> HOST_ARCH="amd64"
> ```
> 
> With patch:
> ```
> bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
> HOST_ARCH="amd64"
> bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
> "^HOST_ARCH="
> HOST_ARCH="amd64"
> bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep
> "^HOST_ARCH="
> HOST_ARCH="armhf"
> bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep
> "^HOST_ARCH="
> HOST_ARCH="armhf"
> ```
> 
> For some reasons this affects only "armhf" targets...
> 

... And it looks to be caused by line 69 in "imagetypes.bbclass":
ISAR_CROSS_COMPILE:armhf = "${@bb.utils.contains('IMAGE_BASETYPES',
'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"

With (,"expand=False") in case of armhf the wrong DISTRO_ARCH value
assigned to HOST_ARCH.

> As a result, in case of rpi-arm-v7 ${DL_DIR}/deb/debian-bullseye/ has
> wrongly imported non-debian packages and second (cached from base-
> apt)
> build does not work since it can't resolve the dependencies.
> 

Re: [PATCH v2] base: Fix HOST_ARCH for native builds

[Uladzimir Bely]

On Fri, 2023-09-29 at 20:25 +0300, Uladzimir Bely wrote:
> On Fri, 2023-09-29 at 19:36 +0300, Uladzimir Bely wrote:
> > On Mon, 2023-09-25 at 13:41 +0200, Jan Kiszka wrote:
> > > From: Jan Kiszka <jan.kiszka@siemens.com>
> > > 
> > > HOST_ARCH must be DISTRO_ARCH when we are not cross-building.
> > > Otherwise,
> > > recipes that set PACKAGE_ARCH to it will fail in native builds.
> > > 
> > > Use late expansion for ISAR_CROSS_COMPILE to avoid recursion
> > > issues.
> > > 
> > > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> > > ---
> > > 
> > > That "expand=False" seems to work fine, so let's move forward
> > > with
> > > it.
> > > 
> > >  meta/classes/base.bbclass | 6 ++++--
> > >  1 file changed, 4 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/meta/classes/base.bbclass
> > > b/meta/classes/base.bbclass
> > > index 88004120..8ca089a2 100644
> > > --- a/meta/classes/base.bbclass
> > > +++ b/meta/classes/base.bbclass
> > > @@ -49,13 +49,15 @@ def oe_import(d):
> > >  # We need the oe module name space early (before INHERITs get
> > > added)
> > >  OE_IMPORTED := "${@oe_import(d)}"
> > >  
> > > -def get_deb_host_arch():
> > > +def get_deb_host_arch(d):
> > >      import subprocess
> > > +    if d.getVar("ISAR_CROSS_COMPILE", expand=False) != "1":
> > > +        return d.getVar("DISTRO_ARCH")
> > >      host_arch = subprocess.check_output(
> > >          ["dpkg", "--print-architecture"]
> > >      ).decode('utf-8').strip()
> > >      return host_arch
> > > -HOST_ARCH ??= "${@get_deb_host_arch()}"
> > > +HOST_ARCH ??= "${@get_deb_host_arch(d)}"
> > >  HOST_DISTRO ??= "${DISTRO}"
> > >  
> > >  die() {
> > 
> > For some reason this doesn't properly work for some targets in case
> > of
> > cross-compile mode.
> > 
> > Without patch (default local.conf with ISAR_CROSS_COMPILE set):
> > ```
> > bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
> > HOST_ARCH="amd64"
> > bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
> > "^HOST_ARCH="
> > HOST_ARCH="amd64"
> > bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep
> > "^HOST_ARCH="
> > HOST_ARCH="amd64"
> > bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep
> > "^HOST_ARCH="
> > HOST_ARCH="amd64"
> > ```
> > 
> > With patch:
> > ```
> > bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
> > HOST_ARCH="amd64"
> > bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
> > "^HOST_ARCH="
> > HOST_ARCH="amd64"
> > bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep
> > "^HOST_ARCH="
> > HOST_ARCH="armhf"
> > bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep
> > "^HOST_ARCH="
> > HOST_ARCH="armhf"
> > ```
> > 
> > For some reasons this affects only "armhf" targets...
> > 
> 
> ... And it looks to be caused by line 69 in "imagetypes.bbclass":
> ISAR_CROSS_COMPILE:armhf = "${@bb.utils.contains('IMAGE_BASETYPES',
> 'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"
> 
... And it seems extactly this line caused the issue with parsing
"armhf" multiconfigs in patch v1.

> With (,"expand=False") in case of armhf the wrong DISTRO_ARCH value
> assigned to HOST_ARCH.
> 
> > As a result, in case of rpi-arm-v7 ${DL_DIR}/deb/debian-bullseye/
> > has
> > wrongly imported non-debian packages and second (cached from base-
> > apt)
> > build does not work since it can't resolve the dependencies.
> > 
> 

Re: [PATCH v2] base: Fix HOST_ARCH for native builds

[Jan Kiszka]

On 29.09.23 19:33, Uladzimir Bely wrote:
> On Fri, 2023-09-29 at 20:25 +0300, Uladzimir Bely wrote:
>> On Fri, 2023-09-29 at 19:36 +0300, Uladzimir Bely wrote:
>>> On Mon, 2023-09-25 at 13:41 +0200, Jan Kiszka wrote:
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>
>>>> HOST_ARCH must be DISTRO_ARCH when we are not cross-building.
>>>> Otherwise,
>>>> recipes that set PACKAGE_ARCH to it will fail in native builds.
>>>>
>>>> Use late expansion for ISAR_CROSS_COMPILE to avoid recursion
>>>> issues.
>>>>
>>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>>> ---
>>>>
>>>> That "expand=False" seems to work fine, so let's move forward
>>>> with
>>>> it.
>>>>
>>>>  meta/classes/base.bbclass | 6 ++++--
>>>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/meta/classes/base.bbclass
>>>> b/meta/classes/base.bbclass
>>>> index 88004120..8ca089a2 100644
>>>> --- a/meta/classes/base.bbclass
>>>> +++ b/meta/classes/base.bbclass
>>>> @@ -49,13 +49,15 @@ def oe_import(d):
>>>>  # We need the oe module name space early (before INHERITs get
>>>> added)
>>>>  OE_IMPORTED := "${@oe_import(d)}"
>>>>  
>>>> -def get_deb_host_arch():
>>>> +def get_deb_host_arch(d):
>>>>      import subprocess
>>>> +    if d.getVar("ISAR_CROSS_COMPILE", expand=False) != "1":
>>>> +        return d.getVar("DISTRO_ARCH")
>>>>      host_arch = subprocess.check_output(
>>>>          ["dpkg", "--print-architecture"]
>>>>      ).decode('utf-8').strip()
>>>>      return host_arch
>>>> -HOST_ARCH ??= "${@get_deb_host_arch()}"
>>>> +HOST_ARCH ??= "${@get_deb_host_arch(d)}"
>>>>  HOST_DISTRO ??= "${DISTRO}"
>>>>  
>>>>  die() {
>>>
>>> For some reason this doesn't properly work for some targets in case
>>> of
>>> cross-compile mode.
>>>
>>> Without patch (default local.conf with ISAR_CROSS_COMPILE set):
>>> ```
>>> bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
>>> HOST_ARCH="amd64"
>>> bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
>>> "^HOST_ARCH="
>>> HOST_ARCH="amd64"
>>> bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep
>>> "^HOST_ARCH="
>>> HOST_ARCH="amd64"
>>> bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep
>>> "^HOST_ARCH="
>>> HOST_ARCH="amd64"
>>> ```
>>>
>>> With patch:
>>> ```
>>> bitbake mc:qemuarm64-focal:isar-image-base -e | grep "^HOST_ARCH="
>>> HOST_ARCH="amd64"
>>> bitbake mc:rpi-arm64-v8-bullseye:isar-image-base -e | grep
>>> "^HOST_ARCH="
>>> HOST_ARCH="amd64"
>>> bitbake mc:rpi-arm-v7-bullseye:isar-image-base -e | grep
>>> "^HOST_ARCH="
>>> HOST_ARCH="armhf"
>>> bitbake mc:stm32mp15x-bullseye:isar-image-base -e | grep
>>> "^HOST_ARCH="
>>> HOST_ARCH="armhf"
>>> ```
>>>
>>> For some reasons this affects only "armhf" targets...
>>>
>>
>> ... And it looks to be caused by line 69 in "imagetypes.bbclass":
>> ISAR_CROSS_COMPILE:armhf = "${@bb.utils.contains('IMAGE_BASETYPES',
>> 'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"
>>
> ... And it seems extactly this line caused the issue with parsing
> "armhf" multiconfigs in patch v1.
> 

OK, good to know this. I have no idea yet how to fix all this.

I just tried to avoid the issue by switching the affected recipes to
-native. Would be cleaner by now anyway. However, that mechanism also
uses the broken HOST_ARCH.

crossvars has BUILD_HOST_ARCH which actually seems to have the correct
values. But why do we have to have two of such vars?

Jan

-- 
Siemens AG, Technology
Linux Expert Center