-native working as designed? target vs host sbuild-chroot

-native working as designed? target vs host sbuild-chroot

[cedric.hombourger]

Hello,

I was wondering if handling of -native was working as designed (or if I 
am missing something).

Let's consider the following steps:

     git clone -b next https://github.com/ilbers/isar
     . isar-init-build-env
     mc:qemuarm64-bookworm:hello-native

Observe that the build constructs sbuild-chroot-target when I would have 
expected sbuild-chroot-host. The sbuild environment for hello is indeed 
configured to build an amd64 package while running on an arm64 userland 
=> the build of that package alone takes several minutes (on an Intel 
Xeon 6242 and with /proc/cpuinfo listing 64 processors)

In addition, when using Isar from master (where support for -native and 
-compat is already there), the build fails to install build dependencies 
for hello-native:

|  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but 
it is not installable
|                                          Depends: libstdc++-dev:amd64 
but it is not installable

This issue may also be reproduced on next with 
7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native 
builds) removed. I guess I just making a note that -native may have 
issues on master.

My outstanding question would be: shouldn't -native use the *host* 
sbuild-chroot? If we agree that it should then I am happy to look into this.

Thanks
Cedric

Refs:
     next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in 
KAS_IMAGE_PREINSTALL list
     master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix 
internal deb import for sbuild

RE: -native working as designed? target vs host sbuild-chroot

[Schmidt, Adriaan]

cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
> Hello,
> 
> I was wondering if handling of -native was working as designed (or if I
> am missing something).
> 
> Let's consider the following steps:
> 
>      git clone -b next https://github.com/ilbers/isar
>      . isar-init-build-env
>      mc:qemuarm64-bookworm:hello-native
> 
> Observe that the build constructs sbuild-chroot-target when I would have
> expected sbuild-chroot-host. The sbuild environment for hello is indeed
> configured to build an amd64 package while running on an arm64 userland
> => the build of that package alone takes several minutes (on an Intel
> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
> 
> In addition, when using Isar from master (where support for -native and
> -compat is already there), the build fails to install build dependencies
> for hello-native:
> 
> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
> it is not installable
> |                                          Depends: libstdc++-dev:amd64
> but it is not installable
> 
> This issue may also be reproduced on next with
> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
> builds) removed. I guess I just making a note that -native may have
> issues on master.
> 
> My outstanding question would be: shouldn't -native use the *host*
> sbuild-chroot? If we agree that it should then I am happy to look into this.

Hi Cedric,

Yes, you are right. The problem seems to be that when building -native,
native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
(via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.

Can you try what happens with:

diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
index df26f47f..508ba72d 100644
--- a/meta/classes/crossvars.bbclass
+++ b/meta/classes/crossvars.bbclass
@@ -15,7 +15,7 @@ python __anonymous() {
     flavor = d.getVar('SBUILD_FLAVOR')
     flavor_suffix = ('-' + flavor) if flavor else ''
 
-    distro_arch = d.getVar('DISTRO_ARCH')
+    distro_arch = d.getVar('PACKAGE_ARCH')
     if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
         d.setVar('BUILD_HOST_ARCH', distro_arch)
         schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)

Thanks,
Adriaan

> Thanks
> Cedric
> 
> Refs:
>      next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in
> KAS_IMAGE_PREINSTALL list
>      master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix
> internal deb import for sbuild
> 
> 
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1-
> 0eccd72ef1a0%40siemens.com.

Re: -native working as designed? target vs host sbuild-chroot

[cedric.hombourger]

On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>> Hello,
>>
>> I was wondering if handling of -native was working as designed (or if I
>> am missing something).
>>
>> Let's consider the following steps:
>>
>>       git clone -b next https://github.com/ilbers/isar
>>       . isar-init-build-env
>>       mc:qemuarm64-bookworm:hello-native
>>
>> Observe that the build constructs sbuild-chroot-target when I would have
>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>> configured to build an amd64 package while running on an arm64 userland
>> => the build of that package alone takes several minutes (on an Intel
>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>
>> In addition, when using Isar from master (where support for -native and
>> -compat is already there), the build fails to install build dependencies
>> for hello-native:
>>
>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>> it is not installable
>> |                                          Depends: libstdc++-dev:amd64
>> but it is not installable
>>
>> This issue may also be reproduced on next with
>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>> builds) removed. I guess I just making a note that -native may have
>> issues on master.
>>
>> My outstanding question would be: shouldn't -native use the *host*
>> sbuild-chroot? If we agree that it should then I am happy to look into this.
> Hi Cedric,
>
> Yes, you are right. The problem seems to be that when building -native,
> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>
> Can you try what happens with:
>
> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
> index df26f47f..508ba72d 100644
> --- a/meta/classes/crossvars.bbclass
> +++ b/meta/classes/crossvars.bbclass
> @@ -15,7 +15,7 @@ python __anonymous() {
>       flavor = d.getVar('SBUILD_FLAVOR')
>       flavor_suffix = ('-' + flavor) if flavor else ''
>
> -    distro_arch = d.getVar('DISTRO_ARCH')
> +    distro_arch = d.getVar('PACKAGE_ARCH')

sure - will give this is a try but I fear it won't work because the
condition mode == "0" remains True
You will find further below the changes that I am currently testing (the
hello-native use-case is now working as expected, currently running CI
tests)

>       if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>           d.setVar('BUILD_HOST_ARCH', distro_arch)
>           schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)

My RFC changes below

diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
index df26f47..1378333 100644
--- a/meta/classes/crossvars.bbclass
+++ b/meta/classes/crossvars.bbclass
@@ -15,12 +15,24 @@ python __anonymous() {
      flavor = d.getVar('SBUILD_FLAVOR')
      flavor_suffix = ('-' + flavor) if flavor else ''

+    # get arch triplet
+    host_arch = d.getVar('HOST_ARCH')
      distro_arch = d.getVar('DISTRO_ARCH')
-    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
distro_arch == None:
+    package_arch = d.getVar('PACKAGE_ARCH')
+
+    # -native build within a foreign-arch build? use host
+    if package_arch == host_arch and package_arch != distro_arch:
+        d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
+        schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
+        sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
+        sdk_toolchain = "build-essential"
+    # not cross-compiling? use target
+    elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
distro_arch == None:
          d.setVar('BUILD_HOST_ARCH', distro_arch)
          schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
          sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
          sdk_toolchain = "build-essential"
+    # cross-compiling for the foreign-arch: use host with cross toolchain
      else:
          d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
          schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)


> Thanks,
> Adriaan
>
>> Thanks
>> Cedric
>>
>> Refs:
>>       next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in
>> KAS_IMAGE_PREINSTALL list
>>       master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix
>> internal deb import for sbuild
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "isar-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to isar-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1-
>> 0eccd72ef1a0%40siemens.com.

Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka]

On 06.10.23 04:49, 'cedric.hombourger@siemens.com' via isar-users wrote:
> Hello,
> 
> I was wondering if handling of -native was working as designed (or if I 
> am missing something).
> 
> Let's consider the following steps:
> 
>      git clone -b next https://github.com/ilbers/isar
>      . isar-init-build-env

This gives your ISAR_CROSS_COMPILE = 0.

>      mc:qemuarm64-bookworm:hello-native

This is similar to the issue I'm trying to fix with my patch series sent
yesterday evening: 'hello-native' in a non-cross environment should
actually be 'hello' because non-cross means that there is no separate
host build env. My patches will resolve that contradiction, but only for
dependencies you have in your recipes, not for explicitly requested
builds. I'm not sure if we can and should do anything about such
explicit requests.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka]

On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>> Hello,
>>>
>>> I was wondering if handling of -native was working as designed (or if I
>>> am missing something).
>>>
>>> Let's consider the following steps:
>>>
>>>       git clone -b next https://github.com/ilbers/isar
>>>       . isar-init-build-env
>>>       mc:qemuarm64-bookworm:hello-native
>>>
>>> Observe that the build constructs sbuild-chroot-target when I would have
>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>> configured to build an amd64 package while running on an arm64 userland
>>> => the build of that package alone takes several minutes (on an Intel
>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>
>>> In addition, when using Isar from master (where support for -native and
>>> -compat is already there), the build fails to install build dependencies
>>> for hello-native:
>>>
>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>> it is not installable
>>> |                                          Depends: libstdc++-dev:amd64
>>> but it is not installable
>>>
>>> This issue may also be reproduced on next with
>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>> builds) removed. I guess I just making a note that -native may have
>>> issues on master.
>>>
>>> My outstanding question would be: shouldn't -native use the *host*
>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>> Hi Cedric,
>>
>> Yes, you are right. The problem seems to be that when building -native,
>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>
>> Can you try what happens with:
>>
>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>> index df26f47f..508ba72d 100644
>> --- a/meta/classes/crossvars.bbclass
>> +++ b/meta/classes/crossvars.bbclass
>> @@ -15,7 +15,7 @@ python __anonymous() {
>>       flavor = d.getVar('SBUILD_FLAVOR')
>>       flavor_suffix = ('-' + flavor) if flavor else ''
>>
>> -    distro_arch = d.getVar('DISTRO_ARCH')
>> +    distro_arch = d.getVar('PACKAGE_ARCH')
> 
> sure - will give this is a try but I fear it won't work because the
> condition mode == "0" remains True
> You will find further below the changes that I am currently testing (the
> hello-native use-case is now working as expected, currently running CI
> tests)
> 
>>       if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>           d.setVar('BUILD_HOST_ARCH', distro_arch)
>>           schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
> 
> My RFC changes below
> 
> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
> index df26f47..1378333 100644
> --- a/meta/classes/crossvars.bbclass
> +++ b/meta/classes/crossvars.bbclass
> @@ -15,12 +15,24 @@ python __anonymous() {
>       flavor = d.getVar('SBUILD_FLAVOR')
>       flavor_suffix = ('-' + flavor) if flavor else ''
> 
> +    # get arch triplet
> +    host_arch = d.getVar('HOST_ARCH')
>       distro_arch = d.getVar('DISTRO_ARCH')
> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
> distro_arch == None:
> +    package_arch = d.getVar('PACKAGE_ARCH')
> +
> +    # -native build within a foreign-arch build? use host
> +    if package_arch == host_arch and package_arch != distro_arch:

package_arch == host_arch looks redundant here.

> +        d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
> +        schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
> +        sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
> +        sdk_toolchain = "build-essential"

These blocks have more and more redundancies, specifically as we only
have two possible outcomes: cross or native build.

> +    # not cross-compiling? use target
> +    elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
> distro_arch == None:

You have extracted host_arch above already.

If the special case you found can be simply addressed here, we can do
that. I doubt, though, that it has a lot practical relevance. It's more
for consistency.

Jan

>           d.setVar('BUILD_HOST_ARCH', distro_arch)
>           schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>           sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>           sdk_toolchain = "build-essential"
> +    # cross-compiling for the foreign-arch: use host with cross toolchain
>       else:
>           d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>           schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
> 
> 
>> Thanks,
>> Adriaan
>>
>>> Thanks
>>> Cedric
>>>
>>> Refs:
>>>       next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in
>>> KAS_IMAGE_PREINSTALL list
>>>       master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix
>>> internal deb import for sbuild
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "isar-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to isar-users+unsubscribe@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1-
>>> 0eccd72ef1a0%40siemens.com.
> 
> 

-- 
Siemens AG, Technology
Linux Expert Center

Re: -native working as designed? target vs host sbuild-chroot

[cedric.hombourger]

On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>> Hello,
>>>>
>>>> I was wondering if handling of -native was working as designed (or if I
>>>> am missing something).
>>>>
>>>> Let's consider the following steps:
>>>>
>>>>        git clone -b next https://github.com/ilbers/isar
>>>>        . isar-init-build-env
>>>>        mc:qemuarm64-bookworm:hello-native
>>>>
>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>> configured to build an amd64 package while running on an arm64 userland
>>>> => the build of that package alone takes several minutes (on an Intel
>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>
>>>> In addition, when using Isar from master (where support for -native and
>>>> -compat is already there), the build fails to install build dependencies
>>>> for hello-native:
>>>>
>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>> it is not installable
>>>> |                                          Depends: libstdc++-dev:amd64
>>>> but it is not installable
>>>>
>>>> This issue may also be reproduced on next with
>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>> builds) removed. I guess I just making a note that -native may have
>>>> issues on master.
>>>>
>>>> My outstanding question would be: shouldn't -native use the *host*
>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>> Hi Cedric,
>>>
>>> Yes, you are right. The problem seems to be that when building -native,
>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>
>>> Can you try what happens with:
>>>
>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>> index df26f47f..508ba72d 100644
>>> --- a/meta/classes/crossvars.bbclass
>>> +++ b/meta/classes/crossvars.bbclass
>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>
>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>> sure - will give this is a try but I fear it won't work because the
>> condition mode == "0" remains True
>> You will find further below the changes that I am currently testing (the
>> hello-native use-case is now working as expected, currently running CI
>> tests)
>>
>>>        if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>> My RFC changes below
>>
>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>> index df26f47..1378333 100644
>> --- a/meta/classes/crossvars.bbclass
>> +++ b/meta/classes/crossvars.bbclass
>> @@ -15,12 +15,24 @@ python __anonymous() {
>>        flavor = d.getVar('SBUILD_FLAVOR')
>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>
>> +    # get arch triplet
>> +    host_arch = d.getVar('HOST_ARCH')
>>        distro_arch = d.getVar('DISTRO_ARCH')
>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>> distro_arch == None:
>> +    package_arch = d.getVar('PACKAGE_ARCH')
>> +
>> +    # -native build within a foreign-arch build? use host
>> +    if package_arch == host_arch and package_arch != distro_arch:
> package_arch == host_arch looks redundant here.

wouldn't dropping it make the second case from the below table pick the
host when we really want the target?

+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuarm64-bookworm | hello-native | amd64        | arm64       |
amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuarm64-bookworm | hello        | arm64        | amd64       |
amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuamd64-bookworm | hello-native | amd64        | amd64       |
amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuamd64-bookworm | hello        | amd64        | amd64       |
amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+

>
>> +        d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>> +        schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>> +        sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
>> +        sdk_toolchain = "build-essential"
> These blocks have more and more redundancies, specifically as we only
> have two possible outcomes: cross or native build.
>
Right. I have made no attempt to refactor the code at this point. The
sole purpose at this time was to check if forcing use of the host
sbuild-chroot for that special case would break something else (which I
am hoping the CI tests that I have started would catch)
>> +    # not cross-compiling? use target
>> +    elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>> distro_arch == None:
> You have extracted host_arch above already.
Indeed and ditto (no attempt to refactor the code)
>
> If the special case you found can be simply addressed here, we can do
> that. I doubt, though, that it has a lot practical relevance. It's more
> for consistency.
Thanks for the quick feedback Jan! I'll make sure that the actual patch
does have the code refactored
> Jan
>
>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>            sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>>            sdk_toolchain = "build-essential"
>> +    # cross-compiling for the foreign-arch: use host with cross toolchain
>>        else:
>>            d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>>            schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>>
>>
>>> Thanks,
>>> Adriaan
>>>
>>>> Thanks
>>>> Cedric
>>>>
>>>> Refs:
>>>>        next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in
>>>> KAS_IMAGE_PREINSTALL list
>>>>        master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix
>>>> internal deb import for sbuild
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "isar-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to isar-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1-
>>>> 0eccd72ef1a0%40siemens.com.
>>

Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka]

On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote:
> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>>> Hello,
>>>>>
>>>>> I was wondering if handling of -native was working as designed (or if I
>>>>> am missing something).
>>>>>
>>>>> Let's consider the following steps:
>>>>>
>>>>>        git clone -b next https://github.com/ilbers/isar
>>>>>        . isar-init-build-env
>>>>>        mc:qemuarm64-bookworm:hello-native
>>>>>
>>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>>> configured to build an amd64 package while running on an arm64 userland
>>>>> => the build of that package alone takes several minutes (on an Intel
>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>>
>>>>> In addition, when using Isar from master (where support for -native and
>>>>> -compat is already there), the build fails to install build dependencies
>>>>> for hello-native:
>>>>>
>>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>>> it is not installable
>>>>> |                                          Depends: libstdc++-dev:amd64
>>>>> but it is not installable
>>>>>
>>>>> This issue may also be reproduced on next with
>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>>> builds) removed. I guess I just making a note that -native may have
>>>>> issues on master.
>>>>>
>>>>> My outstanding question would be: shouldn't -native use the *host*
>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>>> Hi Cedric,
>>>>
>>>> Yes, you are right. The problem seems to be that when building -native,
>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>>
>>>> Can you try what happens with:
>>>>
>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>> index df26f47f..508ba72d 100644
>>>> --- a/meta/classes/crossvars.bbclass
>>>> +++ b/meta/classes/crossvars.bbclass
>>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>>
>>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>>> sure - will give this is a try but I fear it won't work because the
>>> condition mode == "0" remains True
>>> You will find further below the changes that I am currently testing (the
>>> hello-native use-case is now working as expected, currently running CI
>>> tests)
>>>
>>>>        if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>> My RFC changes below
>>>
>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>> index df26f47..1378333 100644
>>> --- a/meta/classes/crossvars.bbclass
>>> +++ b/meta/classes/crossvars.bbclass
>>> @@ -15,12 +15,24 @@ python __anonymous() {
>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>
>>> +    # get arch triplet
>>> +    host_arch = d.getVar('HOST_ARCH')
>>>        distro_arch = d.getVar('DISTRO_ARCH')
>>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>>> distro_arch == None:
>>> +    package_arch = d.getVar('PACKAGE_ARCH')
>>> +
>>> +    # -native build within a foreign-arch build? use host
>>> +    if package_arch == host_arch and package_arch != distro_arch:
>> package_arch == host_arch looks redundant here.
> 
> wouldn't dropping it make the second case from the below table pick the
> host when we really want the target?
> 
> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
> | config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
> HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
> | qemuarm64-bookworm | hello-native | amd64        | arm64       |
> amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
> | qemuarm64-bookworm | hello        | arm64        | amd64       |
> amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
> | qemuamd64-bookworm | hello-native | amd64        | amd64       |
> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
> | qemuamd64-bookworm | hello        | amd64        | amd64       |
> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
> 

This is missing one variable: ISAR_CROSS_COMPILE

How about this condition to decide whether to use the host build env?

if distro_arch != host_arch and \
   (package_arch == host_arch or \
    (package_arch == distro_arch and mode == "1")):
    d.setVar('BUILD_ARCH', host_arch)
    schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
    sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
    sdk_toolchain = "crossbuild-essential-" + distro_arch
else:
    d.setVar('BUILD_ARCH', distro_arch)
    schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
    sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
    sdk_toolchain = "build-essential"

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka]

On 06.10.23 14:02, 'Jan Kiszka' via isar-users wrote:
> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote:
>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>>>> Hello,
>>>>>>
>>>>>> I was wondering if handling of -native was working as designed (or if I
>>>>>> am missing something).
>>>>>>
>>>>>> Let's consider the following steps:
>>>>>>
>>>>>>        git clone -b next https://github.com/ilbers/isar
>>>>>>        . isar-init-build-env
>>>>>>        mc:qemuarm64-bookworm:hello-native
>>>>>>
>>>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>>>> configured to build an amd64 package while running on an arm64 userland
>>>>>> => the build of that package alone takes several minutes (on an Intel
>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>>>
>>>>>> In addition, when using Isar from master (where support for -native and
>>>>>> -compat is already there), the build fails to install build dependencies
>>>>>> for hello-native:
>>>>>>
>>>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>>>> it is not installable
>>>>>> |                                          Depends: libstdc++-dev:amd64
>>>>>> but it is not installable
>>>>>>
>>>>>> This issue may also be reproduced on next with
>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>>>> builds) removed. I guess I just making a note that -native may have
>>>>>> issues on master.
>>>>>>
>>>>>> My outstanding question would be: shouldn't -native use the *host*
>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>>>> Hi Cedric,
>>>>>
>>>>> Yes, you are right. The problem seems to be that when building -native,
>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>>>
>>>>> Can you try what happens with:
>>>>>
>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>> index df26f47f..508ba72d 100644
>>>>> --- a/meta/classes/crossvars.bbclass
>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>
>>>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>>>> sure - will give this is a try but I fear it won't work because the
>>>> condition mode == "0" remains True
>>>> You will find further below the changes that I am currently testing (the
>>>> hello-native use-case is now working as expected, currently running CI
>>>> tests)
>>>>
>>>>>        if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>>> My RFC changes below
>>>>
>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>> index df26f47..1378333 100644
>>>> --- a/meta/classes/crossvars.bbclass
>>>> +++ b/meta/classes/crossvars.bbclass
>>>> @@ -15,12 +15,24 @@ python __anonymous() {
>>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>>
>>>> +    # get arch triplet
>>>> +    host_arch = d.getVar('HOST_ARCH')
>>>>        distro_arch = d.getVar('DISTRO_ARCH')
>>>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>>>> distro_arch == None:
>>>> +    package_arch = d.getVar('PACKAGE_ARCH')
>>>> +
>>>> +    # -native build within a foreign-arch build? use host
>>>> +    if package_arch == host_arch and package_arch != distro_arch:
>>> package_arch == host_arch looks redundant here.
>>
>> wouldn't dropping it make the second case from the below table pick the
>> host when we really want the target?
>>
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
>> HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuarm64-bookworm | hello-native | amd64        | arm64       |
>> amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuarm64-bookworm | hello        | arm64        | amd64       |
>> amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuamd64-bookworm | hello-native | amd64        | amd64       |
>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuamd64-bookworm | hello        | amd64        | amd64       |
>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>
> 
> This is missing one variable: ISAR_CROSS_COMPILE
> 
> How about this condition to decide whether to use the host build env?
> 
> if distro_arch != host_arch and \
>    (package_arch == host_arch or \
>     (package_arch == distro_arch and mode == "1")):

Nope, does not work: There is no PACKAGE_ARCH if we are building images.
PACKAGE_ARCH is only define in dpkg recipes. We need to work without
that variable - let me think...

Jan

>     d.setVar('BUILD_ARCH', host_arch)
>     schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>     sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
>     sdk_toolchain = "crossbuild-essential-" + distro_arch
> else:
>     d.setVar('BUILD_ARCH', distro_arch)
>     schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>     sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>     sdk_toolchain = "build-essential"
> 
> Jan
> 

-- 
Siemens AG, Technology
Linux Expert Center

Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka]

On 06.10.23 14:12, 'Jan Kiszka' via isar-users wrote:
> On 06.10.23 14:02, 'Jan Kiszka' via isar-users wrote:
>> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote:
>>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
>>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I was wondering if handling of -native was working as designed (or if I
>>>>>>> am missing something).
>>>>>>>
>>>>>>> Let's consider the following steps:
>>>>>>>
>>>>>>>        git clone -b next https://github.com/ilbers/isar
>>>>>>>        . isar-init-build-env
>>>>>>>        mc:qemuarm64-bookworm:hello-native
>>>>>>>
>>>>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>>>>> configured to build an amd64 package while running on an arm64 userland
>>>>>>> => the build of that package alone takes several minutes (on an Intel
>>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>>>>
>>>>>>> In addition, when using Isar from master (where support for -native and
>>>>>>> -compat is already there), the build fails to install build dependencies
>>>>>>> for hello-native:
>>>>>>>
>>>>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>>>>> it is not installable
>>>>>>> |                                          Depends: libstdc++-dev:amd64
>>>>>>> but it is not installable
>>>>>>>
>>>>>>> This issue may also be reproduced on next with
>>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>>>>> builds) removed. I guess I just making a note that -native may have
>>>>>>> issues on master.
>>>>>>>
>>>>>>> My outstanding question would be: shouldn't -native use the *host*
>>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>>>>> Hi Cedric,
>>>>>>
>>>>>> Yes, you are right. The problem seems to be that when building -native,
>>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>>>>
>>>>>> Can you try what happens with:
>>>>>>
>>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>>> index df26f47f..508ba72d 100644
>>>>>> --- a/meta/classes/crossvars.bbclass
>>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>>
>>>>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>>>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>>>>> sure - will give this is a try but I fear it won't work because the
>>>>> condition mode == "0" remains True
>>>>> You will find further below the changes that I am currently testing (the
>>>>> hello-native use-case is now working as expected, currently running CI
>>>>> tests)
>>>>>
>>>>>>        if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>>>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>>>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>>>> My RFC changes below
>>>>>
>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>> index df26f47..1378333 100644
>>>>> --- a/meta/classes/crossvars.bbclass
>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>> @@ -15,12 +15,24 @@ python __anonymous() {
>>>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>
>>>>> +    # get arch triplet
>>>>> +    host_arch = d.getVar('HOST_ARCH')
>>>>>        distro_arch = d.getVar('DISTRO_ARCH')
>>>>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>>>>> distro_arch == None:
>>>>> +    package_arch = d.getVar('PACKAGE_ARCH')
>>>>> +
>>>>> +    # -native build within a foreign-arch build? use host
>>>>> +    if package_arch == host_arch and package_arch != distro_arch:
>>>> package_arch == host_arch looks redundant here.
>>>
>>> wouldn't dropping it make the second case from the below table pick the
>>> host when we really want the target?
>>>
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
>>> HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuarm64-bookworm | hello-native | amd64        | arm64       |
>>> amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuarm64-bookworm | hello        | arm64        | amd64       |
>>> amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuamd64-bookworm | hello-native | amd64        | amd64       |
>>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuamd64-bookworm | hello        | amd64        | amd64       |
>>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>>
>>
>> This is missing one variable: ISAR_CROSS_COMPILE
>>
>> How about this condition to decide whether to use the host build env?
>>
>> if distro_arch != host_arch and \
>>    (package_arch == host_arch or \
>>     (package_arch == distro_arch and mode == "1")):
> 
> Nope, does not work: There is no PACKAGE_ARCH if we are building images.
> PACKAGE_ARCH is only define in dpkg recipes. We need to work without
> that variable - let me think...
> 

There is in fact, via bitbake.conf (we need to clean up dpkg.bbclass),
and it's DISTRO_ARCH. This should work as needed.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: -native working as designed? target vs host sbuild-chroot

[cedric.hombourger]

On 06/10/2023 14:02, Kiszka, Jan (T CED) wrote:
> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote:
>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>>>> Hello,
>>>>>>
>>>>>> I was wondering if handling of -native was working as designed (or if I
>>>>>> am missing something).
>>>>>>
>>>>>> Let's consider the following steps:
>>>>>>
>>>>>>         git clone -b next https://github.com/ilbers/isar
>>>>>>         . isar-init-build-env
>>>>>>         mc:qemuarm64-bookworm:hello-native
>>>>>>
>>>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>>>> configured to build an amd64 package while running on an arm64 userland
>>>>>> => the build of that package alone takes several minutes (on an Intel
>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>>>
>>>>>> In addition, when using Isar from master (where support for -native and
>>>>>> -compat is already there), the build fails to install build dependencies
>>>>>> for hello-native:
>>>>>>
>>>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>>>> it is not installable
>>>>>> |                                          Depends: libstdc++-dev:amd64
>>>>>> but it is not installable
>>>>>>
>>>>>> This issue may also be reproduced on next with
>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>>>> builds) removed. I guess I just making a note that -native may have
>>>>>> issues on master.
>>>>>>
>>>>>> My outstanding question would be: shouldn't -native use the *host*
>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>>>> Hi Cedric,
>>>>>
>>>>> Yes, you are right. The problem seems to be that when building -native,
>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>>>
>>>>> Can you try what happens with:
>>>>>
>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>> index df26f47f..508ba72d 100644
>>>>> --- a/meta/classes/crossvars.bbclass
>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>>>         flavor = d.getVar('SBUILD_FLAVOR')
>>>>>         flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>
>>>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>>>> sure - will give this is a try but I fear it won't work because the
>>>> condition mode == "0" remains True
>>>> You will find further below the changes that I am currently testing (the
>>>> hello-native use-case is now working as expected, currently running CI
>>>> tests)
>>>>
>>>>>         if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>>>             d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>>>             schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>>> My RFC changes below
>>>>
>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>> index df26f47..1378333 100644
>>>> --- a/meta/classes/crossvars.bbclass
>>>> +++ b/meta/classes/crossvars.bbclass
>>>> @@ -15,12 +15,24 @@ python __anonymous() {
>>>>         flavor = d.getVar('SBUILD_FLAVOR')
>>>>         flavor_suffix = ('-' + flavor) if flavor else ''
>>>>
>>>> +    # get arch triplet
>>>> +    host_arch = d.getVar('HOST_ARCH')
>>>>         distro_arch = d.getVar('DISTRO_ARCH')
>>>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>>>> distro_arch == None:
>>>> +    package_arch = d.getVar('PACKAGE_ARCH')
>>>> +
>>>> +    # -native build within a foreign-arch build? use host
>>>> +    if package_arch == host_arch and package_arch != distro_arch:
>>> package_arch == host_arch looks redundant here.
>> wouldn't dropping it make the second case from the below table pick the
>> host when we really want the target?
>>
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
>> HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuarm64-bookworm | hello-native | amd64        | arm64       |
>> amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuarm64-bookworm | hello        | arm64        | amd64       |
>> amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuamd64-bookworm | hello-native | amd64        | amd64       |
>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>> | qemuamd64-bookworm | hello        | amd64        | amd64       |
>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>
> This is missing one variable: ISAR_CROSS_COMPILE
>
> How about this condition to decide whether to use the host build env?
>
> if distro_arch != host_arch and \
>     (package_arch == host_arch or \
>      (package_arch == distro_arch and mode == "1")):
>      d.setVar('BUILD_ARCH', host_arch)
>      schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>      sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
>      sdk_toolchain = "crossbuild-essential-" + distro_arch
> else:
>      d.setVar('BUILD_ARCH', distro_arch)
>      schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>      sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>      sdk_toolchain = "build-essential"

I believe that would work. I have copied the conditional to some unit
test code and the sbuild-chroot and sdk-toolchain settings look ok for
all the input configurations

I'm happy to update the patch and let the CI build check for regressions

Thanks!

> Jan
>

Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka]

On 06.10.23 16:56, Hombourger, Cedric (DI SW CAS ES LI) wrote:
> On 06/10/2023 14:02, Kiszka, Jan (T CED) wrote:
>> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote:
>>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
>>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I was wondering if handling of -native was working as designed (or if I
>>>>>>> am missing something).
>>>>>>>
>>>>>>> Let's consider the following steps:
>>>>>>>
>>>>>>>         git clone -b next https://github.com/ilbers/isar
>>>>>>>         . isar-init-build-env
>>>>>>>         mc:qemuarm64-bookworm:hello-native
>>>>>>>
>>>>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>>>>> configured to build an amd64 package while running on an arm64 userland
>>>>>>> => the build of that package alone takes several minutes (on an Intel
>>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>>>>
>>>>>>> In addition, when using Isar from master (where support for -native and
>>>>>>> -compat is already there), the build fails to install build dependencies
>>>>>>> for hello-native:
>>>>>>>
>>>>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>>>>> it is not installable
>>>>>>> |                                          Depends: libstdc++-dev:amd64
>>>>>>> but it is not installable
>>>>>>>
>>>>>>> This issue may also be reproduced on next with
>>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>>>>> builds) removed. I guess I just making a note that -native may have
>>>>>>> issues on master.
>>>>>>>
>>>>>>> My outstanding question would be: shouldn't -native use the *host*
>>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>>>>> Hi Cedric,
>>>>>>
>>>>>> Yes, you are right. The problem seems to be that when building -native,
>>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>>>>
>>>>>> Can you try what happens with:
>>>>>>
>>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>>> index df26f47f..508ba72d 100644
>>>>>> --- a/meta/classes/crossvars.bbclass
>>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>>>>         flavor = d.getVar('SBUILD_FLAVOR')
>>>>>>         flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>>
>>>>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>>>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>>>>> sure - will give this is a try but I fear it won't work because the
>>>>> condition mode == "0" remains True
>>>>> You will find further below the changes that I am currently testing (the
>>>>> hello-native use-case is now working as expected, currently running CI
>>>>> tests)
>>>>>
>>>>>>         if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>>>>             d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>>>>             schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>>>> My RFC changes below
>>>>>
>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>> index df26f47..1378333 100644
>>>>> --- a/meta/classes/crossvars.bbclass
>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>> @@ -15,12 +15,24 @@ python __anonymous() {
>>>>>         flavor = d.getVar('SBUILD_FLAVOR')
>>>>>         flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>
>>>>> +    # get arch triplet
>>>>> +    host_arch = d.getVar('HOST_ARCH')
>>>>>         distro_arch = d.getVar('DISTRO_ARCH')
>>>>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>>>>> distro_arch == None:
>>>>> +    package_arch = d.getVar('PACKAGE_ARCH')
>>>>> +
>>>>> +    # -native build within a foreign-arch build? use host
>>>>> +    if package_arch == host_arch and package_arch != distro_arch:
>>>> package_arch == host_arch looks redundant here.
>>> wouldn't dropping it make the second case from the below table pick the
>>> host when we really want the target?
>>>
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
>>> HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuarm64-bookworm | hello-native | amd64        | arm64       |
>>> amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuarm64-bookworm | hello        | arm64        | amd64       |
>>> amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuamd64-bookworm | hello-native | amd64        | amd64       |
>>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuamd64-bookworm | hello        | amd64        | amd64       |
>>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>>
>> This is missing one variable: ISAR_CROSS_COMPILE
>>
>> How about this condition to decide whether to use the host build env?
>>
>> if distro_arch != host_arch and \
>>     (package_arch == host_arch or \
>>      (package_arch == distro_arch and mode == "1")):
>>      d.setVar('BUILD_ARCH', host_arch)
>>      schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>>      sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
>>      sdk_toolchain = "crossbuild-essential-" + distro_arch
>> else:
>>      d.setVar('BUILD_ARCH', distro_arch)
>>      schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>      sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>>      sdk_toolchain = "build-essential"
> 
> I believe that would work. I have copied the conditional to some unit
> test code and the sbuild-chroot and sdk-toolchain settings look ok for
> all the input configurations
> 
> I'm happy to update the patch and let the CI build check for regressions
> 

I have a corresponding patch in v2 of my related series. Will send out
soon, had some unrelated fallouts.

Jan

-- 
Siemens AG, Technology
Linux Expert Center