Re: -native working as designed? target vs host sbuild-chroot

[Jan Kiszka <jan.kiszka@siemens.com>]

On 06.10.23 16:56, Hombourger, Cedric (DI SW CAS ES LI) wrote:
> On 06/10/2023 14:02, Kiszka, Jan (T CED) wrote:
>> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote:
>>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
>>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I was wondering if handling of -native was working as designed (or if I
>>>>>>> am missing something).
>>>>>>>
>>>>>>> Let's consider the following steps:
>>>>>>>
>>>>>>>         git clone -b next https://github.com/ilbers/isar
>>>>>>>         . isar-init-build-env
>>>>>>>         mc:qemuarm64-bookworm:hello-native
>>>>>>>
>>>>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>>>>> configured to build an amd64 package while running on an arm64 userland
>>>>>>> => the build of that package alone takes several minutes (on an Intel
>>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>>>>
>>>>>>> In addition, when using Isar from master (where support for -native and
>>>>>>> -compat is already there), the build fails to install build dependencies
>>>>>>> for hello-native:
>>>>>>>
>>>>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>>>>> it is not installable
>>>>>>> |                                          Depends: libstdc++-dev:amd64
>>>>>>> but it is not installable
>>>>>>>
>>>>>>> This issue may also be reproduced on next with
>>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>>>>> builds) removed. I guess I just making a note that -native may have
>>>>>>> issues on master.
>>>>>>>
>>>>>>> My outstanding question would be: shouldn't -native use the *host*
>>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>>>>> Hi Cedric,
>>>>>>
>>>>>> Yes, you are right. The problem seems to be that when building -native,
>>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>>>>
>>>>>> Can you try what happens with:
>>>>>>
>>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>>> index df26f47f..508ba72d 100644
>>>>>> --- a/meta/classes/crossvars.bbclass
>>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>>>>         flavor = d.getVar('SBUILD_FLAVOR')
>>>>>>         flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>>
>>>>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>>>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>>>>> sure - will give this is a try but I fear it won't work because the
>>>>> condition mode == "0" remains True
>>>>> You will find further below the changes that I am currently testing (the
>>>>> hello-native use-case is now working as expected, currently running CI
>>>>> tests)
>>>>>
>>>>>>         if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>>>>             d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>>>>             schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>>>> My RFC changes below
>>>>>
>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>>>> index df26f47..1378333 100644
>>>>> --- a/meta/classes/crossvars.bbclass
>>>>> +++ b/meta/classes/crossvars.bbclass
>>>>> @@ -15,12 +15,24 @@ python __anonymous() {
>>>>>         flavor = d.getVar('SBUILD_FLAVOR')
>>>>>         flavor_suffix = ('-' + flavor) if flavor else ''
>>>>>
>>>>> +    # get arch triplet
>>>>> +    host_arch = d.getVar('HOST_ARCH')
>>>>>         distro_arch = d.getVar('DISTRO_ARCH')
>>>>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>>>>> distro_arch == None:
>>>>> +    package_arch = d.getVar('PACKAGE_ARCH')
>>>>> +
>>>>> +    # -native build within a foreign-arch build? use host
>>>>> +    if package_arch == host_arch and package_arch != distro_arch:
>>>> package_arch == host_arch looks redundant here.
>>> wouldn't dropping it make the second case from the below table pick the
>>> host when we really want the target?
>>>
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
>>> HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuarm64-bookworm | hello-native | amd64        | arm64       |
>>> amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuarm64-bookworm | hello        | arm64        | amd64       |
>>> amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuamd64-bookworm | hello-native | amd64        | amd64       |
>>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>> | qemuamd64-bookworm | hello        | amd64        | amd64       |
>>> amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
>>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
>>>
>> This is missing one variable: ISAR_CROSS_COMPILE
>>
>> How about this condition to decide whether to use the host build env?
>>
>> if distro_arch != host_arch and \
>>     (package_arch == host_arch or \
>>      (package_arch == distro_arch and mode == "1")):
>>      d.setVar('BUILD_ARCH', host_arch)
>>      schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>>      sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
>>      sdk_toolchain = "crossbuild-essential-" + distro_arch
>> else:
>>      d.setVar('BUILD_ARCH', distro_arch)
>>      schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>      sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>>      sdk_toolchain = "build-essential"
> 
> I believe that would work. I have copied the conditional to some unit
> test code and the sbuild-chroot and sdk-toolchain settings look ok for
> all the input configurations
> 
> I'm happy to update the patch and let the CI build check for regressions
> 

I have a corresponding patch in v2 of my related series. Will send out
soon, had some unrelated fallouts.

Jan

-- 
Siemens AG, Technology
Linux Expert Center