Re: -native working as designed? target vs host sbuild-chroot

["cedric.hombourger@siemens.com" <cedric.hombourger@siemens.com>]

On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote:
> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote:
>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote:
>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM:
>>>> Hello,
>>>>
>>>> I was wondering if handling of -native was working as designed (or if I
>>>> am missing something).
>>>>
>>>> Let's consider the following steps:
>>>>
>>>>        git clone -b next https://github.com/ilbers/isar
>>>>        . isar-init-build-env
>>>>        mc:qemuarm64-bookworm:hello-native
>>>>
>>>> Observe that the build constructs sbuild-chroot-target when I would have
>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed
>>>> configured to build an amd64 package while running on an arm64 userland
>>>> => the build of that package alone takes several minutes (on an Intel
>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors)
>>>>
>>>> In addition, when using Isar from master (where support for -native and
>>>> -compat is already there), the build fails to install build dependencies
>>>> for hello-native:
>>>>
>>>> |  sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but
>>>> it is not installable
>>>> |                                          Depends: libstdc++-dev:amd64
>>>> but it is not installable
>>>>
>>>> This issue may also be reproduced on next with
>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native
>>>> builds) removed. I guess I just making a note that -native may have
>>>> issues on master.
>>>>
>>>> My outstanding question would be: shouldn't -native use the *host*
>>>> sbuild-chroot? If we agree that it should then I am happy to look into this.
>>> Hi Cedric,
>>>
>>> Yes, you are right. The problem seems to be that when building -native,
>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot
>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH.
>>>
>>> Can you try what happens with:
>>>
>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>>> index df26f47f..508ba72d 100644
>>> --- a/meta/classes/crossvars.bbclass
>>> +++ b/meta/classes/crossvars.bbclass
>>> @@ -15,7 +15,7 @@ python __anonymous() {
>>>        flavor = d.getVar('SBUILD_FLAVOR')
>>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>>
>>> -    distro_arch = d.getVar('DISTRO_ARCH')
>>> +    distro_arch = d.getVar('PACKAGE_ARCH')
>> sure - will give this is a try but I fear it won't work because the
>> condition mode == "0" remains True
>> You will find further below the changes that I am currently testing (the
>> hello-native use-case is now working as expected, currently running CI
>> tests)
>>
>>>        if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None:
>>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>> My RFC changes below
>>
>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
>> index df26f47..1378333 100644
>> --- a/meta/classes/crossvars.bbclass
>> +++ b/meta/classes/crossvars.bbclass
>> @@ -15,12 +15,24 @@ python __anonymous() {
>>        flavor = d.getVar('SBUILD_FLAVOR')
>>        flavor_suffix = ('-' + flavor) if flavor else ''
>>
>> +    # get arch triplet
>> +    host_arch = d.getVar('HOST_ARCH')
>>        distro_arch = d.getVar('DISTRO_ARCH')
>> -    if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>> distro_arch == None:
>> +    package_arch = d.getVar('PACKAGE_ARCH')
>> +
>> +    # -native build within a foreign-arch build? use host
>> +    if package_arch == host_arch and package_arch != distro_arch:
> package_arch == host_arch looks redundant here.

wouldn't dropping it make the second case from the below table pick the
host when we really want the target?

+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| config             | recipe       | PACKAGE_ARCH | DISTRO_ARCH |
HOST_ARCH | sbuild-chroot | proposed check                    | eval  |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuarm64-bookworm | hello-native | amd64        | arm64       |
amd64     | host          | amd64 == amd64 and amd64 != arm64 | True  |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuarm64-bookworm | hello        | arm64        | amd64       |
amd64     | target        | arm64 == amd64 and arm64 != amd64 | False |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuamd64-bookworm | hello-native | amd64        | amd64       |
amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+
| qemuamd64-bookworm | hello        | amd64        | amd64       |
amd64     | target        | amd64 == amd64 and amd64 != amd64 | False |
+--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+

>
>> +        d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>> +        schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>> +        sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build"
>> +        sdk_toolchain = "build-essential"
> These blocks have more and more redundancies, specifically as we only
> have two possible outcomes: cross or native build.
>
Right. I have made no attempt to refactor the code at this point. The
sole purpose at this time was to check if forcing use of the host
sbuild-chroot for that special case would break something else (which I
am hoping the CI tests that I have started would catch)
>> +    # not cross-compiling? use target
>> +    elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or
>> distro_arch == None:
> You have extracted host_arch above already.
Indeed and ditto (no attempt to refactor the code)
>
> If the special case you found can be simply addressed here, we can do
> that. I doubt, though, that it has a lot practical relevance. It's more
> for consistency.
Thanks for the quick feedback Jan! I'll make sure that the actual patch
does have the code refactored
> Jan
>
>>            d.setVar('BUILD_HOST_ARCH', distro_arch)
>>            schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
>>            sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build"
>>            sdk_toolchain = "build-essential"
>> +    # cross-compiling for the foreign-arch: use host with cross toolchain
>>        else:
>>            d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
>>            schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
>>
>>
>>> Thanks,
>>> Adriaan
>>>
>>>> Thanks
>>>> Cedric
>>>>
>>>> Refs:
>>>>        next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in
>>>> KAS_IMAGE_PREINSTALL list
>>>>        master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix
>>>> internal deb import for sbuild
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "isar-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to isar-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1-
>>>> 0eccd72ef1a0%40siemens.com.
>>