* -native working as designed? target vs host sbuild-chroot @ 2023-10-06 2:49 cedric.hombourger 2023-10-06 4:59 ` Schmidt, Adriaan 2023-10-06 5:20 ` Jan Kiszka 0 siblings, 2 replies; 11+ messages in thread From: cedric.hombourger @ 2023-10-06 2:49 UTC (permalink / raw) To: isar-users Hello, I was wondering if handling of -native was working as designed (or if I am missing something). Let's consider the following steps: git clone -b next https://github.com/ilbers/isar . isar-init-build-env mc:qemuarm64-bookworm:hello-native Observe that the build constructs sbuild-chroot-target when I would have expected sbuild-chroot-host. The sbuild environment for hello is indeed configured to build an amd64 package while running on an arm64 userland => the build of that package alone takes several minutes (on an Intel Xeon 6242 and with /proc/cpuinfo listing 64 processors) In addition, when using Isar from master (where support for -native and -compat is already there), the build fails to install build dependencies for hello-native: | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but it is not installable | Depends: libstdc++-dev:amd64 but it is not installable This issue may also be reproduced on next with 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native builds) removed. I guess I just making a note that -native may have issues on master. My outstanding question would be: shouldn't -native use the *host* sbuild-chroot? If we agree that it should then I am happy to look into this. Thanks Cedric Refs: next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in KAS_IMAGE_PREINSTALL list master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix internal deb import for sbuild ^ permalink raw reply [flat|nested] 11+ messages in thread
* RE: -native working as designed? target vs host sbuild-chroot 2023-10-06 2:49 -native working as designed? target vs host sbuild-chroot cedric.hombourger @ 2023-10-06 4:59 ` Schmidt, Adriaan 2023-10-06 5:03 ` cedric.hombourger 2023-10-06 5:20 ` Jan Kiszka 1 sibling, 1 reply; 11+ messages in thread From: Schmidt, Adriaan @ 2023-10-06 4:59 UTC (permalink / raw) To: cedric.hombourger, isar-users cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: > Hello, > > I was wondering if handling of -native was working as designed (or if I > am missing something). > > Let's consider the following steps: > > git clone -b next https://github.com/ilbers/isar > . isar-init-build-env > mc:qemuarm64-bookworm:hello-native > > Observe that the build constructs sbuild-chroot-target when I would have > expected sbuild-chroot-host. The sbuild environment for hello is indeed > configured to build an amd64 package while running on an arm64 userland > => the build of that package alone takes several minutes (on an Intel > Xeon 6242 and with /proc/cpuinfo listing 64 processors) > > In addition, when using Isar from master (where support for -native and > -compat is already there), the build fails to install build dependencies > for hello-native: > > | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but > it is not installable > | Depends: libstdc++-dev:amd64 > but it is not installable > > This issue may also be reproduced on next with > 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native > builds) removed. I guess I just making a note that -native may have > issues on master. > > My outstanding question would be: shouldn't -native use the *host* > sbuild-chroot? If we agree that it should then I am happy to look into this. Hi Cedric, Yes, you are right. The problem seems to be that when building -native, native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. Can you try what happens with: diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass index df26f47f..508ba72d 100644 --- a/meta/classes/crossvars.bbclass +++ b/meta/classes/crossvars.bbclass @@ -15,7 +15,7 @@ python __anonymous() { flavor = d.getVar('SBUILD_FLAVOR') flavor_suffix = ('-' + flavor) if flavor else '' - distro_arch = d.getVar('DISTRO_ARCH') + distro_arch = d.getVar('PACKAGE_ARCH') if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: d.setVar('BUILD_HOST_ARCH', distro_arch) schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) Thanks, Adriaan > Thanks > Cedric > > Refs: > next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in > KAS_IMAGE_PREINSTALL list > master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix > internal deb import for sbuild > > > > -- > You received this message because you are subscribed to the Google Groups > "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1- > 0eccd72ef1a0%40siemens.com. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 4:59 ` Schmidt, Adriaan @ 2023-10-06 5:03 ` cedric.hombourger 2023-10-06 5:27 ` Jan Kiszka 0 siblings, 1 reply; 11+ messages in thread From: cedric.hombourger @ 2023-10-06 5:03 UTC (permalink / raw) To: Schmidt, Adriaan, isar-users On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: > cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >> Hello, >> >> I was wondering if handling of -native was working as designed (or if I >> am missing something). >> >> Let's consider the following steps: >> >> git clone -b next https://github.com/ilbers/isar >> . isar-init-build-env >> mc:qemuarm64-bookworm:hello-native >> >> Observe that the build constructs sbuild-chroot-target when I would have >> expected sbuild-chroot-host. The sbuild environment for hello is indeed >> configured to build an amd64 package while running on an arm64 userland >> => the build of that package alone takes several minutes (on an Intel >> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >> >> In addition, when using Isar from master (where support for -native and >> -compat is already there), the build fails to install build dependencies >> for hello-native: >> >> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >> it is not installable >> | Depends: libstdc++-dev:amd64 >> but it is not installable >> >> This issue may also be reproduced on next with >> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >> builds) removed. I guess I just making a note that -native may have >> issues on master. >> >> My outstanding question would be: shouldn't -native use the *host* >> sbuild-chroot? If we agree that it should then I am happy to look into this. > Hi Cedric, > > Yes, you are right. The problem seems to be that when building -native, > native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot > (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. > > Can you try what happens with: > > diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass > index df26f47f..508ba72d 100644 > --- a/meta/classes/crossvars.bbclass > +++ b/meta/classes/crossvars.bbclass > @@ -15,7 +15,7 @@ python __anonymous() { > flavor = d.getVar('SBUILD_FLAVOR') > flavor_suffix = ('-' + flavor) if flavor else '' > > - distro_arch = d.getVar('DISTRO_ARCH') > + distro_arch = d.getVar('PACKAGE_ARCH') sure - will give this is a try but I fear it won't work because the condition mode == "0" remains True You will find further below the changes that I am currently testing (the hello-native use-case is now working as expected, currently running CI tests) > if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: > d.setVar('BUILD_HOST_ARCH', distro_arch) > schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) My RFC changes below diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass index df26f47..1378333 100644 --- a/meta/classes/crossvars.bbclass +++ b/meta/classes/crossvars.bbclass @@ -15,12 +15,24 @@ python __anonymous() { flavor = d.getVar('SBUILD_FLAVOR') flavor_suffix = ('-' + flavor) if flavor else '' + # get arch triplet + host_arch = d.getVar('HOST_ARCH') distro_arch = d.getVar('DISTRO_ARCH') - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: + package_arch = d.getVar('PACKAGE_ARCH') + + # -native build within a foreign-arch build? use host + if package_arch == host_arch and package_arch != distro_arch: + d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) + schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) + sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" + sdk_toolchain = "build-essential" + # not cross-compiling? use target + elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: d.setVar('BUILD_HOST_ARCH', distro_arch) schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" sdk_toolchain = "build-essential" + # cross-compiling for the foreign-arch: use host with cross toolchain else: d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) > Thanks, > Adriaan > >> Thanks >> Cedric >> >> Refs: >> next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in >> KAS_IMAGE_PREINSTALL list >> master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix >> internal deb import for sbuild >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "isar-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to isar-users+unsubscribe@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1- >> 0eccd72ef1a0%40siemens.com. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 5:03 ` cedric.hombourger @ 2023-10-06 5:27 ` Jan Kiszka 2023-10-06 6:17 ` cedric.hombourger 0 siblings, 1 reply; 11+ messages in thread From: Jan Kiszka @ 2023-10-06 5:27 UTC (permalink / raw) To: cedric.hombourger, Schmidt, Adriaan, isar-users On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: > On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>> Hello, >>> >>> I was wondering if handling of -native was working as designed (or if I >>> am missing something). >>> >>> Let's consider the following steps: >>> >>> git clone -b next https://github.com/ilbers/isar >>> . isar-init-build-env >>> mc:qemuarm64-bookworm:hello-native >>> >>> Observe that the build constructs sbuild-chroot-target when I would have >>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>> configured to build an amd64 package while running on an arm64 userland >>> => the build of that package alone takes several minutes (on an Intel >>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>> >>> In addition, when using Isar from master (where support for -native and >>> -compat is already there), the build fails to install build dependencies >>> for hello-native: >>> >>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>> it is not installable >>> | Depends: libstdc++-dev:amd64 >>> but it is not installable >>> >>> This issue may also be reproduced on next with >>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>> builds) removed. I guess I just making a note that -native may have >>> issues on master. >>> >>> My outstanding question would be: shouldn't -native use the *host* >>> sbuild-chroot? If we agree that it should then I am happy to look into this. >> Hi Cedric, >> >> Yes, you are right. The problem seems to be that when building -native, >> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >> >> Can you try what happens with: >> >> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >> index df26f47f..508ba72d 100644 >> --- a/meta/classes/crossvars.bbclass >> +++ b/meta/classes/crossvars.bbclass >> @@ -15,7 +15,7 @@ python __anonymous() { >> flavor = d.getVar('SBUILD_FLAVOR') >> flavor_suffix = ('-' + flavor) if flavor else '' >> >> - distro_arch = d.getVar('DISTRO_ARCH') >> + distro_arch = d.getVar('PACKAGE_ARCH') > > sure - will give this is a try but I fear it won't work because the > condition mode == "0" remains True > You will find further below the changes that I am currently testing (the > hello-native use-case is now working as expected, currently running CI > tests) > >> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >> d.setVar('BUILD_HOST_ARCH', distro_arch) >> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) > > My RFC changes below > > diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass > index df26f47..1378333 100644 > --- a/meta/classes/crossvars.bbclass > +++ b/meta/classes/crossvars.bbclass > @@ -15,12 +15,24 @@ python __anonymous() { > flavor = d.getVar('SBUILD_FLAVOR') > flavor_suffix = ('-' + flavor) if flavor else '' > > + # get arch triplet > + host_arch = d.getVar('HOST_ARCH') > distro_arch = d.getVar('DISTRO_ARCH') > - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or > distro_arch == None: > + package_arch = d.getVar('PACKAGE_ARCH') > + > + # -native build within a foreign-arch build? use host > + if package_arch == host_arch and package_arch != distro_arch: package_arch == host_arch looks redundant here. > + d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) > + schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) > + sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" > + sdk_toolchain = "build-essential" These blocks have more and more redundancies, specifically as we only have two possible outcomes: cross or native build. > + # not cross-compiling? use target > + elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or > distro_arch == None: You have extracted host_arch above already. If the special case you found can be simply addressed here, we can do that. I doubt, though, that it has a lot practical relevance. It's more for consistency. Jan > d.setVar('BUILD_HOST_ARCH', distro_arch) > schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) > sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" > sdk_toolchain = "build-essential" > + # cross-compiling for the foreign-arch: use host with cross toolchain > else: > d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) > schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) > > >> Thanks, >> Adriaan >> >>> Thanks >>> Cedric >>> >>> Refs: >>> next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in >>> KAS_IMAGE_PREINSTALL list >>> master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix >>> internal deb import for sbuild >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "isar-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to isar-users+unsubscribe@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1- >>> 0eccd72ef1a0%40siemens.com. > > -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 5:27 ` Jan Kiszka @ 2023-10-06 6:17 ` cedric.hombourger 2023-10-06 12:02 ` Jan Kiszka 0 siblings, 1 reply; 11+ messages in thread From: cedric.hombourger @ 2023-10-06 6:17 UTC (permalink / raw) To: Kiszka, Jan, Schmidt, Adriaan, isar-users On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote: > On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: >> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>>> Hello, >>>> >>>> I was wondering if handling of -native was working as designed (or if I >>>> am missing something). >>>> >>>> Let's consider the following steps: >>>> >>>> git clone -b next https://github.com/ilbers/isar >>>> . isar-init-build-env >>>> mc:qemuarm64-bookworm:hello-native >>>> >>>> Observe that the build constructs sbuild-chroot-target when I would have >>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>>> configured to build an amd64 package while running on an arm64 userland >>>> => the build of that package alone takes several minutes (on an Intel >>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>>> >>>> In addition, when using Isar from master (where support for -native and >>>> -compat is already there), the build fails to install build dependencies >>>> for hello-native: >>>> >>>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>>> it is not installable >>>> | Depends: libstdc++-dev:amd64 >>>> but it is not installable >>>> >>>> This issue may also be reproduced on next with >>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>>> builds) removed. I guess I just making a note that -native may have >>>> issues on master. >>>> >>>> My outstanding question would be: shouldn't -native use the *host* >>>> sbuild-chroot? If we agree that it should then I am happy to look into this. >>> Hi Cedric, >>> >>> Yes, you are right. The problem seems to be that when building -native, >>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >>> >>> Can you try what happens with: >>> >>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>> index df26f47f..508ba72d 100644 >>> --- a/meta/classes/crossvars.bbclass >>> +++ b/meta/classes/crossvars.bbclass >>> @@ -15,7 +15,7 @@ python __anonymous() { >>> flavor = d.getVar('SBUILD_FLAVOR') >>> flavor_suffix = ('-' + flavor) if flavor else '' >>> >>> - distro_arch = d.getVar('DISTRO_ARCH') >>> + distro_arch = d.getVar('PACKAGE_ARCH') >> sure - will give this is a try but I fear it won't work because the >> condition mode == "0" remains True >> You will find further below the changes that I am currently testing (the >> hello-native use-case is now working as expected, currently running CI >> tests) >> >>> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >>> d.setVar('BUILD_HOST_ARCH', distro_arch) >>> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >> My RFC changes below >> >> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >> index df26f47..1378333 100644 >> --- a/meta/classes/crossvars.bbclass >> +++ b/meta/classes/crossvars.bbclass >> @@ -15,12 +15,24 @@ python __anonymous() { >> flavor = d.getVar('SBUILD_FLAVOR') >> flavor_suffix = ('-' + flavor) if flavor else '' >> >> + # get arch triplet >> + host_arch = d.getVar('HOST_ARCH') >> distro_arch = d.getVar('DISTRO_ARCH') >> - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >> distro_arch == None: >> + package_arch = d.getVar('PACKAGE_ARCH') >> + >> + # -native build within a foreign-arch build? use host >> + if package_arch == host_arch and package_arch != distro_arch: > package_arch == host_arch looks redundant here. wouldn't dropping it make the second case from the below table pick the host when we really want the target? +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ | config | recipe | PACKAGE_ARCH | DISTRO_ARCH | HOST_ARCH | sbuild-chroot | proposed check | eval | +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ | qemuarm64-bookworm | hello-native | amd64 | arm64 | amd64 | host | amd64 == amd64 and amd64 != arm64 | True | +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ | qemuarm64-bookworm | hello | arm64 | amd64 | amd64 | target | arm64 == amd64 and arm64 != amd64 | False | +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ | qemuamd64-bookworm | hello-native | amd64 | amd64 | amd64 | target | amd64 == amd64 and amd64 != amd64 | False | +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ | qemuamd64-bookworm | hello | amd64 | amd64 | amd64 | target | amd64 == amd64 and amd64 != amd64 | False | +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > >> + d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) >> + schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) >> + sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" >> + sdk_toolchain = "build-essential" > These blocks have more and more redundancies, specifically as we only > have two possible outcomes: cross or native build. > Right. I have made no attempt to refactor the code at this point. The sole purpose at this time was to check if forcing use of the host sbuild-chroot for that special case would break something else (which I am hoping the CI tests that I have started would catch) >> + # not cross-compiling? use target >> + elif mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >> distro_arch == None: > You have extracted host_arch above already. Indeed and ditto (no attempt to refactor the code) > > If the special case you found can be simply addressed here, we can do > that. I doubt, though, that it has a lot practical relevance. It's more > for consistency. Thanks for the quick feedback Jan! I'll make sure that the actual patch does have the code refactored > Jan > >> d.setVar('BUILD_HOST_ARCH', distro_arch) >> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >> sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" >> sdk_toolchain = "build-essential" >> + # cross-compiling for the foreign-arch: use host with cross toolchain >> else: >> d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH')) >> schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) >> >> >>> Thanks, >>> Adriaan >>> >>>> Thanks >>>> Cedric >>>> >>>> Refs: >>>> next @ b602e36defa0abacb9ba874b4e98bc9741a006a0 kas: Fix a typo in >>>> KAS_IMAGE_PREINSTALL list >>>> master @ 61086915e6c83fff22effa85cda64a2ac0c2f100 dpkg: Fix >>>> internal deb import for sbuild >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "isar-users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to isar-users+unsubscribe@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/isar-users/207e5270-361a-478b-bfb1- >>>> 0eccd72ef1a0%40siemens.com. >> ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 6:17 ` cedric.hombourger @ 2023-10-06 12:02 ` Jan Kiszka 2023-10-06 12:12 ` Jan Kiszka 2023-10-06 14:56 ` cedric.hombourger 0 siblings, 2 replies; 11+ messages in thread From: Jan Kiszka @ 2023-10-06 12:02 UTC (permalink / raw) To: Hombourger, Cedric (DI SW CAS ES LI), Schmidt, Adriaan (T CED SES-DE), isar-users On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote: > On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote: >> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: >>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>>>> Hello, >>>>> >>>>> I was wondering if handling of -native was working as designed (or if I >>>>> am missing something). >>>>> >>>>> Let's consider the following steps: >>>>> >>>>> git clone -b next https://github.com/ilbers/isar >>>>> . isar-init-build-env >>>>> mc:qemuarm64-bookworm:hello-native >>>>> >>>>> Observe that the build constructs sbuild-chroot-target when I would have >>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>>>> configured to build an amd64 package while running on an arm64 userland >>>>> => the build of that package alone takes several minutes (on an Intel >>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>>>> >>>>> In addition, when using Isar from master (where support for -native and >>>>> -compat is already there), the build fails to install build dependencies >>>>> for hello-native: >>>>> >>>>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>>>> it is not installable >>>>> | Depends: libstdc++-dev:amd64 >>>>> but it is not installable >>>>> >>>>> This issue may also be reproduced on next with >>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>>>> builds) removed. I guess I just making a note that -native may have >>>>> issues on master. >>>>> >>>>> My outstanding question would be: shouldn't -native use the *host* >>>>> sbuild-chroot? If we agree that it should then I am happy to look into this. >>>> Hi Cedric, >>>> >>>> Yes, you are right. The problem seems to be that when building -native, >>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >>>> >>>> Can you try what happens with: >>>> >>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>> index df26f47f..508ba72d 100644 >>>> --- a/meta/classes/crossvars.bbclass >>>> +++ b/meta/classes/crossvars.bbclass >>>> @@ -15,7 +15,7 @@ python __anonymous() { >>>> flavor = d.getVar('SBUILD_FLAVOR') >>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>> >>>> - distro_arch = d.getVar('DISTRO_ARCH') >>>> + distro_arch = d.getVar('PACKAGE_ARCH') >>> sure - will give this is a try but I fear it won't work because the >>> condition mode == "0" remains True >>> You will find further below the changes that I am currently testing (the >>> hello-native use-case is now working as expected, currently running CI >>> tests) >>> >>>> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >>>> d.setVar('BUILD_HOST_ARCH', distro_arch) >>>> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >>> My RFC changes below >>> >>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>> index df26f47..1378333 100644 >>> --- a/meta/classes/crossvars.bbclass >>> +++ b/meta/classes/crossvars.bbclass >>> @@ -15,12 +15,24 @@ python __anonymous() { >>> flavor = d.getVar('SBUILD_FLAVOR') >>> flavor_suffix = ('-' + flavor) if flavor else '' >>> >>> + # get arch triplet >>> + host_arch = d.getVar('HOST_ARCH') >>> distro_arch = d.getVar('DISTRO_ARCH') >>> - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >>> distro_arch == None: >>> + package_arch = d.getVar('PACKAGE_ARCH') >>> + >>> + # -native build within a foreign-arch build? use host >>> + if package_arch == host_arch and package_arch != distro_arch: >> package_arch == host_arch looks redundant here. > > wouldn't dropping it make the second case from the below table pick the > host when we really want the target? > > +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > | config | recipe | PACKAGE_ARCH | DISTRO_ARCH | > HOST_ARCH | sbuild-chroot | proposed check | eval | > +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > | qemuarm64-bookworm | hello-native | amd64 | arm64 | > amd64 | host | amd64 == amd64 and amd64 != arm64 | True | > +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > | qemuarm64-bookworm | hello | arm64 | amd64 | > amd64 | target | arm64 == amd64 and arm64 != amd64 | False | > +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > | qemuamd64-bookworm | hello-native | amd64 | amd64 | > amd64 | target | amd64 == amd64 and amd64 != amd64 | False | > +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > | qemuamd64-bookworm | hello | amd64 | amd64 | > amd64 | target | amd64 == amd64 and amd64 != amd64 | False | > +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ > This is missing one variable: ISAR_CROSS_COMPILE How about this condition to decide whether to use the host build env? if distro_arch != host_arch and \ (package_arch == host_arch or \ (package_arch == distro_arch and mode == "1")): d.setVar('BUILD_ARCH', host_arch) schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" sdk_toolchain = "crossbuild-essential-" + distro_arch else: d.setVar('BUILD_ARCH', distro_arch) schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" sdk_toolchain = "build-essential" Jan -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 12:02 ` Jan Kiszka @ 2023-10-06 12:12 ` Jan Kiszka 2023-10-06 12:30 ` Jan Kiszka 2023-10-06 14:56 ` cedric.hombourger 1 sibling, 1 reply; 11+ messages in thread From: Jan Kiszka @ 2023-10-06 12:12 UTC (permalink / raw) To: Hombourger, Cedric (DI SW CAS ES LI), Schmidt, Adriaan (T CED SES-DE), isar-users On 06.10.23 14:02, 'Jan Kiszka' via isar-users wrote: > On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote: >> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote: >>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: >>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>>>>> Hello, >>>>>> >>>>>> I was wondering if handling of -native was working as designed (or if I >>>>>> am missing something). >>>>>> >>>>>> Let's consider the following steps: >>>>>> >>>>>> git clone -b next https://github.com/ilbers/isar >>>>>> . isar-init-build-env >>>>>> mc:qemuarm64-bookworm:hello-native >>>>>> >>>>>> Observe that the build constructs sbuild-chroot-target when I would have >>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>>>>> configured to build an amd64 package while running on an arm64 userland >>>>>> => the build of that package alone takes several minutes (on an Intel >>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>>>>> >>>>>> In addition, when using Isar from master (where support for -native and >>>>>> -compat is already there), the build fails to install build dependencies >>>>>> for hello-native: >>>>>> >>>>>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>>>>> it is not installable >>>>>> | Depends: libstdc++-dev:amd64 >>>>>> but it is not installable >>>>>> >>>>>> This issue may also be reproduced on next with >>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>>>>> builds) removed. I guess I just making a note that -native may have >>>>>> issues on master. >>>>>> >>>>>> My outstanding question would be: shouldn't -native use the *host* >>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this. >>>>> Hi Cedric, >>>>> >>>>> Yes, you are right. The problem seems to be that when building -native, >>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >>>>> >>>>> Can you try what happens with: >>>>> >>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>>> index df26f47f..508ba72d 100644 >>>>> --- a/meta/classes/crossvars.bbclass >>>>> +++ b/meta/classes/crossvars.bbclass >>>>> @@ -15,7 +15,7 @@ python __anonymous() { >>>>> flavor = d.getVar('SBUILD_FLAVOR') >>>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>>> >>>>> - distro_arch = d.getVar('DISTRO_ARCH') >>>>> + distro_arch = d.getVar('PACKAGE_ARCH') >>>> sure - will give this is a try but I fear it won't work because the >>>> condition mode == "0" remains True >>>> You will find further below the changes that I am currently testing (the >>>> hello-native use-case is now working as expected, currently running CI >>>> tests) >>>> >>>>> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >>>>> d.setVar('BUILD_HOST_ARCH', distro_arch) >>>>> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >>>> My RFC changes below >>>> >>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>> index df26f47..1378333 100644 >>>> --- a/meta/classes/crossvars.bbclass >>>> +++ b/meta/classes/crossvars.bbclass >>>> @@ -15,12 +15,24 @@ python __anonymous() { >>>> flavor = d.getVar('SBUILD_FLAVOR') >>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>> >>>> + # get arch triplet >>>> + host_arch = d.getVar('HOST_ARCH') >>>> distro_arch = d.getVar('DISTRO_ARCH') >>>> - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >>>> distro_arch == None: >>>> + package_arch = d.getVar('PACKAGE_ARCH') >>>> + >>>> + # -native build within a foreign-arch build? use host >>>> + if package_arch == host_arch and package_arch != distro_arch: >>> package_arch == host_arch looks redundant here. >> >> wouldn't dropping it make the second case from the below table pick the >> host when we really want the target? >> >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | config | recipe | PACKAGE_ARCH | DISTRO_ARCH | >> HOST_ARCH | sbuild-chroot | proposed check | eval | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuarm64-bookworm | hello-native | amd64 | arm64 | >> amd64 | host | amd64 == amd64 and amd64 != arm64 | True | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuarm64-bookworm | hello | arm64 | amd64 | >> amd64 | target | arm64 == amd64 and arm64 != amd64 | False | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuamd64-bookworm | hello-native | amd64 | amd64 | >> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuamd64-bookworm | hello | amd64 | amd64 | >> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> > > This is missing one variable: ISAR_CROSS_COMPILE > > How about this condition to decide whether to use the host build env? > > if distro_arch != host_arch and \ > (package_arch == host_arch or \ > (package_arch == distro_arch and mode == "1")): Nope, does not work: There is no PACKAGE_ARCH if we are building images. PACKAGE_ARCH is only define in dpkg recipes. We need to work without that variable - let me think... Jan > d.setVar('BUILD_ARCH', host_arch) > schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) > sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" > sdk_toolchain = "crossbuild-essential-" + distro_arch > else: > d.setVar('BUILD_ARCH', distro_arch) > schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) > sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" > sdk_toolchain = "build-essential" > > Jan > -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 12:12 ` Jan Kiszka @ 2023-10-06 12:30 ` Jan Kiszka 0 siblings, 0 replies; 11+ messages in thread From: Jan Kiszka @ 2023-10-06 12:30 UTC (permalink / raw) To: Hombourger, Cedric (DI SW CAS ES LI), Schmidt, Adriaan (T CED SES-DE), isar-users On 06.10.23 14:12, 'Jan Kiszka' via isar-users wrote: > On 06.10.23 14:02, 'Jan Kiszka' via isar-users wrote: >> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote: >>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote: >>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: >>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>>>>>> Hello, >>>>>>> >>>>>>> I was wondering if handling of -native was working as designed (or if I >>>>>>> am missing something). >>>>>>> >>>>>>> Let's consider the following steps: >>>>>>> >>>>>>> git clone -b next https://github.com/ilbers/isar >>>>>>> . isar-init-build-env >>>>>>> mc:qemuarm64-bookworm:hello-native >>>>>>> >>>>>>> Observe that the build constructs sbuild-chroot-target when I would have >>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>>>>>> configured to build an amd64 package while running on an arm64 userland >>>>>>> => the build of that package alone takes several minutes (on an Intel >>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>>>>>> >>>>>>> In addition, when using Isar from master (where support for -native and >>>>>>> -compat is already there), the build fails to install build dependencies >>>>>>> for hello-native: >>>>>>> >>>>>>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>>>>>> it is not installable >>>>>>> | Depends: libstdc++-dev:amd64 >>>>>>> but it is not installable >>>>>>> >>>>>>> This issue may also be reproduced on next with >>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>>>>>> builds) removed. I guess I just making a note that -native may have >>>>>>> issues on master. >>>>>>> >>>>>>> My outstanding question would be: shouldn't -native use the *host* >>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this. >>>>>> Hi Cedric, >>>>>> >>>>>> Yes, you are right. The problem seems to be that when building -native, >>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >>>>>> >>>>>> Can you try what happens with: >>>>>> >>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>>>> index df26f47f..508ba72d 100644 >>>>>> --- a/meta/classes/crossvars.bbclass >>>>>> +++ b/meta/classes/crossvars.bbclass >>>>>> @@ -15,7 +15,7 @@ python __anonymous() { >>>>>> flavor = d.getVar('SBUILD_FLAVOR') >>>>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>>>> >>>>>> - distro_arch = d.getVar('DISTRO_ARCH') >>>>>> + distro_arch = d.getVar('PACKAGE_ARCH') >>>>> sure - will give this is a try but I fear it won't work because the >>>>> condition mode == "0" remains True >>>>> You will find further below the changes that I am currently testing (the >>>>> hello-native use-case is now working as expected, currently running CI >>>>> tests) >>>>> >>>>>> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >>>>>> d.setVar('BUILD_HOST_ARCH', distro_arch) >>>>>> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >>>>> My RFC changes below >>>>> >>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>>> index df26f47..1378333 100644 >>>>> --- a/meta/classes/crossvars.bbclass >>>>> +++ b/meta/classes/crossvars.bbclass >>>>> @@ -15,12 +15,24 @@ python __anonymous() { >>>>> flavor = d.getVar('SBUILD_FLAVOR') >>>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>>> >>>>> + # get arch triplet >>>>> + host_arch = d.getVar('HOST_ARCH') >>>>> distro_arch = d.getVar('DISTRO_ARCH') >>>>> - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >>>>> distro_arch == None: >>>>> + package_arch = d.getVar('PACKAGE_ARCH') >>>>> + >>>>> + # -native build within a foreign-arch build? use host >>>>> + if package_arch == host_arch and package_arch != distro_arch: >>>> package_arch == host_arch looks redundant here. >>> >>> wouldn't dropping it make the second case from the below table pick the >>> host when we really want the target? >>> >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | config | recipe | PACKAGE_ARCH | DISTRO_ARCH | >>> HOST_ARCH | sbuild-chroot | proposed check | eval | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuarm64-bookworm | hello-native | amd64 | arm64 | >>> amd64 | host | amd64 == amd64 and amd64 != arm64 | True | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuarm64-bookworm | hello | arm64 | amd64 | >>> amd64 | target | arm64 == amd64 and arm64 != amd64 | False | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuamd64-bookworm | hello-native | amd64 | amd64 | >>> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuamd64-bookworm | hello | amd64 | amd64 | >>> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> >> >> This is missing one variable: ISAR_CROSS_COMPILE >> >> How about this condition to decide whether to use the host build env? >> >> if distro_arch != host_arch and \ >> (package_arch == host_arch or \ >> (package_arch == distro_arch and mode == "1")): > > Nope, does not work: There is no PACKAGE_ARCH if we are building images. > PACKAGE_ARCH is only define in dpkg recipes. We need to work without > that variable - let me think... > There is in fact, via bitbake.conf (we need to clean up dpkg.bbclass), and it's DISTRO_ARCH. This should work as needed. Jan -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 12:02 ` Jan Kiszka 2023-10-06 12:12 ` Jan Kiszka @ 2023-10-06 14:56 ` cedric.hombourger 2023-10-06 15:19 ` Jan Kiszka 1 sibling, 1 reply; 11+ messages in thread From: cedric.hombourger @ 2023-10-06 14:56 UTC (permalink / raw) To: Kiszka, Jan, Schmidt, Adriaan, isar-users On 06/10/2023 14:02, Kiszka, Jan (T CED) wrote: > On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote: >> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote: >>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: >>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>>>>> Hello, >>>>>> >>>>>> I was wondering if handling of -native was working as designed (or if I >>>>>> am missing something). >>>>>> >>>>>> Let's consider the following steps: >>>>>> >>>>>> git clone -b next https://github.com/ilbers/isar >>>>>> . isar-init-build-env >>>>>> mc:qemuarm64-bookworm:hello-native >>>>>> >>>>>> Observe that the build constructs sbuild-chroot-target when I would have >>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>>>>> configured to build an amd64 package while running on an arm64 userland >>>>>> => the build of that package alone takes several minutes (on an Intel >>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>>>>> >>>>>> In addition, when using Isar from master (where support for -native and >>>>>> -compat is already there), the build fails to install build dependencies >>>>>> for hello-native: >>>>>> >>>>>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>>>>> it is not installable >>>>>> | Depends: libstdc++-dev:amd64 >>>>>> but it is not installable >>>>>> >>>>>> This issue may also be reproduced on next with >>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>>>>> builds) removed. I guess I just making a note that -native may have >>>>>> issues on master. >>>>>> >>>>>> My outstanding question would be: shouldn't -native use the *host* >>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this. >>>>> Hi Cedric, >>>>> >>>>> Yes, you are right. The problem seems to be that when building -native, >>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >>>>> >>>>> Can you try what happens with: >>>>> >>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>>> index df26f47f..508ba72d 100644 >>>>> --- a/meta/classes/crossvars.bbclass >>>>> +++ b/meta/classes/crossvars.bbclass >>>>> @@ -15,7 +15,7 @@ python __anonymous() { >>>>> flavor = d.getVar('SBUILD_FLAVOR') >>>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>>> >>>>> - distro_arch = d.getVar('DISTRO_ARCH') >>>>> + distro_arch = d.getVar('PACKAGE_ARCH') >>>> sure - will give this is a try but I fear it won't work because the >>>> condition mode == "0" remains True >>>> You will find further below the changes that I am currently testing (the >>>> hello-native use-case is now working as expected, currently running CI >>>> tests) >>>> >>>>> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >>>>> d.setVar('BUILD_HOST_ARCH', distro_arch) >>>>> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >>>> My RFC changes below >>>> >>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>> index df26f47..1378333 100644 >>>> --- a/meta/classes/crossvars.bbclass >>>> +++ b/meta/classes/crossvars.bbclass >>>> @@ -15,12 +15,24 @@ python __anonymous() { >>>> flavor = d.getVar('SBUILD_FLAVOR') >>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>> >>>> + # get arch triplet >>>> + host_arch = d.getVar('HOST_ARCH') >>>> distro_arch = d.getVar('DISTRO_ARCH') >>>> - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >>>> distro_arch == None: >>>> + package_arch = d.getVar('PACKAGE_ARCH') >>>> + >>>> + # -native build within a foreign-arch build? use host >>>> + if package_arch == host_arch and package_arch != distro_arch: >>> package_arch == host_arch looks redundant here. >> wouldn't dropping it make the second case from the below table pick the >> host when we really want the target? >> >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | config | recipe | PACKAGE_ARCH | DISTRO_ARCH | >> HOST_ARCH | sbuild-chroot | proposed check | eval | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuarm64-bookworm | hello-native | amd64 | arm64 | >> amd64 | host | amd64 == amd64 and amd64 != arm64 | True | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuarm64-bookworm | hello | arm64 | amd64 | >> amd64 | target | arm64 == amd64 and arm64 != amd64 | False | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuamd64-bookworm | hello-native | amd64 | amd64 | >> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> | qemuamd64-bookworm | hello | amd64 | amd64 | >> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >> > This is missing one variable: ISAR_CROSS_COMPILE > > How about this condition to decide whether to use the host build env? > > if distro_arch != host_arch and \ > (package_arch == host_arch or \ > (package_arch == distro_arch and mode == "1")): > d.setVar('BUILD_ARCH', host_arch) > schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) > sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" > sdk_toolchain = "crossbuild-essential-" + distro_arch > else: > d.setVar('BUILD_ARCH', distro_arch) > schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) > sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" > sdk_toolchain = "build-essential" I believe that would work. I have copied the conditional to some unit test code and the sbuild-chroot and sdk-toolchain settings look ok for all the input configurations I'm happy to update the patch and let the CI build check for regressions Thanks! > Jan > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 14:56 ` cedric.hombourger @ 2023-10-06 15:19 ` Jan Kiszka 0 siblings, 0 replies; 11+ messages in thread From: Jan Kiszka @ 2023-10-06 15:19 UTC (permalink / raw) To: Hombourger, Cedric (DI SW CAS ES LI), Schmidt, Adriaan (T CED SES-DE), isar-users On 06.10.23 16:56, Hombourger, Cedric (DI SW CAS ES LI) wrote: > On 06/10/2023 14:02, Kiszka, Jan (T CED) wrote: >> On 06.10.23 08:17, Hombourger, Cedric (DI SW CAS ES LI) wrote: >>> On 06/10/2023 07:27, Kiszka, Jan (T CED) wrote: >>>> On 06.10.23 07:03, 'cedric.hombourger@siemens.com' via isar-users wrote: >>>>> On 06/10/2023 06:59, Schmidt, Adriaan (T CED SES-DE) wrote: >>>>>> cedric.hombourger@siemens.com, Friday, October 6, 2023 4:50 AM: >>>>>>> Hello, >>>>>>> >>>>>>> I was wondering if handling of -native was working as designed (or if I >>>>>>> am missing something). >>>>>>> >>>>>>> Let's consider the following steps: >>>>>>> >>>>>>> git clone -b next https://github.com/ilbers/isar >>>>>>> . isar-init-build-env >>>>>>> mc:qemuarm64-bookworm:hello-native >>>>>>> >>>>>>> Observe that the build constructs sbuild-chroot-target when I would have >>>>>>> expected sbuild-chroot-host. The sbuild environment for hello is indeed >>>>>>> configured to build an amd64 package while running on an arm64 userland >>>>>>> => the build of that package alone takes several minutes (on an Intel >>>>>>> Xeon 6242 and with /proc/cpuinfo listing 64 processors) >>>>>>> >>>>>>> In addition, when using Isar from master (where support for -native and >>>>>>> -compat is already there), the build fails to install build dependencies >>>>>>> for hello-native: >>>>>>> >>>>>>> | sbuild-build-depends-main-dummy:amd64 : Depends: libc-dev:amd64 but >>>>>>> it is not installable >>>>>>> | Depends: libstdc++-dev:amd64 >>>>>>> but it is not installable >>>>>>> >>>>>>> This issue may also be reproduced on next with >>>>>>> 7bc68bb27f562257977bb5f701564ab08d4c5b3a (base: Fix HOST_ARCH for native >>>>>>> builds) removed. I guess I just making a note that -native may have >>>>>>> issues on master. >>>>>>> >>>>>>> My outstanding question would be: shouldn't -native use the *host* >>>>>>> sbuild-chroot? If we agree that it should then I am happy to look into this. >>>>>> Hi Cedric, >>>>>> >>>>>> Yes, you are right. The problem seems to be that when building -native, >>>>>> native.bbclass adapts PACKAGE_ARCH, but the selection of the sbuild-chroot >>>>>> (via SCHROOT_DEP in crossvars.bbclass) selects based on DISTRO_ARCH. >>>>>> >>>>>> Can you try what happens with: >>>>>> >>>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>>>> index df26f47f..508ba72d 100644 >>>>>> --- a/meta/classes/crossvars.bbclass >>>>>> +++ b/meta/classes/crossvars.bbclass >>>>>> @@ -15,7 +15,7 @@ python __anonymous() { >>>>>> flavor = d.getVar('SBUILD_FLAVOR') >>>>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>>>> >>>>>> - distro_arch = d.getVar('DISTRO_ARCH') >>>>>> + distro_arch = d.getVar('PACKAGE_ARCH') >>>>> sure - will give this is a try but I fear it won't work because the >>>>> condition mode == "0" remains True >>>>> You will find further below the changes that I am currently testing (the >>>>> hello-native use-case is now working as expected, currently running CI >>>>> tests) >>>>> >>>>>> if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or distro_arch == None: >>>>>> d.setVar('BUILD_HOST_ARCH', distro_arch) >>>>>> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >>>>> My RFC changes below >>>>> >>>>> diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass >>>>> index df26f47..1378333 100644 >>>>> --- a/meta/classes/crossvars.bbclass >>>>> +++ b/meta/classes/crossvars.bbclass >>>>> @@ -15,12 +15,24 @@ python __anonymous() { >>>>> flavor = d.getVar('SBUILD_FLAVOR') >>>>> flavor_suffix = ('-' + flavor) if flavor else '' >>>>> >>>>> + # get arch triplet >>>>> + host_arch = d.getVar('HOST_ARCH') >>>>> distro_arch = d.getVar('DISTRO_ARCH') >>>>> - if mode == "0" or d.getVar('HOST_ARCH') == distro_arch or >>>>> distro_arch == None: >>>>> + package_arch = d.getVar('PACKAGE_ARCH') >>>>> + >>>>> + # -native build within a foreign-arch build? use host >>>>> + if package_arch == host_arch and package_arch != distro_arch: >>>> package_arch == host_arch looks redundant here. >>> wouldn't dropping it make the second case from the below table pick the >>> host when we really want the target? >>> >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | config | recipe | PACKAGE_ARCH | DISTRO_ARCH | >>> HOST_ARCH | sbuild-chroot | proposed check | eval | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuarm64-bookworm | hello-native | amd64 | arm64 | >>> amd64 | host | amd64 == amd64 and amd64 != arm64 | True | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuarm64-bookworm | hello | arm64 | amd64 | >>> amd64 | target | arm64 == amd64 and arm64 != amd64 | False | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuamd64-bookworm | hello-native | amd64 | amd64 | >>> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> | qemuamd64-bookworm | hello | amd64 | amd64 | >>> amd64 | target | amd64 == amd64 and amd64 != amd64 | False | >>> +--------------------+--------------+--------------+-------------+-----------+---------------+-----------------------------------+-------+ >>> >> This is missing one variable: ISAR_CROSS_COMPILE >> >> How about this condition to decide whether to use the host build env? >> >> if distro_arch != host_arch and \ >> (package_arch == host_arch or \ >> (package_arch == distro_arch and mode == "1")): >> d.setVar('BUILD_ARCH', host_arch) >> schroot_dir = d.getVar('SCHROOT_HOST_DIR', False) >> sbuild_dep = "sbuild-chroot-host" + flavor_suffix + ":do_build" >> sdk_toolchain = "crossbuild-essential-" + distro_arch >> else: >> d.setVar('BUILD_ARCH', distro_arch) >> schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False) >> sbuild_dep = "sbuild-chroot-target" + flavor_suffix + ":do_build" >> sdk_toolchain = "build-essential" > > I believe that would work. I have copied the conditional to some unit > test code and the sbuild-chroot and sdk-toolchain settings look ok for > all the input configurations > > I'm happy to update the patch and let the CI build check for regressions > I have a corresponding patch in v2 of my related series. Will send out soon, had some unrelated fallouts. Jan -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: -native working as designed? target vs host sbuild-chroot 2023-10-06 2:49 -native working as designed? target vs host sbuild-chroot cedric.hombourger 2023-10-06 4:59 ` Schmidt, Adriaan @ 2023-10-06 5:20 ` Jan Kiszka 1 sibling, 0 replies; 11+ messages in thread From: Jan Kiszka @ 2023-10-06 5:20 UTC (permalink / raw) To: cedric.hombourger, isar-users, Schmidt, Adriaan On 06.10.23 04:49, 'cedric.hombourger@siemens.com' via isar-users wrote: > Hello, > > I was wondering if handling of -native was working as designed (or if I > am missing something). > > Let's consider the following steps: > > git clone -b next https://github.com/ilbers/isar > . isar-init-build-env This gives your ISAR_CROSS_COMPILE = 0. > mc:qemuarm64-bookworm:hello-native This is similar to the issue I'm trying to fix with my patch series sent yesterday evening: 'hello-native' in a non-cross environment should actually be 'hello' because non-cross means that there is no separate host build env. My patches will resolve that contradiction, but only for dependencies you have in your recipes, not for explicitly requested builds. I'm not sure if we can and should do anything about such explicit requests. Jan -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2023-10-06 15:20 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-10-06 2:49 -native working as designed? target vs host sbuild-chroot cedric.hombourger 2023-10-06 4:59 ` Schmidt, Adriaan 2023-10-06 5:03 ` cedric.hombourger 2023-10-06 5:27 ` Jan Kiszka 2023-10-06 6:17 ` cedric.hombourger 2023-10-06 12:02 ` Jan Kiszka 2023-10-06 12:12 ` Jan Kiszka 2023-10-06 12:30 ` Jan Kiszka 2023-10-06 14:56 ` cedric.hombourger 2023-10-06 15:19 ` Jan Kiszka 2023-10-06 5:20 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox