[PATCH v2 0/6] Evolve SDK to chroot-free usage

[PATCH v2 0/6] Evolve SDK to chroot-free usage

[Jan Kiszka]

Changes in v2:
 - add "restore-chroot" mode to relocation script
 - add help to relocation script
 - update README.sdk

Original cover letter:

Currently, our SDK can primarily be used by chroot'ing into it, taking
the to-be-built project with you (bind mount etc.). This was enough for
a start but we can do better.

This series enhance the SDK to become usage as a normal cross-toolchain
on your host system. For that purpose, we add a script that relocates
the toolchain after installation, adjust binary search paths and
sysroot. So, the only remaining difference to a yocto or buildroot
toolchain is the need to run the relocation script once after unpacking
the SDK.

The series furthermore allows to customize the SDK content via the
image-generating layer.

The series (together with all other pending patching of mine) can also
be found at https://github.com/siemens/isar/commits/jan/queue.

While I already shrunk the SDK a bit by removing the now obsolete
isar-apt repo, I suspect we could further reduce the deployment size by
dropping chroot support completely, only including the compiler and
their library dependencies. OTOH, sdk-debian-buster-arm64.tar.xz is now
142M here, unpacked 680M, while ARM's
gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz has
260M packed and 1.5G(!) unpacked. So we are either not really off or are
still missing something.

Jan

Jan Kiszka (6):
  sdk: Add support for adding self-defined sdk packages
  sdk: Make all links in the SDK chroot relative
  sdk: Add script to relocate SDK
  sdk: Do not ship the isar-apt repo
  sdk: Inject sysroot path when calling relocated toolchain
  sdk: Update README.sdk

 doc/user_manual.md                                 |  1 +
 meta/classes/image-sdk-extension.bbclass           | 28 ++++++++-
 meta/recipes-devtools/sdkchroot/files/README.sdk   | 71 +++++++++++-----------
 .../sdkchroot/files/gcc-sysroot-wrapper.sh         | 16 +++++
 .../sdkchroot/files/relocate-sdk.sh                | 44 ++++++++++++++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb       | 35 ++++++-----
 6 files changed, 141 insertions(+), 54 deletions(-)
 create mode 100755 meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
 create mode 100755 meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh

-- 
2.16.4

[PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

We do not yet have a good algorithm for automatically adding build
dependencies to the sdk beyond the basic set, let's allow users to
append what they need by appending SDK_PREINSTALL. Analogously to other
images, also allow to install self-built packages, consequently using
SDK_INSTALL.

Based on original patch by Le Jin.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 doc/user_manual.md                           |  1 +
 meta/recipes-devtools/sdkchroot/sdkchroot.bb | 25 +++++++++++++++----------
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 0582ac74..6d95f26b 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -744,6 +744,7 @@ target binary artifacts. Developer chroots to sdk rootfs and develops applicatio
 
 User manually triggers creation of SDK root filesystem for his target platform by launching the task `do_populate_sdk` for target image, f.e.
 `bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
+Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
 
 The resulting SDK rootfs is archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz`.
 It is additionally available for direct use under `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}/`.
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index 467e6824..ab0a66dc 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -13,6 +13,10 @@ SRC_URI = " \
     file://README.sdk"
 PV = "0.1"
 
+SDK_INSTALL ?= ""
+
+DEPENDS += "${SDK_INSTALL}"
+
 TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
 TOOLCHAIN_${HOST_ARCH} = "build-essential"
 TOOLCHAIN_i386 = "build-essential"
@@ -21,7 +25,7 @@ inherit rootfs
 ROOTFS_ARCH = "${HOST_ARCH}"
 ROOTFS_DISTRO = "${HOST_DISTRO}"
 ROOTFSDIR = "${S}"
-ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}"
+ROOTFS_PACKAGES = "${SDK_PREINSTALL} ${SDK_INSTALL} ${TOOLCHAIN}"
 ROOTFS_FEATURES += "clean-package-cache generate-manifest"
 ROOTFS_MANIFEST_DEPLOY_DIR = "${DEPLOY_DIR_SDKCHROOT}"
 
@@ -31,15 +35,16 @@ python() {
             d.getVar("ROOTFS_ARCH")))
 }
 
-SDKCHROOT_PREINSTALL := "debhelper \
-                           autotools-dev \
-                           dpkg \
-                           locales \
-                           docbook-to-man \
-                           apt \
-                           automake \
-                           devscripts \
-                           equivs"
+SDK_PREINSTALL += " \
+    debhelper \
+    autotools-dev \
+    dpkg \
+    locales \
+    docbook-to-man \
+    apt \
+    automake \
+    devscripts \
+    equivs"
 
 S = "${WORKDIR}/rootfs"
 
-- 
2.16.4

Re: [PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Jan Kiszka]

On 28.03.20 12:24, [ext] Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> We do not yet have a good algorithm for automatically adding build
> dependencies to the sdk beyond the basic set, let's allow users to
> append what they need by appending SDK_PREINSTALL. Analogously to other
> images, also allow to install self-built packages, consequently using
> SDK_INSTALL.
> 
> Based on original patch by Le Jin.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  doc/user_manual.md                           |  1 +
>  meta/recipes-devtools/sdkchroot/sdkchroot.bb | 25 +++++++++++++++----------
>  2 files changed, 16 insertions(+), 10 deletions(-)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 0582ac74..6d95f26b 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -744,6 +744,7 @@ target binary artifacts. Developer chroots to sdk rootfs and develops applicatio
>  
>  User manually triggers creation of SDK root filesystem for his target platform by launching the task `do_populate_sdk` for target image, f.e.
>  `bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
> +Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
>  
>  The resulting SDK rootfs is archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz`.
>  It is additionally available for direct use under `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}/`.
> diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> index 467e6824..ab0a66dc 100644
> --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> @@ -13,6 +13,10 @@ SRC_URI = " \
>      file://README.sdk"
>  PV = "0.1"
>  
> +SDK_INSTALL ?= ""
> +
> +DEPENDS += "${SDK_INSTALL}"
> +
>  TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
>  TOOLCHAIN_${HOST_ARCH} = "build-essential"
>  TOOLCHAIN_i386 = "build-essential"
> @@ -21,7 +25,7 @@ inherit rootfs
>  ROOTFS_ARCH = "${HOST_ARCH}"
>  ROOTFS_DISTRO = "${HOST_DISTRO}"
>  ROOTFSDIR = "${S}"
> -ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}"
> +ROOTFS_PACKAGES = "${SDK_PREINSTALL} ${SDK_INSTALL} ${TOOLCHAIN}"
>  ROOTFS_FEATURES += "clean-package-cache generate-manifest"
>  ROOTFS_MANIFEST_DEPLOY_DIR = "${DEPLOY_DIR_SDKCHROOT}"
>  
> @@ -31,15 +35,16 @@ python() {
>              d.getVar("ROOTFS_ARCH")))
>  }
>  
> -SDKCHROOT_PREINSTALL := "debhelper \
> -                           autotools-dev \
> -                           dpkg \
> -                           locales \
> -                           docbook-to-man \
> -                           apt \
> -                           automake \
> -                           devscripts \
> -                           equivs"
> +SDK_PREINSTALL += " \
> +    debhelper \
> +    autotools-dev \
> +    dpkg \
> +    locales \
> +    docbook-to-man \
> +    apt \
> +    automake \
> +    devscripts \
> +    equivs"
>  
>  S = "${WORKDIR}/rootfs"
>  
> 

I just realized another design issue with current sdk building:
sdkchroot is shared with all image recipes that request
do_populated_sdk. You may expect that defining SDP_[PRE]INSTALL there
would have any impact, but that's not true therefore. Only adding those
to an appended sdkchroot recipe or the global configuration works right
now. As these additions may very well be specific to individual target
images, neither of those options are perfect. We rather need to make the
sdkchroot specific to the target image requesting it.

That said, such a rework could happen later because the current form is
already fine for a lot of use cases (and is being used for a longer
while now).

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Baurzhan Ismagulov]

Hello Jan,

On Mon, Jun 08, 2020 at 12:22:26PM +0200, Jan Kiszka wrote:
> I just realized another design issue with current sdk building:
> sdkchroot is shared with all image recipes that request
> do_populated_sdk. You may expect that defining SDP_[PRE]INSTALL there
> would have any impact, but that's not true therefore. Only adding those
> to an appended sdkchroot recipe or the global configuration works right
> now. As these additions may very well be specific to individual target
> images, neither of those options are perfect. We rather need to make the
> sdkchroot specific to the target image requesting it.
> 
> That said, such a rework could happen later because the current form is
> already fine for a lot of use cases (and is being used for a longer
> while now).

Thanks for the heads up, this was not on our radar. Let me check it and get
back to you.

Our comments so far:

1. I think the series is a good improvement, thanks.

2. isar-apt is added to the SDK so that the developer can install anything
   after the initial preparation of the SDK. E.g., the user has SDK_INSTALL +=
   libhello-dev but during the work he decides to install also libbye-dev using
   apt. I suggest to keep isar-apt configured unless there is a better solution
   for this use case. This also implies that we should keep both approaches --
   relocate and chroot -- usable.

3. relocate-sdk -r puts two slashes into the binaries. Also, strictly speaking,
   it doesn't restore the binary into the state it was before, since rpath was
   not set by default. Should we just remove rpath instead?

4. It is also possible to set rpath to a relative path (in combination with
   $ORIGIN). In that way, the same SDK would be usable with or without chroot
   -- the use case is e.g. #2.

5. I'd suggest to drop "recommended" and "fallback" from README.sdk.
   relocate-sdk is already listed as the first option. The rest is a matter of
   developers' preference. Or is there a specific reason to prefer one?

   E.g., I personally use chroot to avoid host environment leakage into the
   built stuff. I'd like to work on the host, but in a Debian way (think
   multiarch -- which IIUC currently doesn't support amd64-stretch on
   amd64-buster).

6. rpath and wrapping doesn't address all use cases.

   1. Shared libraries may use other shared libraries.

      E.g., ffmpeg fails to find libopenal.so.1 if
      usr/lib/x86_64-linux-gnu/libavdevice.so.58 is not patched.

   2. Binaries outside of bin are not patched, e.g. clang-format.

   3. Wrapping doesn't cover e.g. arm-linux-gnueabihf-gcc on qemuarm-buster.

   What is your vision on this? I think we will never cover all possible cases
   and should rather strive for a working subset -- which your patch provides.
   I also think this should be documented, because I assume that Yocto SDK is
   not buggy in this regard (manual usability due to -isystem and stuff left
   aside) and encountering such problems with the Isar SDK would be a bad
   surprise.

With kind regards,
Baurzhan.

Re: [PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Jan Kiszka]

On 12.06.20 11:34, Baurzhan Ismagulov wrote:
> Hello Jan,
> 
> On Mon, Jun 08, 2020 at 12:22:26PM +0200, Jan Kiszka wrote:
>> I just realized another design issue with current sdk building:
>> sdkchroot is shared with all image recipes that request
>> do_populated_sdk. You may expect that defining SDP_[PRE]INSTALL there
>> would have any impact, but that's not true therefore. Only adding those
>> to an appended sdkchroot recipe or the global configuration works right
>> now. As these additions may very well be specific to individual target
>> images, neither of those options are perfect. We rather need to make the
>> sdkchroot specific to the target image requesting it.
>>
>> That said, such a rework could happen later because the current form is
>> already fine for a lot of use cases (and is being used for a longer
>> while now).
> 
> Thanks for the heads up, this was not on our radar. Let me check it and get
> back to you.
> 
> Our comments so far:
> 
> 1. I think the series is a good improvement, thanks.
> 
> 2. isar-apt is added to the SDK so that the developer can install anything
>    after the initial preparation of the SDK. E.g., the user has SDK_INSTALL +=
>    libhello-dev but during the work he decides to install also libbye-dev using
>    apt. I suggest to keep isar-apt configured unless there is a better solution
>    for this use case. This also implies that we should keep both approaches --
>    relocate and chroot -- usable.

If you want something in the SDK, add it upfront, via patch 1. Shipping
isar-apt is pointless bloat. Therefore the removal.

> 
> 3. relocate-sdk -r puts two slashes into the binaries. Also, strictly speaking,
>    it doesn't restore the binary into the state it was before, since rpath was
>    not set by default. Should we just remove rpath instead?

Of course, we could also remore rpath. Does it practically make a
difference?

> 
> 4. It is also possible to set rpath to a relative path (in combination with
>    $ORIGIN). In that way, the same SDK would be usable with or without chroot
>    -- the use case is e.g. #2.

How does that work then? Would we have to set rpath relative to the
respective binary location? If that worked, it would completely obsolete
a runtime script, right?

> 
> 5. I'd suggest to drop "recommended" and "fallback" from README.sdk.
>    relocate-sdk is already listed as the first option. The rest is a matter of
>    developers' preference. Or is there a specific reason to prefer one?

It's not just a preference because the chroot is more involved, requires
privileged and prevents many SDK use cases. Therefore this clear
recommendation.

> 
>    E.g., I personally use chroot to avoid host environment leakage into the
>    built stuff. I'd like to work on the host, but in a Debian way (think
>    multiarch -- which IIUC currently doesn't support amd64-stretch on
>    amd64-buster).

The whole point of the SDK is to enable easy application development.
Production builds would come an Isar environment, or a package builder.
You surely do not want to tell an ordinary developer to use sudo chroot
for daily application building.

> 
> 6. rpath and wrapping doesn't address all use cases.
> 
>    1. Shared libraries may use other shared libraries.
> 
>       E.g., ffmpeg fails to find libopenal.so.1 if
>       usr/lib/x86_64-linux-gnu/libavdevice.so.58 is not patched.

LD_LIBRARY_PATH

We may need some env tuning script for that.

> 
>    2. Binaries outside of bin are not patched, e.g. clang-format.

Then let's fix that.

> 
>    3. Wrapping doesn't cover e.g. arm-linux-gnueabihf-gcc on qemuarm-buster.
> 

Then let's fix that.

>    What is your vision on this? I think we will never cover all possible cases
>    and should rather strive for a working subset -- which your patch provides.
>    I also think this should be documented, because I assume that Yocto SDK is
>    not buggy in this regard (manual usability due to -isystem and stuff left
>    aside) and encountering such problems with the Isar SDK would be a bad
>    surprise.

The vision is clearly to resolve the remaining issues. It will be a
product feature and the way we promote the SDK. chroot is for legacy
only, may even be phased out eventually. Then we can remove also all
those unneeded debian tools from the image, only providing the compiler
with its libs.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Jan Kiszka]

On 12.06.20 11:54, Jan Kiszka wrote:
> On 12.06.20 11:34, Baurzhan Ismagulov wrote:
>> Hello Jan,
>>
>> On Mon, Jun 08, 2020 at 12:22:26PM +0200, Jan Kiszka wrote:
>>> I just realized another design issue with current sdk building:
>>> sdkchroot is shared with all image recipes that request
>>> do_populated_sdk. You may expect that defining SDP_[PRE]INSTALL there
>>> would have any impact, but that's not true therefore. Only adding those
>>> to an appended sdkchroot recipe or the global configuration works right
>>> now. As these additions may very well be specific to individual target
>>> images, neither of those options are perfect. We rather need to make the
>>> sdkchroot specific to the target image requesting it.
>>>
>>> That said, such a rework could happen later because the current form is
>>> already fine for a lot of use cases (and is being used for a longer
>>> while now).
>>
>> Thanks for the heads up, this was not on our radar. Let me check it and get
>> back to you.
>>
>> Our comments so far:
>>
>> 1. I think the series is a good improvement, thanks.
>>
>> 2. isar-apt is added to the SDK so that the developer can install anything
>>    after the initial preparation of the SDK. E.g., the user has SDK_INSTALL +=
>>    libhello-dev but during the work he decides to install also libbye-dev using
>>    apt. I suggest to keep isar-apt configured unless there is a better solution
>>    for this use case. This also implies that we should keep both approaches --
>>    relocate and chroot -- usable.
> 
> If you want something in the SDK, add it upfront, via patch 1. Shipping
> isar-apt is pointless bloat. Therefore the removal.
> 
>>
>> 3. relocate-sdk -r puts two slashes into the binaries. Also, strictly speaking,
>>    it doesn't restore the binary into the state it was before, since rpath was
>>    not set by default. Should we just remove rpath instead?
> 
> Of course, we could also remore rpath. Does it practically make a
> difference?
> 
>>
>> 4. It is also possible to set rpath to a relative path (in combination with
>>    $ORIGIN). In that way, the same SDK would be usable with or without chroot
>>    -- the use case is e.g. #2.
> 
> How does that work then? Would we have to set rpath relative to the
> respective binary location? If that worked, it would completely obsolete
> a runtime script, right?

--set-rpath
'$ORIGIN/../../usr/lib:$ORIGIN/../../usr/lib/${arch}-linux-gnu' ? That
would be cool...


> 
>>
>> 5. I'd suggest to drop "recommended" and "fallback" from README.sdk.
>>    relocate-sdk is already listed as the first option. The rest is a matter of
>>    developers' preference. Or is there a specific reason to prefer one?
> 
> It's not just a preference because the chroot is more involved, requires
> privileged and prevents many SDK use cases. Therefore this clear
> recommendation.
> 
>>
>>    E.g., I personally use chroot to avoid host environment leakage into the
>>    built stuff. I'd like to work on the host, but in a Debian way (think
>>    multiarch -- which IIUC currently doesn't support amd64-stretch on
>>    amd64-buster).
> 
> The whole point of the SDK is to enable easy application development.
> Production builds would come an Isar environment, or a package builder.
> You surely do not want to tell an ordinary developer to use sudo chroot
> for daily application building.
> 
>>
>> 6. rpath and wrapping doesn't address all use cases.
>>
>>    1. Shared libraries may use other shared libraries.
>>
>>       E.g., ffmpeg fails to find libopenal.so.1 if
>>       usr/lib/x86_64-linux-gnu/libavdevice.so.58 is not patched.
> 
> LD_LIBRARY_PATH
> 
> We may need some env tuning script for that.
> 

Or can we simply include the libs into patching?

Jan

>>
>>    2. Binaries outside of bin are not patched, e.g. clang-format.
> 
> Then let's fix that.
> 
>>
>>    3. Wrapping doesn't cover e.g. arm-linux-gnueabihf-gcc on qemuarm-buster.
>>
> 
> Then let's fix that.
> 
>>    What is your vision on this? I think we will never cover all possible cases
>>    and should rather strive for a working subset -- which your patch provides.
>>    I also think this should be documented, because I assume that Yocto SDK is
>>    not buggy in this regard (manual usability due to -isystem and stuff left
>>    aside) and encountering such problems with the Isar SDK would be a bad
>>    surprise.
> 
> The vision is clearly to resolve the remaining issues. It will be a
> product feature and the way we promote the SDK. chroot is for legacy
> only, may even be phased out eventually. Then we can remove also all
> those unneeded debian tools from the image, only providing the compiler
> with its libs.
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Jan Kiszka]

On 12.06.20 12:05, Jan Kiszka wrote:
> On 12.06.20 11:54, Jan Kiszka wrote:
>> On 12.06.20 11:34, Baurzhan Ismagulov wrote:
>>> Hello Jan,
>>>
>>> On Mon, Jun 08, 2020 at 12:22:26PM +0200, Jan Kiszka wrote:
>>>> I just realized another design issue with current sdk building:
>>>> sdkchroot is shared with all image recipes that request
>>>> do_populated_sdk. You may expect that defining SDP_[PRE]INSTALL there
>>>> would have any impact, but that's not true therefore. Only adding those
>>>> to an appended sdkchroot recipe or the global configuration works right
>>>> now. As these additions may very well be specific to individual target
>>>> images, neither of those options are perfect. We rather need to make the
>>>> sdkchroot specific to the target image requesting it.
>>>>
>>>> That said, such a rework could happen later because the current form is
>>>> already fine for a lot of use cases (and is being used for a longer
>>>> while now).
>>>
>>> Thanks for the heads up, this was not on our radar. Let me check it and get
>>> back to you.
>>>
>>> Our comments so far:
>>>
>>> 1. I think the series is a good improvement, thanks.
>>>
>>> 2. isar-apt is added to the SDK so that the developer can install anything
>>>    after the initial preparation of the SDK. E.g., the user has SDK_INSTALL +=
>>>    libhello-dev but during the work he decides to install also libbye-dev using
>>>    apt. I suggest to keep isar-apt configured unless there is a better solution
>>>    for this use case. This also implies that we should keep both approaches --
>>>    relocate and chroot -- usable.
>>
>> If you want something in the SDK, add it upfront, via patch 1. Shipping
>> isar-apt is pointless bloat. Therefore the removal.
>>
>>>
>>> 3. relocate-sdk -r puts two slashes into the binaries. Also, strictly speaking,
>>>    it doesn't restore the binary into the state it was before, since rpath was
>>>    not set by default. Should we just remove rpath instead?
>>
>> Of course, we could also remore rpath. Does it practically make a
>> difference?
>>
>>>
>>> 4. It is also possible to set rpath to a relative path (in combination with
>>>    $ORIGIN). In that way, the same SDK would be usable with or without chroot
>>>    -- the use case is e.g. #2.
>>
>> How does that work then? Would we have to set rpath relative to the
>> respective binary location? If that worked, it would completely obsolete
>> a runtime script, right?
> 
> --set-rpath
> '$ORIGIN/../../usr/lib:$ORIGIN/../../usr/lib/${arch}-linux-gnu' ? That
> would be cool...
> 

Yeah, would have been cool. The interpreter is biting us:

https://stackoverflow.com/questions/48452793/using-origin-to-specify-the-interpreter-in-elf-binaries-isnt-working

Just realized that I've been there already.

Jan

> 
>>
>>>
>>> 5. I'd suggest to drop "recommended" and "fallback" from README.sdk.
>>>    relocate-sdk is already listed as the first option. The rest is a matter of
>>>    developers' preference. Or is there a specific reason to prefer one?
>>
>> It's not just a preference because the chroot is more involved, requires
>> privileged and prevents many SDK use cases. Therefore this clear
>> recommendation.
>>
>>>
>>>    E.g., I personally use chroot to avoid host environment leakage into the
>>>    built stuff. I'd like to work on the host, but in a Debian way (think
>>>    multiarch -- which IIUC currently doesn't support amd64-stretch on
>>>    amd64-buster).
>>
>> The whole point of the SDK is to enable easy application development.
>> Production builds would come an Isar environment, or a package builder.
>> You surely do not want to tell an ordinary developer to use sudo chroot
>> for daily application building.
>>
>>>
>>> 6. rpath and wrapping doesn't address all use cases.
>>>
>>>    1. Shared libraries may use other shared libraries.
>>>
>>>       E.g., ffmpeg fails to find libopenal.so.1 if
>>>       usr/lib/x86_64-linux-gnu/libavdevice.so.58 is not patched.
>>
>> LD_LIBRARY_PATH
>>
>> We may need some env tuning script for that.
>>
> 
> Or can we simply include the libs into patching?
> 
> Jan
> 
>>>
>>>    2. Binaries outside of bin are not patched, e.g. clang-format.
>>
>> Then let's fix that.
>>
>>>
>>>    3. Wrapping doesn't cover e.g. arm-linux-gnueabihf-gcc on qemuarm-buster.
>>>
>>
>> Then let's fix that.
>>
>>>    What is your vision on this? I think we will never cover all possible cases
>>>    and should rather strive for a working subset -- which your patch provides.
>>>    I also think this should be documented, because I assume that Yocto SDK is
>>>    not buggy in this regard (manual usability due to -isystem and stuff left
>>>    aside) and encountering such problems with the Isar SDK would be a bad
>>>    surprise.
>>
>> The vision is clearly to resolve the remaining issues. It will be a
>> product feature and the way we promote the SDK. chroot is for legacy
>> only, may even be phased out eventually. Then we can remove also all
>> those unneeded debian tools from the image, only providing the compiler
>> with its libs.
>>
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v2 1/6] sdk: Add support for adding self-defined sdk packages

[Jan Kiszka]

On 12.06.20 12:05, [ext] Jan Kiszka wrote:
> On 12.06.20 11:54, Jan Kiszka wrote:
>> On 12.06.20 11:34, Baurzhan Ismagulov wrote:
>>> Hello Jan,
>>>
>>> On Mon, Jun 08, 2020 at 12:22:26PM +0200, Jan Kiszka wrote:
>>>> I just realized another design issue with current sdk building:
>>>> sdkchroot is shared with all image recipes that request
>>>> do_populated_sdk. You may expect that defining SDP_[PRE]INSTALL there
>>>> would have any impact, but that's not true therefore. Only adding those
>>>> to an appended sdkchroot recipe or the global configuration works right
>>>> now. As these additions may very well be specific to individual target
>>>> images, neither of those options are perfect. We rather need to make the
>>>> sdkchroot specific to the target image requesting it.
>>>>
>>>> That said, such a rework could happen later because the current form is
>>>> already fine for a lot of use cases (and is being used for a longer
>>>> while now).
>>>
>>> Thanks for the heads up, this was not on our radar. Let me check it and get
>>> back to you.
>>>
>>> Our comments so far:
>>>
>>> 1. I think the series is a good improvement, thanks.
>>>
>>> 2. isar-apt is added to the SDK so that the developer can install anything
>>>    after the initial preparation of the SDK. E.g., the user has SDK_INSTALL +=
>>>    libhello-dev but during the work he decides to install also libbye-dev using
>>>    apt. I suggest to keep isar-apt configured unless there is a better solution
>>>    for this use case. This also implies that we should keep both approaches --
>>>    relocate and chroot -- usable.
>>
>> If you want something in the SDK, add it upfront, via patch 1. Shipping
>> isar-apt is pointless bloat. Therefore the removal.
>>
>>>
>>> 3. relocate-sdk -r puts two slashes into the binaries. Also, strictly speaking,
>>>    it doesn't restore the binary into the state it was before, since rpath was
>>>    not set by default. Should we just remove rpath instead?
>>
>> Of course, we could also remore rpath. Does it practically make a
>> difference?
>>
>>>
>>> 4. It is also possible to set rpath to a relative path (in combination with
>>>    $ORIGIN). In that way, the same SDK would be usable with or without chroot
>>>    -- the use case is e.g. #2.
>>
>> How does that work then? Would we have to set rpath relative to the
>> respective binary location? If that worked, it would completely obsolete
>> a runtime script, right?
> 
> --set-rpath
> '$ORIGIN/../../usr/lib:$ORIGIN/../../usr/lib/${arch}-linux-gnu' ? That
> would be cool...
> 
> 
>>
>>>
>>> 5. I'd suggest to drop "recommended" and "fallback" from README.sdk.
>>>    relocate-sdk is already listed as the first option. The rest is a matter of
>>>    developers' preference. Or is there a specific reason to prefer one?
>>
>> It's not just a preference because the chroot is more involved, requires
>> privileged and prevents many SDK use cases. Therefore this clear
>> recommendation.
>>
>>>
>>>    E.g., I personally use chroot to avoid host environment leakage into the
>>>    built stuff. I'd like to work on the host, but in a Debian way (think
>>>    multiarch -- which IIUC currently doesn't support amd64-stretch on
>>>    amd64-buster).
>>
>> The whole point of the SDK is to enable easy application development.
>> Production builds would come an Isar environment, or a package builder.
>> You surely do not want to tell an ordinary developer to use sudo chroot
>> for daily application building.
>>
>>>
>>> 6. rpath and wrapping doesn't address all use cases.
>>>
>>>    1. Shared libraries may use other shared libraries.
>>>
>>>       E.g., ffmpeg fails to find libopenal.so.1 if
>>>       usr/lib/x86_64-linux-gnu/libavdevice.so.58 is not patched.
>>
>> LD_LIBRARY_PATH
>>
>> We may need some env tuning script for that.
>>
> 
> Or can we simply include the libs into patching?
> 

Any idea on that? And what exactly did you test?

> Jan
> 
>>>
>>>    2. Binaries outside of bin are not patched, e.g. clang-format.
>>
>> Then let's fix that.

Not sure if that need to be in scope. If you look into a cross
toolchain, you find <arch>-linux-gnu-* binaries, and this is generally
sufficient.

>>
>>>
>>>    3. Wrapping doesn't cover e.g. arm-linux-gnueabihf-gcc on qemuarm-buster.
>>>
>>
>> Then let's fix that.

Trivially fixed now.

>>
>>>    What is your vision on this? I think we will never cover all possible cases
>>>    and should rather strive for a working subset -- which your patch provides.
>>>    I also think this should be documented, because I assume that Yocto SDK is
>>>    not buggy in this regard (manual usability due to -isystem and stuff left
>>>    aside) and encountering such problems with the Isar SDK would be a bad
>>>    surprise.
>>
>> The vision is clearly to resolve the remaining issues. It will be a
>> product feature and the way we promote the SDK. chroot is for legacy
>> only, may even be phased out eventually. Then we can remove also all
>> those unneeded debian tools from the image, only providing the compiler
>> with its libs.
>>
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[PATCH v2 2/6] sdk: Make all links in the SDK chroot relative

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This, in combination with interp and rpatch rewriting, will allow to use
the SDK outside of its chroot.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/image-sdk-extension.bbclass | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index b9f2cf4a..84ac9c6d 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -18,6 +18,20 @@ do_populate_sdk() {
     # Remove setup scripts
     sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh ${SDKCHROOT_DIR}/configscript.sh
 
+    # Make all links relative
+    for link in $(find ${SDKCHROOT_DIR}/ -type l); do
+        target=$(readlink $link)
+
+        if [ "${target#/}" != "${target}" ]; then
+            basedir=$(dirname $link)
+            new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${SDKCHROOT_DIR}/${target})
+
+            # remove first to allow rewriting directory links
+            sudo rm $link
+            sudo ln -s $new_target $link
+        fi
+    done
+
     # Copy mount_chroot.sh for convenience
     sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
 
-- 
2.16.4

[PATCH v2 3/6] sdk: Add script to relocate SDK

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

When run inside the unpacked SDK, this script tunes interp and rpath
entry in relevant binaries so that the cross conpilation tool can be
called outside of the chroot, irrespective of the host distribution.
Then only "--sysroot /path/to/sdkroot" needs to be passed to the
compiler.

The script also supports restoring the original chroot-mode when invoked
with the --restore-chroot option.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .../sdkchroot/files/relocate-sdk.sh                | 41 ++++++++++++++++++++++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  2 ++
 2 files changed, 43 insertions(+)
 create mode 100755 meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh

diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
new file mode 100755
index 00000000..1c9b02fa
--- /dev/null
+++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
@@ -0,0 +1,41 @@
+#!/bin/sh
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+sdkroot=$(realpath $(dirname $0))
+arch=$(uname -m)
+
+new_sdkroot=$sdkroot
+
+case "$1" in
+--help|-h)
+	echo "Usage: $0 [--restore-chroot|-r]"
+	exit 0
+	;;
+--restore-chroot|-r)
+	new_sdkroot=/
+	;;
+esac
+
+if [ -z $(which patchelf 2>/dev/null) ]; then
+	echo "Please install 'patchelf' package first."
+	exit 1
+fi
+
+echo -n "Adjusting path of SDK to '${new_sdkroot}'... "
+
+for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/gcc* -executable -type f); do
+	interpreter=$(patchelf --print-interpreter ${binary} 2>/dev/null)
+	oldpath=${interpreter%/lib*/ld-linux*}
+	interpreter=${interpreter#${oldpath}}
+	if [ -n "${interpreter}" ]; then
+		patchelf --set-interpreter ${new_sdkroot}${interpreter} \
+			--set-rpath ${new_sdkroot}/usr/lib:${new_sdkroot}/usr/lib/${arch}-linux-gnu \
+			$binary 2>/dev/null
+	fi
+done
+
+echo "done"
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index ab0a66dc..2bc9d291 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -10,6 +10,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
 
 SRC_URI = " \
     file://configscript.sh \
+    file://relocate-sdk.sh \
     file://README.sdk"
 PV = "0.1"
 
@@ -58,6 +59,7 @@ ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
 sdkchroot_install_files() {
     # Configure root filesystem
     sudo install -m 644 ${WORKDIR}/README.sdk ${S}
+    sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
     sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
     sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
 }
-- 
2.16.4

[PATCH v2 4/6] sdk: Do not ship the isar-apt repo

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Users can add what should be included via SDK_INSTALL now.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/image-sdk-extension.bbclass     | 6 +++---
 meta/recipes-devtools/sdkchroot/sdkchroot.bb | 6 ------
 2 files changed, 3 insertions(+), 9 deletions(-)

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 84ac9c6d..6e76b04d 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -8,13 +8,13 @@
 do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 do_populate_sdk[depends] = "sdkchroot:do_build"
 do_populate_sdk() {
-    # Copy isar-apt with deployed Isar packages
-    sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO}  ${SDKCHROOT_DIR}/isar-apt
-
     sudo umount -R ${SDKCHROOT_DIR}/dev || true
     sudo umount ${SDKCHROOT_DIR}/proc || true
     sudo umount -R ${SDKCHROOT_DIR}/sys || true
 
+    # Remove isar-apt repo entry
+    sudo rm -f ${SDKCHROOT_DIR}/etc/apt/sources.list.d/isar-apt.list
+
     # Remove setup scripts
     sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh ${SDKCHROOT_DIR}/configscript.sh
 
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index 2bc9d291..dc765046 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -49,12 +49,6 @@ SDK_PREINSTALL += " \
 
 S = "${WORKDIR}/rootfs"
 
-ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir"
-rootfs_configure_isar_apt_dir() {
-    # Copy isar-apt instead of mounting:
-    sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
-}
-
 ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
 sdkchroot_install_files() {
     # Configure root filesystem
-- 
2.16.4

[PATCH v2 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This removes the need to specify --sysroot=/path/to/sdkroot when calling
gcc or ld from the SDK.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/image-sdk-extension.bbclass                 |  8 ++++++++
 .../sdkchroot/files/gcc-sysroot-wrapper.sh               | 16 ++++++++++++++++
 meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh    |  3 +++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb             |  2 ++
 4 files changed, 29 insertions(+)
 create mode 100755 meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 6e76b04d..aed8408c 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -32,6 +32,14 @@ do_populate_sdk() {
         fi
     done
 
+    # Set up sysroot wrapper
+    for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
+        for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu-${tool_pattern}"); do
+            sudo mv "${tool}" "${tool}.bin"
+            sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+        done
+    done
+
     # Copy mount_chroot.sh for convenience
     sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
 
diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
new file mode 100755
index 00000000..feead1a1
--- /dev/null
+++ b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+GCC_SYSROOT=
+
+NEXT_TARGET=$0
+until [ "${NEXT_TARGET##*/}" = "gcc-sysroot-wrapper.sh" ]; do
+	TARGET=${NEXT_TARGET}
+	NEXT_TARGET=$(dirname ${TARGET})/$(readlink ${TARGET})
+done
+
+${TARGET}.bin --sysroot=${GCC_SYSROOT} "$@"
diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
index 1c9b02fa..0d1c6330 100755
--- a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
+++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
@@ -38,4 +38,7 @@ for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/g
 	fi
 done
 
+sed -i 's|^GCC_SYSROOT=.*|GCC_SYSROOT="'"${new_sdkroot}"'"|' \
+    ${sdkroot}/usr/bin/gcc-sysroot-wrapper.sh
+
 echo "done"
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index dc765046..bf3f6fb4 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -11,6 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
 SRC_URI = " \
     file://configscript.sh \
     file://relocate-sdk.sh \
+    file://gcc-sysroot-wrapper.sh \
     file://README.sdk"
 PV = "0.1"
 
@@ -54,6 +55,7 @@ sdkchroot_install_files() {
     # Configure root filesystem
     sudo install -m 644 ${WORKDIR}/README.sdk ${S}
     sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
+    sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
     sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
     sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
 }
-- 
2.16.4

[PATCH v3 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This removes the need to specify --sysroot=/path/to/sdkroot when calling
gcc or ld from the SDK.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

Changes in v3:
 - fix ARMv7

I just had to pull this into another downstream layer today, so I 
thought it would be good to send this update and remember the whole 
series.

 meta/classes/image-sdk-extension.bbclass         |  8 ++++++++
 .../sdkchroot/files/gcc-sysroot-wrapper.sh       | 16 ++++++++++++++++
 .../sdkchroot/files/relocate-sdk.sh              |  3 +++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb     |  2 ++
 4 files changed, 29 insertions(+)
 create mode 100755 meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 6e76b04d..ebc6f342 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -32,6 +32,14 @@ do_populate_sdk() {
         fi
     done
 
+    # Set up sysroot wrapper
+    for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
+        for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
+            sudo mv "${tool}" "${tool}.bin"
+            sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+        done
+    done
+
     # Copy mount_chroot.sh for convenience
     sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
 
diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
new file mode 100755
index 00000000..feead1a1
--- /dev/null
+++ b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+GCC_SYSROOT=
+
+NEXT_TARGET=$0
+until [ "${NEXT_TARGET##*/}" = "gcc-sysroot-wrapper.sh" ]; do
+	TARGET=${NEXT_TARGET}
+	NEXT_TARGET=$(dirname ${TARGET})/$(readlink ${TARGET})
+done
+
+${TARGET}.bin --sysroot=${GCC_SYSROOT} "$@"
diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
index 1c9b02fa..0d1c6330 100755
--- a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
+++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
@@ -38,4 +38,7 @@ for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/g
 	fi
 done
 
+sed -i 's|^GCC_SYSROOT=.*|GCC_SYSROOT="'"${new_sdkroot}"'"|' \
+    ${sdkroot}/usr/bin/gcc-sysroot-wrapper.sh
+
 echo "done"
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index dc765046..bf3f6fb4 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -11,6 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
 SRC_URI = " \
     file://configscript.sh \
     file://relocate-sdk.sh \
+    file://gcc-sysroot-wrapper.sh \
     file://README.sdk"
 PV = "0.1"
 
@@ -54,6 +55,7 @@ sdkchroot_install_files() {
     # Configure root filesystem
     sudo install -m 644 ${WORKDIR}/README.sdk ${S}
     sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
+    sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
     sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
     sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
 }
-- 
2.26.2

Re: [PATCH v3 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

On 14.07.20 22:22, [ext] Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> This removes the need to specify --sysroot=/path/to/sdkroot when calling
> gcc or ld from the SDK.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> Changes in v3:
>  - fix ARMv7
> 
> I just had to pull this into another downstream layer today, so I 
> thought it would be good to send this update and remember the whole 
> series.
> 
>  meta/classes/image-sdk-extension.bbclass         |  8 ++++++++
>  .../sdkchroot/files/gcc-sysroot-wrapper.sh       | 16 ++++++++++++++++
>  .../sdkchroot/files/relocate-sdk.sh              |  3 +++
>  meta/recipes-devtools/sdkchroot/sdkchroot.bb     |  2 ++
>  4 files changed, 29 insertions(+)
>  create mode 100755 meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
> 
> diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
> index 6e76b04d..ebc6f342 100644
> --- a/meta/classes/image-sdk-extension.bbclass
> +++ b/meta/classes/image-sdk-extension.bbclass
> @@ -32,6 +32,14 @@ do_populate_sdk() {
>          fi
>      done
>  
> +    # Set up sysroot wrapper
> +    for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
> +        for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
> +            sudo mv "${tool}" "${tool}.bin"
> +            sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
> +        done
> +    done
> +
>      # Copy mount_chroot.sh for convenience
>      sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
>  
> diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
> new file mode 100755
> index 00000000..feead1a1
> --- /dev/null
> +++ b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
> @@ -0,0 +1,16 @@
> +#!/bin/sh
> +#
> +# This software is a part of ISAR.
> +# Copyright (c) Siemens AG, 2020
> +#
> +# SPDX-License-Identifier: MIT
> +
> +GCC_SYSROOT=
> +
> +NEXT_TARGET=$0
> +until [ "${NEXT_TARGET##*/}" = "gcc-sysroot-wrapper.sh" ]; do
> +	TARGET=${NEXT_TARGET}
> +	NEXT_TARGET=$(dirname ${TARGET})/$(readlink ${TARGET})
> +done
> +
> +${TARGET}.bin --sysroot=${GCC_SYSROOT} "$@"
> diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
> index 1c9b02fa..0d1c6330 100755
> --- a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
> +++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
> @@ -38,4 +38,7 @@ for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/g
>  	fi
>  done
>  
> +sed -i 's|^GCC_SYSROOT=.*|GCC_SYSROOT="'"${new_sdkroot}"'"|' \
> +    ${sdkroot}/usr/bin/gcc-sysroot-wrapper.sh
> +
>  echo "done"
> diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> index dc765046..bf3f6fb4 100644
> --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> @@ -11,6 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
>  SRC_URI = " \
>      file://configscript.sh \
>      file://relocate-sdk.sh \
> +    file://gcc-sysroot-wrapper.sh \
>      file://README.sdk"
>  PV = "0.1"
>  
> @@ -54,6 +55,7 @@ sdkchroot_install_files() {
>      # Configure root filesystem
>      sudo install -m 644 ${WORKDIR}/README.sdk ${S}
>      sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
> +    sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
>      sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
>      sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
>  }
> 

Ping³ on the whole series. I carrying too many patches downstream for
too long.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v3 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

On 31.08.20 18:32, [ext] Jan Kiszka wrote:
> On 14.07.20 22:22, [ext] Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> This removes the need to specify --sysroot=/path/to/sdkroot when calling
>> gcc or ld from the SDK.
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>>
>> Changes in v3:
>>   - fix ARMv7
>>
>> I just had to pull this into another downstream layer today, so I
>> thought it would be good to send this update and remember the whole
>> series.
>>
>>   meta/classes/image-sdk-extension.bbclass         |  8 ++++++++
>>   .../sdkchroot/files/gcc-sysroot-wrapper.sh       | 16 ++++++++++++++++
>>   .../sdkchroot/files/relocate-sdk.sh              |  3 +++
>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb     |  2 ++
>>   4 files changed, 29 insertions(+)
>>   create mode 100755 meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
>>
>> diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
>> index 6e76b04d..ebc6f342 100644
>> --- a/meta/classes/image-sdk-extension.bbclass
>> +++ b/meta/classes/image-sdk-extension.bbclass
>> @@ -32,6 +32,14 @@ do_populate_sdk() {
>>           fi
>>       done
>>   
>> +    # Set up sysroot wrapper
>> +    for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
>> +        for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
>> +            sudo mv "${tool}" "${tool}.bin"
>> +            sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
>> +        done
>> +    done
>> +
>>       # Copy mount_chroot.sh for convenience
>>       sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
>>   
>> diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
>> new file mode 100755
>> index 00000000..feead1a1
>> --- /dev/null
>> +++ b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
>> @@ -0,0 +1,16 @@
>> +#!/bin/sh
>> +#
>> +# This software is a part of ISAR.
>> +# Copyright (c) Siemens AG, 2020
>> +#
>> +# SPDX-License-Identifier: MIT
>> +
>> +GCC_SYSROOT=
>> +
>> +NEXT_TARGET=$0
>> +until [ "${NEXT_TARGET##*/}" = "gcc-sysroot-wrapper.sh" ]; do
>> +	TARGET=${NEXT_TARGET}
>> +	NEXT_TARGET=$(dirname ${TARGET})/$(readlink ${TARGET})
>> +done
>> +
>> +${TARGET}.bin --sysroot=${GCC_SYSROOT} "$@"
>> diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
>> index 1c9b02fa..0d1c6330 100755
>> --- a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
>> +++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
>> @@ -38,4 +38,7 @@ for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/g
>>   	fi
>>   done
>>   
>> +sed -i 's|^GCC_SYSROOT=.*|GCC_SYSROOT="'"${new_sdkroot}"'"|' \
>> +    ${sdkroot}/usr/bin/gcc-sysroot-wrapper.sh
>> +
>>   echo "done"
>> diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
>> index dc765046..bf3f6fb4 100644
>> --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
>> +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
>> @@ -11,6 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
>>   SRC_URI = " \
>>       file://configscript.sh \
>>       file://relocate-sdk.sh \
>> +    file://gcc-sysroot-wrapper.sh \
>>       file://README.sdk"
>>   PV = "0.1"
>>   
>> @@ -54,6 +55,7 @@ sdkchroot_install_files() {
>>       # Configure root filesystem
>>       sudo install -m 644 ${WORKDIR}/README.sdk ${S}
>>       sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
>> +    sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
>>       sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
>>       sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
>>   }
>>
> 
> Ping³ on the whole series. I carrying too many patches downstream for
> too long.
> 

Still pending. This is part of several product layers by now, and I only 
have positive feedback on the usability of the generated SDK so far. Can 
we move forward?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v3 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Baurzhan Ismagulov]

On Mon, Sep 21, 2020 at 11:27:32AM +0200, Jan Kiszka wrote:
> Still pending. This is part of several product layers by now, and I only
> have positive feedback on the usability of the generated SDK so far. Can we
> move forward?

Here we have to agree on the chroot approach and the way of shipping isar-apt.

Regarding dropping chroot support: While I welcome the Yocto-like SDK which is
familiar for the Yocto developers, chroot is currently the official Debian way
of building [1]. With Debian implementing multiarch despite being alone in it
and many Debian developers actively disliking non-Debian building, we can't
truly bridge the two by dismissing the Debian way of building in Isar. Both
approaches clearly have their respective advantages and followers, and I think
we should keep both, for developers and production. Declaring the one way as
preferred doesn't help making friends in the other camp, either.

Regarding dropping isar-apt: Debian is about tools that facilitate different
workflows without imposing One True Way. The strength of shipping isar-apt is
in the developer's ability to install packages afterwards that maybe were not
necessary in the beginning or are not interesting for the most developers.
Taken from this perspective, requiring an SDK rebuild for a new package is
waste of time, especially if different groups are responsible for that and if
further packages are going to be needed shortly afterwards.

So, I'd like to see how we address these issues with this SDK improvement. If
there are specific problems (e.g., bloat), maybe we could make stuff
configurable, etc., but not break the existing use case outright.

1. https://wiki.debian.org/sbuild

With kind regards,
Baurzhan.

Re: [PATCH v3 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

On 21.09.20 12:05, Baurzhan Ismagulov wrote:
> On Mon, Sep 21, 2020 at 11:27:32AM +0200, Jan Kiszka wrote:
>> Still pending. This is part of several product layers by now, and I only
>> have positive feedback on the usability of the generated SDK so far. Can we
>> move forward?
> 
> Here we have to agree on the chroot approach and the way of shipping isar-apt.
> 
> Regarding dropping chroot support: While I welcome the Yocto-like SDK which is
> familiar for the Yocto developers, chroot is currently the official Debian way
> of building [1]. With Debian implementing multiarch despite being alone in it
> and many Debian developers actively disliking non-Debian building, we can't
> truly bridge the two by dismissing the Debian way of building in Isar. Both
> approaches clearly have their respective advantages and followers, and I think
> we should keep both, for developers and production. Declaring the one way as
> preferred doesn't help making friends in the other camp, either.

We are not dropping the chroot model at this stage. We can keep it 
longer, but I'm afraid it may eventually bitrot as it remains a niche 
scenario. The true debian way is pushing all results and the a debian 
snapshot into own repos that people who like to build inside debian can 
pull from. An SDK as we generate it does not serve the expert scenario, 
and building inside a chroot is surely not for beginners due to all the 
complication with the outer world.

> 
> Regarding dropping isar-apt: Debian is about tools that facilitate different
> workflows without imposing One True Way. The strength of shipping isar-apt is
> in the developer's ability to install packages afterwards that maybe were not
> necessary in the beginning or are not interesting for the most developers.
> Taken from this perspective, requiring an SDK rebuild for a new package is
> waste of time, especially if different groups are responsible for that and if
> further packages are going to be needed shortly afterwards.
> 
> So, I'd like to see how we address these issues with this SDK improvement. If
> there are specific problems (e.g., bloat), maybe we could make stuff
> configurable, etc., but not break the existing use case outright.

I can keep isar-apt opt-in for the time being, expect v3 soon.

But my vision remains SDK_INSTALL for this purpose (including chroot), 
in the future maybe more automated. Blinding shipping the complete 
isar-apt is, well, a very simplistic approach to the problem.

Jan

> 
> 1. https://wiki.debian.org/sbuild
> 
> With kind regards,
> Baurzhan.
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[PATCH v2 6/6] sdk: Update README.sdk

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Document the out-of-chroot invocation and make it the preferred option.
Also remove irrelevant information.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/recipes-devtools/sdkchroot/files/README.sdk | 71 ++++++++++++------------
 1 file changed, 36 insertions(+), 35 deletions(-)

diff --git a/meta/recipes-devtools/sdkchroot/files/README.sdk b/meta/recipes-devtools/sdkchroot/files/README.sdk
index 9c1af6d3..3e06d8c5 100644
--- a/meta/recipes-devtools/sdkchroot/files/README.sdk
+++ b/meta/recipes-devtools/sdkchroot/files/README.sdk
@@ -1,42 +1,43 @@
-Building applications for targets in ISAR takes a lot of time as they are built under QEMU.
-SDK helps to develop applications for target platform in crossbuild environment.
+ISAR Target SDK
+===============
 
-SDK contains cross-toolchain for target architecture and a copy of isar-apt repo with
-locally prebuilt target debian packages.
+This SDK helps to develop applications for an ISAR target platform in a
+crossbuild environment. It contains a cross-toolchain and development packages
+corresponding to the original target.
 
- - First one have to mount the system directories for proper operation in chroot environement.
-Just call supplied with sdk tarball  in  udo rootfs as an argument to the script `mount_chroot.sh`:
+The SDK can be used in two ways, described in the following.
 
-$ sudo mount_chroot.sh  <sdk_rootfs>
 
- - chroot to isar SDK rootfs:
+Option 1 (recommended): Use cross-compiler in host environment
+--------------------------------------------------------------
+
+After unpacking the SDK at the desired location, it has to be relocated once:
+
+$ <sdk_rootfs>/relocate-sdk.sh
+
+Now you can add <sdk_rootfs>/usr/bin to the local path or adjust your project
+to call the cross-compiler from the SDK.
+
+
+Option 2 (fallback): Build inside chroot
+----------------------------------------
+
+First you have to mount the system directories for proper operation into the
+chroot environment. Call the helper script supplied with SDK tarball:
+
+$ sudo <sdk_rootfs>/mount_chroot.sh <sdk_rootfs>
+
+Bind-mount the project into the rootfs:
+
+$ sudo mount -o bind /path/to/project <sdk_rootfs>/mnt
+
+If you have relocated the SDK previously for using option 1, you need to call
+this next:
+
+$ <sdk_rootfs>/relocate-sdk.sh --restore-chroot
+
+Then chroot into the SDK rootfs:
 
 $ sudo chroot <sdk_rootfs>
 
- - Check that cross toolchains are installed
-
-:~# dpkg -l | grep crossbuild-essential-armhf
-ii  crossbuild-essential-armhf           12.3                   all          Informational list of cross-build-essential packages
-
- - Install needed prebuilt target packages.
-
-:~# apt-get update
-:~# apt-get install libhello-dev:armhf
-
- - Check the contents of the installed target package
-
-:~# dpkg -L libhello-dev
-/.
-/usr
-/usr/include
-/usr/include/hello.h
-/usr/lib
-/usr/lib/arm-linux-gnueabihf
-/usr/lib/arm-linux-gnueabihf/libhello.a
-/usr/lib/arm-linux-gnueabihf/libhello.la
-/usr/share
-/usr/share/doc
-/usr/share/doc/libhello-dev
-/usr/share/doc/libhello-dev/changelog.gz
-/usr/share/doc/libhello-dev/copyright
-~#
+Now you can build the project under /mnt.
-- 
2.16.4

[PATCH 0/6] Evolve SDK to chroot-free usage

[Jan Kiszka]

Currently, our SDK can primarily be used by chroot'ing into it, taking
the to-be-built project with you (bind mount etc.). This was enough for
a start but we can do better.

This series enhance the SDK to become usage as a normal cross-toolchain
on your host system. For that purpose, we add a script that relocates
the toolchain after installation, adjust binary search paths and
sysroot. So, the only remaining difference to a yocto or buildroot
toolchain is the need to run the relocation script once after unpacking
the SDK.

The series furthermore allows to customize the SDK content via the
image-generating layer.

The series (together with all other pending patching of mine) can also
be found at https://github.com/siemens/isar/commits/jan/queue.

While I already shrunk the SDK a bit by removing the now obsolete
isar-apt repo, I suspect we could further reduce the deployment size by
dropping chroot support completely, only including the compiler and
their library dependencies. OTOH, sdk-debian-buster-arm64.tar.xz is now
142M here, unpacked 680M, while ARM's
gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz has
260M packed and 1.5G(!) unpacked. So we are either not really off or are
still missing something.

Jan

Jan Kiszka (6):
  sdk: Add support for adding self-defined sdk packages
  sdk: Make all links in the SDK chroot relative
  sdk: Add script to relocate SDK
  sdk: Do not ship the isar-apt repo
  sdk: Inject sysroot path when calling relocated toolchain
  sdk: Update README.sdk

 doc/user_manual.md                                 |  1 +
 meta/classes/image-sdk-extension.bbclass           | 28 ++++++++-
 meta/recipes-devtools/sdkchroot/files/README.sdk   | 66 ++++++++++------------
 .../sdkchroot/files/relocate-sdk.sh                | 32 +++++++++++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb       | 35 ++++++------
 5 files changed, 108 insertions(+), 54 deletions(-)
 create mode 100755 meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh

-- 
2.16.4

[PATCH 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This removes the need to specify --sysroot=/path/to/sdkroot when calling
gcc or ld from the SDK.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/image-sdk-extension.bbclass              | 8 ++++++++
 meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh | 3 +++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb          | 2 ++
 3 files changed, 13 insertions(+)

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 6e76b04d..aed8408c 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -32,6 +32,14 @@ do_populate_sdk() {
         fi
     done
 
+    # Set up sysroot wrapper
+    for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
+        for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu-${tool_pattern}"); do
+            sudo mv "${tool}" "${tool}.bin"
+            sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+        done
+    done
+
     # Copy mount_chroot.sh for convenience
     sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
 
diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
index 98827139..6be8d82f 100755
--- a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
+++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
@@ -26,4 +26,7 @@ for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/g
 	fi
 done
 
+sed -i 's|^GCC_SYSROOT=.*|GCC_SYSROOT="'"${sdkroot}"'"|' \
+    ${sdkroot}/usr/bin/gcc-sysroot-wrapper.sh
+
 echo "done"
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index dc765046..bf3f6fb4 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -11,6 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
 SRC_URI = " \
     file://configscript.sh \
     file://relocate-sdk.sh \
+    file://gcc-sysroot-wrapper.sh \
     file://README.sdk"
 PV = "0.1"
 
@@ -54,6 +55,7 @@ sdkchroot_install_files() {
     # Configure root filesystem
     sudo install -m 644 ${WORKDIR}/README.sdk ${S}
     sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
+    sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
     sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
     sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
 }
-- 
2.16.4

[PATCH v2 5/6] sdk: Inject sysroot path when calling relocated toolchain

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This removes the need to specify --sysroot=/path/to/sdkroot when calling
gcc or ld from the SDK.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

Changes in v2:
 - include forgotten gcc-sysroot-wrapper.sh

 meta/classes/image-sdk-extension.bbclass                 |  8 ++++++++
 .../sdkchroot/files/gcc-sysroot-wrapper.sh               | 16 ++++++++++++++++
 meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh    |  3 +++
 meta/recipes-devtools/sdkchroot/sdkchroot.bb             |  2 ++
 4 files changed, 29 insertions(+)
 create mode 100755 meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 6e76b04d..aed8408c 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -32,6 +32,14 @@ do_populate_sdk() {
         fi
     done
 
+    # Set up sysroot wrapper
+    for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
+        for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu-${tool_pattern}"); do
+            sudo mv "${tool}" "${tool}.bin"
+            sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+        done
+    done
+
     # Copy mount_chroot.sh for convenience
     sudo cp ${ISARROOT}/scripts/mount_chroot.sh ${SDKCHROOT_DIR}
 
diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
new file mode 100755
index 00000000..feead1a1
--- /dev/null
+++ b/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+GCC_SYSROOT=
+
+NEXT_TARGET=$0
+until [ "${NEXT_TARGET##*/}" = "gcc-sysroot-wrapper.sh" ]; do
+	TARGET=${NEXT_TARGET}
+	NEXT_TARGET=$(dirname ${TARGET})/$(readlink ${TARGET})
+done
+
+${TARGET}.bin --sysroot=${GCC_SYSROOT} "$@"
diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
index 98827139..6be8d82f 100755
--- a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
+++ b/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
@@ -26,4 +26,7 @@ for binary in $(find ${sdkroot}/usr/bin ${sdkroot}/usr/sbin ${sdkroot}/usr/lib/g
 	fi
 done
 
+sed -i 's|^GCC_SYSROOT=.*|GCC_SYSROOT="'"${sdkroot}"'"|' \
+    ${sdkroot}/usr/bin/gcc-sysroot-wrapper.sh
+
 echo "done"
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index dc765046..bf3f6fb4 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -11,6 +11,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
 SRC_URI = " \
     file://configscript.sh \
     file://relocate-sdk.sh \
+    file://gcc-sysroot-wrapper.sh \
     file://README.sdk"
 PV = "0.1"
 
@@ -54,6 +55,7 @@ sdkchroot_install_files() {
     # Configure root filesystem
     sudo install -m 644 ${WORKDIR}/README.sdk ${S}
     sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
+    sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
     sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
     sudo chroot ${S} /configscript.sh  ${DISTRO_ARCH}
 }
-- 
2.16.4