[PATCH v2 0/2] ISAR SDK image support

[PATCH v2 0/2] ISAR SDK image support

[Maxim Yu. Osipov]

Hello everybody,

Please ignore previous patch series.

This series introduces ISAR SDK image support.

In order to create ISAR SDK image just add package isar-sdk into 
variable IMAGE_INSTALL in bitbake's conf/local.conf and fireup the ISAR image build.

When build is done, perform the following steps:

1) Copy host's hostname and resolv.conf into resulting rootfs:

For example,

sudo cp /etc/hostname  $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
sudo cp /etc/resolv.conf  $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc

2) Mount the following directories in chroot by passing resulting rootfs as an argument
to the script mount_chroot.sh:

$ cat mount_chroot.sh 
#!/bin/bash
sudo mount /tmp     $1/tmp                 -o bind
sudo mount proc     $1/proc    -t proc     -o nosuid,noexec,nodev
sudo mount sysfs    $1/sys     -t sysfs    -o nosuid,noexec,nodev
sudo mount devtmpfs $1/dev     -t devtmpfs -o mode=0755,nosuid
sudo mount devpts   $1/dev/pts -t devpts   -o gid=5,mode=620
sudo mount tmpfs    $1/dev/shm -t tmpfs    -o rw,seclabel,nosuid,nodev

$./mount_chroot.sh $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs

3) chroot to isar SDK rootfs:

$sudo chroot $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs

4) In chroot change user to 'builder', cd to directory isar and start the isar image generation as usual.

For example,

root@tiberius:/# su - builder
builder@tiberius:~$ cd isar/
builder@tiberius:~/isar$ . isar-init-build-env ../build/
builder@tiberius:~/build$ bitbake multiconfig:qemuamd64-stretch:isar-image-base


Kind regards,
Maxim.

Alexander Smirnov (1):
  isar-bitbake: Add AUTOREV definition

Maxim Yu. Osipov (1):
  image: Add ISAR SDK package to create SDK from any rootfs image.

 meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
 meta/conf/isar-bitbake.conf                     |  2 ++
 meta/recipes-devtools/isar-sdk/files/README     |  2 ++
 meta/recipes-devtools/isar-sdk/files/postinst   | 21 +++++++++++++
 meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42 +++++++++++++++++++++++++
 5 files changed, 76 insertions(+)
 create mode 100644 meta-isar/recipes-core/images/isar-image-sdk.bb
 create mode 100644 meta/recipes-devtools/isar-sdk/files/README
 create mode 100644 meta/recipes-devtools/isar-sdk/files/postinst
 create mode 100644 meta/recipes-devtools/isar-sdk/isar-sdk.bb

-- 
2.11.0

[PATCH v2 1/2] isar-bitbake: Add AUTOREV definition

[Maxim Yu. Osipov]

From: Alexander Smirnov <asmirnov@ilbers.de>

Define AUTOREV variable to automatically fetch latest repository head.
Derived from Yocto.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta/conf/isar-bitbake.conf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index 8a1d86b..72d1ee0 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -44,6 +44,8 @@ BB_NUMBER_THREADS ?= "${@bb.utils.cpu_count()}"
 
 BBINCLUDELOGS ??= "yes"
 
+AUTOREV ?= "${@bb.fetch2.get_autorev(d)}"
+
 # Add event handlers for bitbake
 INHERIT += "isar-events"
 
-- 
2.11.0

[PATCH v2 2/2] image: Add ISAR SDK package to create SDK from any rootfs image.

[Maxim Yu. Osipov]

Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
 meta/recipes-devtools/isar-sdk/files/README     |  2 ++
 meta/recipes-devtools/isar-sdk/files/postinst   | 21 +++++++++++++
 meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42 +++++++++++++++++++++++++
 4 files changed, 74 insertions(+)
 create mode 100644 meta-isar/recipes-core/images/isar-image-sdk.bb
 create mode 100644 meta/recipes-devtools/isar-sdk/files/README
 create mode 100644 meta/recipes-devtools/isar-sdk/files/postinst
 create mode 100644 meta/recipes-devtools/isar-sdk/isar-sdk.bb

diff --git a/meta-isar/recipes-core/images/isar-image-sdk.bb b/meta-isar/recipes-core/images/isar-image-sdk.bb
new file mode 100644
index 0000000..5128a03
--- /dev/null
+++ b/meta-isar/recipes-core/images/isar-image-sdk.bb
@@ -0,0 +1,9 @@
+# ISAR SDK based on isar-image-base
+#
+# This software is a part of ISAR.
+# Copyright (C) 2018 ilbers GmbH
+
+include recipes-core/images/isar-image-base.bb
+
+IMAGE_INSTALL += "isar-sdk"
+
diff --git a/meta/recipes-devtools/isar-sdk/files/README b/meta/recipes-devtools/isar-sdk/files/README
new file mode 100644
index 0000000..3b80848
--- /dev/null
+++ b/meta/recipes-devtools/isar-sdk/files/README
@@ -0,0 +1,2 @@
+Include isar-sdk into IMAGE_INSTALL of image recipe to create ISAR SDK
+based on this image.
diff --git a/meta/recipes-devtools/isar-sdk/files/postinst b/meta/recipes-devtools/isar-sdk/files/postinst
new file mode 100644
index 0000000..2c6e6e0
--- /dev/null
+++ b/meta/recipes-devtools/isar-sdk/files/postinst
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+set -e
+
+readonly USER=builder
+
+adduser --disabled-password --gecos "ISAR Builder" $USER
+# Empty user password
+passwd -d $USER
+
+echo "export LC_ALL=C.UTF-8" >> /home/$USER/.profile
+
+cat >> /etc/sudoers << EOF
+$USER	ALL=NOPASSWD: ALL
+%$USER	ALL=NOPASSWD: ALL
+EOF
+
+#Check for errors
+visudo -c
+
+chown -R $USER:$USER /home/$USER/isar /home/$USER/build
diff --git a/meta/recipes-devtools/isar-sdk/isar-sdk.bb b/meta/recipes-devtools/isar-sdk/isar-sdk.bb
new file mode 100644
index 0000000..4a3176a
--- /dev/null
+++ b/meta/recipes-devtools/isar-sdk/isar-sdk.bb
@@ -0,0 +1,42 @@
+# This software is a part of ISAR.
+# Copyright (C) 2018 ilbers GmbH
+
+DESCRIPTION = "Package to be installed to create ISAR SDK target filesystem"
+MAINTAINER = ""
+DEBIAN_DEPENDS = "apt (>= 0.4.2), \
+                  passwd, \
+                  patch, \
+                  ca-certificates, \
+                  dbus, \
+                  dosfstools, \
+                  git,\
+                  debootstrap, \
+                  localepurge, \
+                  parted, \
+                  python3, \
+                  rxvt-unicode, \
+                  qemu, \
+                  qemu-user-static, \
+                  binfmt-support, \
+                  sudo, \
+                  reprepro"
+
+SRC_URI = "git://github.com/ilbers/isar.git;protocol=https"
+SRCREV = "${AUTOREV}"
+
+SRC_URI += "file://README \
+	   file://postinst"
+
+inherit dpkg-raw
+
+do_install() {
+
+	bbnote "Installing ISAR tree"
+	install -v -d ${D}/home/builder/isar
+	git clone http://github.com/ilbers/isar.git ${D}/home/builder/isar
+
+	install -v -d ${D}/home/builder/build
+
+	bbnote "Copying ${DL_DIR} directory"
+	cp -a ${DL_DIR} ${D}/home/builder/build/
+}
-- 
2.11.0

Re: [PATCH v2 0/2] ISAR SDK image support

[Baurzhan Ismagulov]

Hello Maxim,

On Wed, May 16, 2018 at 05:56:51PM +0200, Maxim Yu. Osipov wrote:
> Please ignore previous patch series.

Your mails landed in spam, so I've deleted the previous mails to save people's
time.


> This series introduces ISAR SDK image support.

Thanks. Could you please document this in the manual?


With this implementation, one can qemu or chroot into the SDK and build
artifacts (e.g., isar-image-base) in the native (e.g., armhf) buildchroot. The
workflow looks like this:

1. System integrator prepares his build host, builds the system image and the
   SDK, and distributes them.

2. Developers flash the image to the target and use SDK to develop the
   application, and either:

   1. Build new images and flash them, or

   2. Build hello.deb and install it to the target manually.

In this way, the current SDK:

1. Removes the necessity to prepare developer's build hosts.

2. Is usable on non-Debian hosts.

3. Provides downloads.


@Jan: Do I understand your idea correctly: The SDK should contain a
cross-compiler for the developers to build their applications? Or is this
thought as a second step, when Isar is able to build packages in
crossbuildchroot (or however we call it)?


With kind regards,
Baurzhan.

Re: [PATCH v2 2/2] image: Add ISAR SDK package to create SDK from any rootfs image.

[Henning Schild]

Am Wed, 16 May 2018 17:56:53 +0200
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
> ---
>  meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
>  meta/recipes-devtools/isar-sdk/files/README     |  2 ++
>  meta/recipes-devtools/isar-sdk/files/postinst   | 21 +++++++++++++
>  meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42
> +++++++++++++++++++++++++ 4 files changed, 74 insertions(+)
>  create mode 100644 meta-isar/recipes-core/images/isar-image-sdk.bb
>  create mode 100644 meta/recipes-devtools/isar-sdk/files/README
>  create mode 100644 meta/recipes-devtools/isar-sdk/files/postinst
>  create mode 100644 meta/recipes-devtools/isar-sdk/isar-sdk.bb
> 
> diff --git a/meta-isar/recipes-core/images/isar-image-sdk.bb
> b/meta-isar/recipes-core/images/isar-image-sdk.bb new file mode 100644
> index 0000000..5128a03
> --- /dev/null
> +++ b/meta-isar/recipes-core/images/isar-image-sdk.bb
> @@ -0,0 +1,9 @@
> +# ISAR SDK based on isar-image-base
> +#
> +# This software is a part of ISAR.
> +# Copyright (C) 2018 ilbers GmbH
> +
> +include recipes-core/images/isar-image-base.bb
> +
> +IMAGE_INSTALL += "isar-sdk"
> +
> diff --git a/meta/recipes-devtools/isar-sdk/files/README
> b/meta/recipes-devtools/isar-sdk/files/README new file mode 100644
> index 0000000..3b80848
> --- /dev/null
> +++ b/meta/recipes-devtools/isar-sdk/files/README
> @@ -0,0 +1,2 @@
> +Include isar-sdk into IMAGE_INSTALL of image recipe to create ISAR
> SDK +based on this image.
> diff --git a/meta/recipes-devtools/isar-sdk/files/postinst
> b/meta/recipes-devtools/isar-sdk/files/postinst new file mode 100644
> index 0000000..2c6e6e0
> --- /dev/null
> +++ b/meta/recipes-devtools/isar-sdk/files/postinst
> @@ -0,0 +1,21 @@
> +#!/bin/sh
> +
> +set -e
> +
> +readonly USER=builder
> +
> +adduser --disabled-password --gecos "ISAR Builder" $USER
> +# Empty user password
> +passwd -d $USER
> +
> +echo "export LC_ALL=C.UTF-8" >> /home/$USER/.profile
> +
> +cat >> /etc/sudoers << EOF
> +$USER	ALL=NOPASSWD: ALL
> +%$USER	ALL=NOPASSWD: ALL
> +EOF

Use /etc/sudoers.d/ instead.

> +#Check for errors
> +visudo -c
> +
> +chown -R $USER:$USER /home/$USER/isar /home/$USER/build
> diff --git a/meta/recipes-devtools/isar-sdk/isar-sdk.bb
> b/meta/recipes-devtools/isar-sdk/isar-sdk.bb new file mode 100644
> index 0000000..4a3176a
> --- /dev/null
> +++ b/meta/recipes-devtools/isar-sdk/isar-sdk.bb
> @@ -0,0 +1,42 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2018 ilbers GmbH
> +
> +DESCRIPTION = "Package to be installed to create ISAR SDK target
> filesystem" +MAINTAINER = ""
> +DEBIAN_DEPENDS = "apt (>= 0.4.2), \
> +                  passwd, \
> +                  patch, \
> +                  ca-certificates, \
> +                  dbus, \
> +                  dosfstools, \
> +                  git,\
> +                  debootstrap, \
> +                  localepurge, \
> +                  parted, \
> +                  python3, \
> +                  rxvt-unicode, \
> +                  qemu, \
> +                  qemu-user-static, \
> +                  binfmt-support, \
> +                  sudo, \
> +                  reprepro"
> +
> +SRC_URI = "git://github.com/ilbers/isar.git;protocol=https"
> +SRCREV = "${AUTOREV}"
> +
> +SRC_URI += "file://README \
> +	   file://postinst"
> +
> +inherit dpkg-raw
> +
> +do_install() {
> +
> +	bbnote "Installing ISAR tree"
> +	install -v -d ${D}/home/builder/isar
> +	git clone http://github.com/ilbers/isar.git
> ${D}/home/builder/isar +
> +	install -v -d ${D}/home/builder/build
> +
> +	bbnote "Copying ${DL_DIR} directory"
> +	cp -a ${DL_DIR} ${D}/home/builder/build/
> +}

Why do we need Isar in there? And if we need it you should get it via
SRC_URI and not fetch again. What you do here will potentially not work
behind a proxy.

Henning

Re: [PATCH v2 2/2] image: Add ISAR SDK package to create SDK from any rootfs image.

[Henning Schild]

Am Wed, 16 May 2018 17:56:53 +0200
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
> ---
>  meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
>  meta/recipes-devtools/isar-sdk/files/README     |  2 ++
>  meta/recipes-devtools/isar-sdk/files/postinst   | 21 +++++++++++++
>  meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42
> +++++++++++++++++++++++++ 4 files changed, 74 insertions(+)
>  create mode 100644 meta-isar/recipes-core/images/isar-image-sdk.bb
>  create mode 100644 meta/recipes-devtools/isar-sdk/files/README
>  create mode 100644 meta/recipes-devtools/isar-sdk/files/postinst
>  create mode 100644 meta/recipes-devtools/isar-sdk/isar-sdk.bb
> 
> diff --git a/meta-isar/recipes-core/images/isar-image-sdk.bb
> b/meta-isar/recipes-core/images/isar-image-sdk.bb new file mode 100644
> index 0000000..5128a03
> --- /dev/null
> +++ b/meta-isar/recipes-core/images/isar-image-sdk.bb
> @@ -0,0 +1,9 @@
> +# ISAR SDK based on isar-image-base
> +#
> +# This software is a part of ISAR.
> +# Copyright (C) 2018 ilbers GmbH
> +
> +include recipes-core/images/isar-image-base.bb
> +
> +IMAGE_INSTALL += "isar-sdk"
> +
> diff --git a/meta/recipes-devtools/isar-sdk/files/README
> b/meta/recipes-devtools/isar-sdk/files/README new file mode 100644
> index 0000000..3b80848
> --- /dev/null
> +++ b/meta/recipes-devtools/isar-sdk/files/README
> @@ -0,0 +1,2 @@
> +Include isar-sdk into IMAGE_INSTALL of image recipe to create ISAR
> SDK +based on this image.
> diff --git a/meta/recipes-devtools/isar-sdk/files/postinst
> b/meta/recipes-devtools/isar-sdk/files/postinst new file mode 100644
> index 0000000..2c6e6e0
> --- /dev/null
> +++ b/meta/recipes-devtools/isar-sdk/files/postinst
> @@ -0,0 +1,21 @@
> +#!/bin/sh
> +
> +set -e
> +
> +readonly USER=builder
> +
> +adduser --disabled-password --gecos "ISAR Builder" $USER
> +# Empty user password
> +passwd -d $USER
> +
> +echo "export LC_ALL=C.UTF-8" >> /home/$USER/.profile
> +
> +cat >> /etc/sudoers << EOF
> +$USER	ALL=NOPASSWD: ALL
> +%$USER	ALL=NOPASSWD: ALL
> +EOF
> +
> +#Check for errors
> +visudo -c
> +
> +chown -R $USER:$USER /home/$USER/isar /home/$USER/build
> diff --git a/meta/recipes-devtools/isar-sdk/isar-sdk.bb
> b/meta/recipes-devtools/isar-sdk/isar-sdk.bb new file mode 100644
> index 0000000..4a3176a
> --- /dev/null
> +++ b/meta/recipes-devtools/isar-sdk/isar-sdk.bb
> @@ -0,0 +1,42 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2018 ilbers GmbH
> +
> +DESCRIPTION = "Package to be installed to create ISAR SDK target
> filesystem" +MAINTAINER = ""
> +DEBIAN_DEPENDS = "apt (>= 0.4.2), \
> +                  passwd, \
> +                  patch, \
> +                  ca-certificates, \
> +                  dbus, \
> +                  dosfstools, \
> +                  git,\
> +                  debootstrap, \
> +                  localepurge, \
> +                  parted, \
> +                  python3, \
> +                  rxvt-unicode, \
> +                  qemu, \
> +                  qemu-user-static, \

This can not work. qemu-user-static is a package that can/must not be
installed in an Isar rootfs. The moment you install it you break the
binfmt chroot mechanism you need to chroot into the rootfs.
The kernel will report a "symlink loop" after you have replaced the
alien qemu that was smuggeled into the rootfs by debootstrap.

Henning

> +                  binfmt-support, \
> +                  sudo, \
> +                  reprepro"
> +
> +SRC_URI = "git://github.com/ilbers/isar.git;protocol=https"
> +SRCREV = "${AUTOREV}"
> +
> +SRC_URI += "file://README \
> +	   file://postinst"
> +
> +inherit dpkg-raw
> +
> +do_install() {
> +
> +	bbnote "Installing ISAR tree"
> +	install -v -d ${D}/home/builder/isar
> +	git clone http://github.com/ilbers/isar.git
> ${D}/home/builder/isar +
> +	install -v -d ${D}/home/builder/build
> +
> +	bbnote "Copying ${DL_DIR} directory"
> +	cp -a ${DL_DIR} ${D}/home/builder/build/
> +}

Re: [PATCH v2 0/2] ISAR SDK image support

[Henning Schild]

Am Wed, 16 May 2018 17:56:51 +0200
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Hello everybody,
> 
> Please ignore previous patch series.
> 
> This series introduces ISAR SDK image support.

Thanks for looking into that!

As i explained here
https://groups.google.com/forum/#!topic/isar-users/bxmZQD8sSDQ

There are two general ways to prepare an SDK. The image+devtools and
the "--sysroot" of buildchroot way. You have implemented image+devtools
which is only practical for non qemu-user-static setups ... basically
just for x86 targets.

While it will work for arm(64) on x86 you will end up with a horribly
slow SDK. Horribly slow is OK for building the image with Isar, not for
the fast turn-around you are looking for as a developer.

What should be done instead is extracting the buildchroot and using it
from the outside with "--sysroot", add example for make, CMake, eclipse
And maybe creating an x86 Debian rootfs with the cross compiler that
contains all that.

Henning

> In order to create ISAR SDK image just add package isar-sdk into 
> variable IMAGE_INSTALL in bitbake's conf/local.conf and fireup the
> ISAR image build.
> 
> When build is done, perform the following steps:
> 
> 1) Copy host's hostname and resolv.conf into resulting rootfs:
> 
> For example,
> 
> sudo cp /etc/hostname
> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
> sudo cp /etc/resolv.conf
> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
> 
> 2) Mount the following directories in chroot by passing resulting
> rootfs as an argument to the script mount_chroot.sh:
> 
> $ cat mount_chroot.sh 
> #!/bin/bash
> sudo mount /tmp     $1/tmp                 -o bind
> sudo mount proc     $1/proc    -t proc     -o nosuid,noexec,nodev
> sudo mount sysfs    $1/sys     -t sysfs    -o nosuid,noexec,nodev
> sudo mount devtmpfs $1/dev     -t devtmpfs -o mode=0755,nosuid
> sudo mount devpts   $1/dev/pts -t devpts   -o gid=5,mode=620
> sudo mount tmpfs    $1/dev/shm -t tmpfs    -o rw,seclabel,nosuid,nodev
> 
> $./mount_chroot.sh
> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs

Should that helper not be part of the patch series?

> 3) chroot to isar SDK rootfs:
> 
> $sudo chroot
> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs
> 
> 4) In chroot change user to 'builder', cd to directory isar and start
> the isar image generation as usual.
> 
> For example,
> 
> root@tiberius:/# su - builder
> builder@tiberius:~$ cd isar/
> builder@tiberius:~/isar$ . isar-init-build-env ../build/
> builder@tiberius:~/build$ bitbake
> multiconfig:qemuamd64-stretch:isar-image-base
> 
> 
> Kind regards,
> Maxim.
> 
> Alexander Smirnov (1):
>   isar-bitbake: Add AUTOREV definition
> 
> Maxim Yu. Osipov (1):
>   image: Add ISAR SDK package to create SDK from any rootfs image.
> 
>  meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
>  meta/conf/isar-bitbake.conf                     |  2 ++
>  meta/recipes-devtools/isar-sdk/files/README     |  2 ++
>  meta/recipes-devtools/isar-sdk/files/postinst   | 21 +++++++++++++
>  meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42
> +++++++++++++++++++++++++ 5 files changed, 76 insertions(+)
>  create mode 100644 meta-isar/recipes-core/images/isar-image-sdk.bb
>  create mode 100644 meta/recipes-devtools/isar-sdk/files/README
>  create mode 100644 meta/recipes-devtools/isar-sdk/files/postinst
>  create mode 100644 meta/recipes-devtools/isar-sdk/isar-sdk.bb
> 

Re: [PATCH v2 0/2] ISAR SDK image support

[Maxim Yu. Osipov]

Hello Henning,

Thank you very much for your feedback.
I'm new in the project and your suggestions
are very welcome.

Let me first describe my understanding of the ISAR SDK image
use case scenario:

We have a team of developers who need to develop application in ISAR 
ecosystem. Somebody (let's call him project's leader) has Debian based 
PC workstation with installed ISAR ecosystem and he has to quickly pass 
ISAR build environment to other developers, which probably have other 
Linux distributions like Fedora/Suse etc.

The provided patch solves this problem - team leader just builds 
required isar-image-sdk and passes the resulting ISAR image or ISAR 
rootfs tarball to developers - they just chroot to this image and may 
start developing application.

This works fine for x86, but as you absolutely right

 > While it will work for arm(64) on x86 you will end up with a horribly
 > slow SDK. Horribly slow is OK for building the image with Isar, not for
 > the fast turn-around you are looking for as a developer.

The proposed solution is just to flash ISAR image on the ARM target (as 
modern ARM targets are quite powerful) and develop the application 
natively on the target (like this is done in raspbian world).

I hope this clarifies the idea behind the patch,

Kind regards,
Maxim.

On 05/18/2018 10:33 AM, Henning Schild wrote:
> Am Wed, 16 May 2018 17:56:51 +0200
> schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:
> 
>> Hello everybody,
>>
>> Please ignore previous patch series.
>>
>> This series introduces ISAR SDK image support.
> 
> Thanks for looking into that!
> 
> As i explained here
> https://groups.google.com/forum/#!topic/isar-users/bxmZQD8sSDQ

> There are two general ways to prepare an SDK. The image+devtools and
> the "--sysroot" of buildchroot way. You have implemented image+devtools
> which is only practical for non qemu-user-static setups ... basically
> just for x86 targets.
> 
> While it will work for arm(64) on x86 you will end up with a horribly
> slow SDK. Horribly slow is OK for building the image with Isar, not for
> the fast turn-around you are looking for as a developer.
> 
> What should be done instead is extracting the buildchroot and using it
> from the outside with "--sysroot", add example for make, CMake, eclipse
> And maybe creating an x86 Debian rootfs with the cross compiler that
> contains all that.
> 
> Henning
> 
>> In order to create ISAR SDK image just add package isar-sdk into
>> variable IMAGE_INSTALL in bitbake's conf/local.conf and fireup the
>> ISAR image build.
>>
>> When build is done, perform the following steps:
>>
>> 1) Copy host's hostname and resolv.conf into resulting rootfs:
>>
>> For example,
>>
>> sudo cp /etc/hostname
>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
>> sudo cp /etc/resolv.conf
>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
>>
>> 2) Mount the following directories in chroot by passing resulting
>> rootfs as an argument to the script mount_chroot.sh:
>>
>> $ cat mount_chroot.sh
>> #!/bin/bash
>> sudo mount /tmp     $1/tmp                 -o bind
>> sudo mount proc     $1/proc    -t proc     -o nosuid,noexec,nodev
>> sudo mount sysfs    $1/sys     -t sysfs    -o nosuid,noexec,nodev
>> sudo mount devtmpfs $1/dev     -t devtmpfs -o mode=0755,nosuid
>> sudo mount devpts   $1/dev/pts -t devpts   -o gid=5,mode=620
>> sudo mount tmpfs    $1/dev/shm -t tmpfs    -o rw,seclabel,nosuid,nodev
>>
>> $./mount_chroot.sh
>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs
> 
> Should that helper not be part of the patch series?
> 
>> 3) chroot to isar SDK rootfs:
>>
>> $sudo chroot
>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs
>>
>> 4) In chroot change user to 'builder', cd to directory isar and start
>> the isar image generation as usual.
>>
>> For example,
>>
>> root@tiberius:/# su - builder
>> builder@tiberius:~$ cd isar/
>> builder@tiberius:~/isar$ . isar-init-build-env ../build/
>> builder@tiberius:~/build$ bitbake
>> multiconfig:qemuamd64-stretch:isar-image-base
>>
>>
>> Kind regards,
>> Maxim.
>>
>> Alexander Smirnov (1):
>>    isar-bitbake: Add AUTOREV definition
>>
>> Maxim Yu. Osipov (1):
>>    image: Add ISAR SDK package to create SDK from any rootfs image.
>>
>>   meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
>>   meta/conf/isar-bitbake.conf                     |  2 ++
>>   meta/recipes-devtools/isar-sdk/files/README     |  2 ++
>>   meta/recipes-devtools/isar-sdk/files/postinst   | 21 +++++++++++++
>>   meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42
>> +++++++++++++++++++++++++ 5 files changed, 76 insertions(+)
>>   create mode 100644 meta-isar/recipes-core/images/isar-image-sdk.bb
>>   create mode 100644 meta/recipes-devtools/isar-sdk/files/README
>>   create mode 100644 meta/recipes-devtools/isar-sdk/files/postinst
>>   create mode 100644 meta/recipes-devtools/isar-sdk/isar-sdk.bb
>>
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 0/2] ISAR SDK image support

[Henning Schild]

Am Fri, 18 May 2018 13:01:08 +0200
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Hello Henning,
> 
> Thank you very much for your feedback.
> I'm new in the project and your suggestions
> are very welcome.
> 
> Let me first describe my understanding of the ISAR SDK image
> use case scenario:
> 
> We have a team of developers who need to develop application in ISAR 
> ecosystem. Somebody (let's call him project's leader) has Debian
> based PC workstation with installed ISAR ecosystem and he has to
> quickly pass ISAR build environment to other developers, which
> probably have other Linux distributions like Fedora/Suse etc.

So far so good, but they might have Windows or MacOS as well.

> The provided patch solves this problem - team leader just builds 
> required isar-image-sdk and passes the resulting ISAR image or ISAR 
> rootfs tarball to developers - they just chroot to this image and may 
> start developing application.

chroot is not available on the OS the developers are used/forced to work
in

> This works fine for x86, but as you absolutely right
> 
>  > While it will work for arm(64) on x86 you will end up with a
>  > horribly slow SDK. Horribly slow is OK for building the image with
>  > Isar, not for the fast turn-around you are looking for as a
>  > developer.  
> 
> The proposed solution is just to flash ISAR image on the ARM target
> (as modern ARM targets are quite powerful) and develop the
> application natively on the target (like this is done in raspbian
> world).

And this is exactly the point where i think you are wrong. Developers
do not want to or can not deal with additional hardware. It is
cumbersome and HW might not be available. We need something that works
on one machine (the developers notebook).
So the --sysroot for devs with any Linux + cross toolchain, and for the
Windows guys a whole Linux + cross toolchain to wrap it all into a
VM-image.

> I hope this clarifies the idea behind the patch,

The idea was clear but that way has the mentioned limitations.

Henning

> Kind regards,
> Maxim.
> 
> On 05/18/2018 10:33 AM, Henning Schild wrote:
> > Am Wed, 16 May 2018 17:56:51 +0200
> > schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:
> >   
> >> Hello everybody,
> >>
> >> Please ignore previous patch series.
> >>
> >> This series introduces ISAR SDK image support.  
> > 
> > Thanks for looking into that!
> > 
> > As i explained here
> > https://groups.google.com/forum/#!topic/isar-users/bxmZQD8sSDQ  
> 
> > There are two general ways to prepare an SDK. The image+devtools and
> > the "--sysroot" of buildchroot way. You have implemented
> > image+devtools which is only practical for non qemu-user-static
> > setups ... basically just for x86 targets.
> > 
> > While it will work for arm(64) on x86 you will end up with a
> > horribly slow SDK. Horribly slow is OK for building the image with
> > Isar, not for the fast turn-around you are looking for as a
> > developer.
> > 
> > What should be done instead is extracting the buildchroot and using
> > it from the outside with "--sysroot", add example for make, CMake,
> > eclipse And maybe creating an x86 Debian rootfs with the cross
> > compiler that contains all that.
> > 
> > Henning
> >   
> >> In order to create ISAR SDK image just add package isar-sdk into
> >> variable IMAGE_INSTALL in bitbake's conf/local.conf and fireup the
> >> ISAR image build.
> >>
> >> When build is done, perform the following steps:
> >>
> >> 1) Copy host's hostname and resolv.conf into resulting rootfs:
> >>
> >> For example,
> >>
> >> sudo cp /etc/hostname
> >> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
> >> sudo cp /etc/resolv.conf
> >> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
> >>
> >> 2) Mount the following directories in chroot by passing resulting
> >> rootfs as an argument to the script mount_chroot.sh:
> >>
> >> $ cat mount_chroot.sh
> >> #!/bin/bash
> >> sudo mount /tmp     $1/tmp                 -o bind
> >> sudo mount proc     $1/proc    -t proc     -o nosuid,noexec,nodev
> >> sudo mount sysfs    $1/sys     -t sysfs    -o nosuid,noexec,nodev
> >> sudo mount devtmpfs $1/dev     -t devtmpfs -o mode=0755,nosuid
> >> sudo mount devpts   $1/dev/pts -t devpts   -o gid=5,mode=620
> >> sudo mount tmpfs    $1/dev/shm -t tmpfs    -o
> >> rw,seclabel,nosuid,nodev
> >>
> >> $./mount_chroot.sh
> >> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs  
> > 
> > Should that helper not be part of the patch series?
> >   
> >> 3) chroot to isar SDK rootfs:
> >>
> >> $sudo chroot
> >> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs
> >>
> >> 4) In chroot change user to 'builder', cd to directory isar and
> >> start the isar image generation as usual.
> >>
> >> For example,
> >>
> >> root@tiberius:/# su - builder
> >> builder@tiberius:~$ cd isar/
> >> builder@tiberius:~/isar$ . isar-init-build-env ../build/
> >> builder@tiberius:~/build$ bitbake
> >> multiconfig:qemuamd64-stretch:isar-image-base
> >>
> >>
> >> Kind regards,
> >> Maxim.
> >>
> >> Alexander Smirnov (1):
> >>    isar-bitbake: Add AUTOREV definition
> >>
> >> Maxim Yu. Osipov (1):
> >>    image: Add ISAR SDK package to create SDK from any rootfs image.
> >>
> >>   meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
> >>   meta/conf/isar-bitbake.conf                     |  2 ++
> >>   meta/recipes-devtools/isar-sdk/files/README     |  2 ++
> >>   meta/recipes-devtools/isar-sdk/files/postinst   | 21
> >> +++++++++++++ meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42
> >> +++++++++++++++++++++++++ 5 files changed, 76 insertions(+)
> >>   create mode 100644
> >> meta-isar/recipes-core/images/isar-image-sdk.bb create mode 100644
> >> meta/recipes-devtools/isar-sdk/files/README create mode 100644
> >> meta/recipes-devtools/isar-sdk/files/postinst create mode 100644
> >> meta/recipes-devtools/isar-sdk/isar-sdk.bb 
> >   
> 
> 

Re: [PATCH v2 0/2] ISAR SDK image support

[Jan Kiszka]

On 2018-05-22 10:26, [ext] Henning Schild wrote:
> Am Fri, 18 May 2018 13:01:08 +0200
> schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:
> 
>> Hello Henning,
>>
>> Thank you very much for your feedback.
>> I'm new in the project and your suggestions
>> are very welcome.
>>
>> Let me first describe my understanding of the ISAR SDK image
>> use case scenario:
>>
>> We have a team of developers who need to develop application in ISAR 
>> ecosystem. Somebody (let's call him project's leader) has Debian
>> based PC workstation with installed ISAR ecosystem and he has to
>> quickly pass ISAR build environment to other developers, which
>> probably have other Linux distributions like Fedora/Suse etc.
> 
> So far so good, but they might have Windows or MacOS as well.

That's a VM topic in the foreseeable future.

> 
>> The provided patch solves this problem - team leader just builds 
>> required isar-image-sdk and passes the resulting ISAR image or ISAR 
>> rootfs tarball to developers - they just chroot to this image and may 
>> start developing application.
> 
> chroot is not available on the OS the developers are used/forced to work
> in

chroot is specifically problematic on Linux machines where the user
lacks the required privileges.

> 
>> This works fine for x86, but as you absolutely right
>>
>>  > While it will work for arm(64) on x86 you will end up with a
>>  > horribly slow SDK. Horribly slow is OK for building the image with
>>  > Isar, not for the fast turn-around you are looking for as a
>>  > developer.  
>>
>> The proposed solution is just to flash ISAR image on the ARM target
>> (as modern ARM targets are quite powerful) and develop the
>> application natively on the target (like this is done in raspbian
>> world).

Nope, that's not an SDK as we need to provide. A key point of the SDK is
speed in the turnaround. And not all ARM targets are 8-core Cortex-A57
or better yet, not to speak of their storage devices.

Jan

> 
> And this is exactly the point where i think you are wrong. Developers
> do not want to or can not deal with additional hardware. It is
> cumbersome and HW might not be available. We need something that works
> on one machine (the developers notebook).
> So the --sysroot for devs with any Linux + cross toolchain, and for the
> Windows guys a whole Linux + cross toolchain to wrap it all into a
> VM-image.
> 
>> I hope this clarifies the idea behind the patch,
> 
> The idea was clear but that way has the mentioned limitations.
> 
> Henning
> 
>> Kind regards,
>> Maxim.
>>
>> On 05/18/2018 10:33 AM, Henning Schild wrote:
>>> Am Wed, 16 May 2018 17:56:51 +0200
>>> schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:
>>>   
>>>> Hello everybody,
>>>>
>>>> Please ignore previous patch series.
>>>>
>>>> This series introduces ISAR SDK image support.  
>>>
>>> Thanks for looking into that!
>>>
>>> As i explained here
>>> https://groups.google.com/forum/#!topic/isar-users/bxmZQD8sSDQ  
>>
>>> There are two general ways to prepare an SDK. The image+devtools and
>>> the "--sysroot" of buildchroot way. You have implemented
>>> image+devtools which is only practical for non qemu-user-static
>>> setups ... basically just for x86 targets.
>>>
>>> While it will work for arm(64) on x86 you will end up with a
>>> horribly slow SDK. Horribly slow is OK for building the image with
>>> Isar, not for the fast turn-around you are looking for as a
>>> developer.
>>>
>>> What should be done instead is extracting the buildchroot and using
>>> it from the outside with "--sysroot", add example for make, CMake,
>>> eclipse And maybe creating an x86 Debian rootfs with the cross
>>> compiler that contains all that.
>>>
>>> Henning
>>>   
>>>> In order to create ISAR SDK image just add package isar-sdk into
>>>> variable IMAGE_INSTALL in bitbake's conf/local.conf and fireup the
>>>> ISAR image build.
>>>>
>>>> When build is done, perform the following steps:
>>>>
>>>> 1) Copy host's hostname and resolv.conf into resulting rootfs:
>>>>
>>>> For example,
>>>>
>>>> sudo cp /etc/hostname
>>>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
>>>> sudo cp /etc/resolv.conf
>>>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs/etc
>>>>
>>>> 2) Mount the following directories in chroot by passing resulting
>>>> rootfs as an argument to the script mount_chroot.sh:
>>>>
>>>> $ cat mount_chroot.sh
>>>> #!/bin/bash
>>>> sudo mount /tmp     $1/tmp                 -o bind
>>>> sudo mount proc     $1/proc    -t proc     -o nosuid,noexec,nodev
>>>> sudo mount sysfs    $1/sys     -t sysfs    -o nosuid,noexec,nodev
>>>> sudo mount devtmpfs $1/dev     -t devtmpfs -o mode=0755,nosuid
>>>> sudo mount devpts   $1/dev/pts -t devpts   -o gid=5,mode=620
>>>> sudo mount tmpfs    $1/dev/shm -t tmpfs    -o
>>>> rw,seclabel,nosuid,nodev
>>>>
>>>> $./mount_chroot.sh
>>>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs  
>>>
>>> Should that helper not be part of the patch series?
>>>   
>>>> 3) chroot to isar SDK rootfs:
>>>>
>>>> $sudo chroot
>>>> $HOME/isar/src/build/tmp/work/debian-stretch-amd64/isar-image-base/rootfs
>>>>
>>>> 4) In chroot change user to 'builder', cd to directory isar and
>>>> start the isar image generation as usual.
>>>>
>>>> For example,
>>>>
>>>> root@tiberius:/# su - builder
>>>> builder@tiberius:~$ cd isar/
>>>> builder@tiberius:~/isar$ . isar-init-build-env ../build/
>>>> builder@tiberius:~/build$ bitbake
>>>> multiconfig:qemuamd64-stretch:isar-image-base
>>>>
>>>>
>>>> Kind regards,
>>>> Maxim.
>>>>
>>>> Alexander Smirnov (1):
>>>>    isar-bitbake: Add AUTOREV definition
>>>>
>>>> Maxim Yu. Osipov (1):
>>>>    image: Add ISAR SDK package to create SDK from any rootfs image.
>>>>
>>>>   meta-isar/recipes-core/images/isar-image-sdk.bb |  9 ++++++
>>>>   meta/conf/isar-bitbake.conf                     |  2 ++
>>>>   meta/recipes-devtools/isar-sdk/files/README     |  2 ++
>>>>   meta/recipes-devtools/isar-sdk/files/postinst   | 21
>>>> +++++++++++++ meta/recipes-devtools/isar-sdk/isar-sdk.bb      | 42
>>>> +++++++++++++++++++++++++ 5 files changed, 76 insertions(+)
>>>>   create mode 100644
>>>> meta-isar/recipes-core/images/isar-image-sdk.bb create mode 100644
>>>> meta/recipes-devtools/isar-sdk/files/README create mode 100644
>>>> meta/recipes-devtools/isar-sdk/files/postinst create mode 100644
>>>> meta/recipes-devtools/isar-sdk/isar-sdk.bb 
>>>   
>>
>>
>