isar error log

isar error log

[daniel.sangorrin]

[-- Attachment #1: Type: text/plain, Size: 5867 bytes --]

Hi Jan,

I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.
I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
I already installed qemu-user-static. The system service for binfmt seems to fail.

Thanks,
Daniel

update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument
update-binfmts: warning: unable to enable binary format qemu-aarch64
update-binfmts: exiting due to previous errors
2019-04-01 15:37:22 - INFO     - kas 1.0 started
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t ee5518cc6ecdade53dcafabfbb40ba4b0c505777
2019-04-01 15:37:22 - INFO     - Repository isar already contains ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec. Nothing to do.
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build cip-core-image
Loading cache: 100% |######################################################################################################################################| Time: 0:00:00
Loaded 19 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |#################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_bootstrap.77)
ERROR: Logfile of failure stored in: /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_bootstrap.77
Log data follows:
| DEBUG: Executing shell function do_bootstrap
| W: Target architecture is the same as host architecture; disabling QEMU support
| I: Running command: debootstrap --arch amd64 --verbose --variant=minbase --include=locales --components=main,contrib,non-free stretch /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs http://ftp.de.debian.org/debian
| /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/test-dev-null: Permission denied
| E: Cannot install into target '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs' mounted with noexec or nodev
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_bootstrap.77)
ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1'
ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/temp/log.do_bootstrap.79)
ERROR: Logfile of failure stored in: /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/temp/log.do_bootstrap.79
Log data follows:
| DEBUG: Executing shell function do_bootstrap
| W: Target architecture is the same as host architecture; disabling QEMU support
| I: Running command: debootstrap --arch amd64 --verbose --variant=minbase --include=locales --components=main,contrib,non-free stretch /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/rootfs http://ftp.de.debian.org/debian
| /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/rootfs/test-dev-null: Permission denied
| E: Cannot install into target '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/rootfs' mounted with noexec or nodev
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/temp/log.do_bootstrap.79)
ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed with exit code '1'

$ systemctl status systemd-binfmt.service
● systemd-binfmt.service - Set Up Additional Binary Formats
   Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
   Active: inactive (dead)
Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
           none of the trigger conditions were met
     Docs: man:systemd-binfmt.service(8)
           man:binfmt.d(5)
           https://www.kernel.org/doc/Documentation/binfmt_misc.txt

[-- Attachment #2: Type: text/html, Size: 10631 bytes --]

Re: isar error log

[Jan Kiszka]

On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
> Hi Jan,
> 
> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.

It's supposed to be fixed in the :latest kas-isar image. Are you using that?

> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
> I already installed qemu-user-static. The system service for binfmt seems to fail.
> 
> Thanks,
> Daniel
> 
> update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument
> update-binfmts: warning: unable to enable binary format qemu-aarch64
> update-binfmts: exiting due to previous errors

That indeed indicate broken binfmt settings.

> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> 2019-04-01 15:37:22 - INFO     - Repository isar already contains ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec. Nothing to do.
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build cip-core-image
> Loading cache: 100% |######################################################################################################################################| Time: 0:00:00
> Loaded 19 entries from dependency cache.
> NOTE: Resolving any missing task queue dependencies
> Initialising tasks: 100% |#################################################################################################################################| Time: 0:00:00
> NOTE: Executing RunQueue Tasks
> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_bootstrap.77)
> ERROR: Logfile of failure stored in: /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_bootstrap.77
> Log data follows:
> | DEBUG: Executing shell function do_bootstrap
> | W: Target architecture is the same as host architecture; disabling QEMU support
> | I: Running command: debootstrap --arch amd64 --verbose --variant=minbase --include=locales --components=main,contrib,non-free stretch /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs http://ftp.de.debian.org/debian
> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/test-dev-null: Permission denied
> | E: Cannot install into target '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs' mounted with noexec or nodev

Hmm, there are more strange errors. Could it be that you forgot the "--isar"  
after "kas-docker"? That would mean you are trying to run unprivileged which is  
doomed to fail.

> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_bootstrap.77)
> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1'
> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/temp/log.do_bootstrap.79)
> ERROR: Logfile of failure stored in: /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/temp/log.do_bootstrap.79
> Log data follows:
> | DEBUG: Executing shell function do_bootstrap
> | W: Target architecture is the same as host architecture; disabling QEMU support
> | I: Running command: debootstrap --arch amd64 --verbose --variant=minbase --include=locales --components=main,contrib,non-free stretch /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/rootfs http://ftp.de.debian.org/debian
> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/rootfs/test-dev-null: Permission denied
> | E: Cannot install into target '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/rootfs' mounted with noexec or nodev
> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_bootstrap (log file is located at /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch-amd64/temp/log.do_bootstrap.79)
> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed with exit code '1'
> 
> $ systemctl status systemd-binfmt.service
> ● systemd-binfmt.service - Set Up Additional Binary Formats
>     Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
>     Active: inactive (dead)
> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
>             none of the trigger conditions were met
>       Docs: man:systemd-binfmt.service(8)
>             man:binfmt.d(5)
>             https://www.kernel.org/doc/Documentation/binfmt_misc.txt
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

RE: isar error log

[daniel.sangorrin]

Hi Jan,

Yes I am following the Readme.

The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am using
kas-docker --isar build kas.yml:board-qemu-amd64.yml

Yes, it seems there are multiple errors:
/proc/sys/fs/binfmt_misc/register: Invalid argument
/test-dev-null: Permission denied

Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.

Thanks,
Daniel

-----Original Message-----
From: Jan Kiszka <jan.kiszka@siemens.com> 
Sent: Tuesday, April 2, 2019 3:14 PM
To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com
Subject: Re: isar error log

On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
> Hi Jan,
> 
> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.

It's supposed to be fixed in the :latest kas-isar image. Are you using that?

> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
> I already installed qemu-user-static. The system service for binfmt seems to fail.
> 
> Thanks,
> Daniel
> 
> update-binfmts: warning: unable to close 
> /proc/sys/fs/binfmt_misc/register: Invalid argument
> update-binfmts: warning: unable to enable binary format qemu-aarch64
> update-binfmts: exiting due to previous errors

That indeed indicate broken binfmt settings.

> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> 2019-04-01 15:37:22 - INFO     - Repository isar already contains ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec. Nothing to do.
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build cip-core-image
> Loading cache: 100% 
> |######################################################################################################################################| Time: 0:00:00 Loaded 19 entries from dependency cache.
> NOTE: Resolving any missing task queue dependencies Initialising 
> tasks: 100% 
> |#####################################################################
> ############################################################| Time: 
> 0:00:00
> NOTE: Executing RunQueue Tasks
> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed: 
> do_bootstrap (log file is located at 
> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> bootstrap.77)
> ERROR: Logfile of failure stored in: 
> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> bootstrap.77
> Log data follows:
> | DEBUG: Executing shell function do_bootstrap
> | W: Target architecture is the same as host architecture; disabling 
> | QEMU support
> | I: Running command: debootstrap --arch amd64 --verbose 
> | --variant=minbase --include=locales 
> | --components=main,contrib,non-free stretch 
> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs 
> | http://ftp.de.debian.org/debian
> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create 
> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/tes
> | t-dev-null: Permission denied
> | E: Cannot install into target 
> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs' 
> | mounted with noexec or nodev

Hmm, there are more strange errors. Could it be that you forgot the "--isar"  
after "kas-docker"? That would mean you are trying to run unprivileged which is doomed to fail.

> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_bootstrap (log file is located at 
> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.d
> | o_bootstrap.77)
> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1'
> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed: 
> do_bootstrap (log file is located at 
> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> -amd64/temp/log.do_bootstrap.79)
> ERROR: Logfile of failure stored in: 
> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> -amd64/temp/log.do_bootstrap.79
> Log data follows:
> | DEBUG: Executing shell function do_bootstrap
> | W: Target architecture is the same as host architecture; disabling 
> | QEMU support
> | I: Running command: debootstrap --arch amd64 --verbose 
> | --variant=minbase --include=locales 
> | --components=main,contrib,non-free stretch 
> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> | ch-amd64/rootfs http://ftp.de.debian.org/debian
> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create 
> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> | ch-amd64/rootfs/test-dev-null: Permission denied
> | E: Cannot install into target 
> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stre
> | tch-amd64/rootfs' mounted with noexec or nodev
> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_bootstrap (log file is located at 
> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> | ch-amd64/temp/log.do_bootstrap.79)
> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed with exit code '1'
> 
> $ systemctl status systemd-binfmt.service ● systemd-binfmt.service - 
> Set Up Additional Binary Formats
>     Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
>     Active: inactive (dead)
> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
>             none of the trigger conditions were met
>       Docs: man:systemd-binfmt.service(8)
>             man:binfmt.d(5)
>             https://www.kernel.org/doc/Documentation/binfmt_misc.txt
> 

Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux

Re: isar error log

[Jan Kiszka]

On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:
> Hi Jan,
> 
> Yes I am following the Readme.
> 
> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am using

That's the same version I have, and it works for me.

> kas-docker --isar build kas.yml:board-qemu-amd64.yml
> 
> Yes, it seems there are multiple errors:
> /proc/sys/fs/binfmt_misc/register: Invalid argument
> /test-dev-null: Permission denied
> 
> Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.
> 

Adding kas-devel: Would be interesting to understand this so that we can fix it 
or at least document a workaround. Maybe you need to be in some privileged group 
in order to use privileged docker containers? That's how things work on SUSE.

No one around is using Ubuntu, so that distro is likely a blind spot for our 
testing.

Thanks
Jan

> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: Tuesday, April 2, 2019 3:14 PM
> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
> Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com
> Subject: Re: isar error log
> 
> On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
>> Hi Jan,
>>
>> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.
> 
> It's supposed to be fixed in the :latest kas-isar image. Are you using that?
> 
>> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
>> I already installed qemu-user-static. The system service for binfmt seems to fail.
>>
>> Thanks,
>> Daniel
>>
>> update-binfmts: warning: unable to close
>> /proc/sys/fs/binfmt_misc/register: Invalid argument
>> update-binfmts: warning: unable to enable binary format qemu-aarch64
>> update-binfmts: exiting due to previous errors
> 
> That indeed indicate broken binfmt settings.
> 
>> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t ee5518cc6ecdade53dcafabfbb40ba4b0c505777
>> 2019-04-01 15:37:22 - INFO     - Repository isar already contains ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
>> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
>> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec. Nothing to do.
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build cip-core-image
>> Loading cache: 100%
>> |######################################################################################################################################| Time: 0:00:00 Loaded 19 entries from dependency cache.
>> NOTE: Resolving any missing task queue dependencies Initialising
>> tasks: 100%
>> |#####################################################################
>> ############################################################| Time:
>> 0:00:00
>> NOTE: Executing RunQueue Tasks
>> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed:
>> do_bootstrap (log file is located at
>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
>> bootstrap.77)
>> ERROR: Logfile of failure stored in:
>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
>> bootstrap.77
>> Log data follows:
>> | DEBUG: Executing shell function do_bootstrap
>> | W: Target architecture is the same as host architecture; disabling
>> | QEMU support
>> | I: Running command: debootstrap --arch amd64 --verbose
>> | --variant=minbase --include=locales
>> | --components=main,contrib,non-free stretch
>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs
>> | http://ftp.de.debian.org/debian
>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/tes
>> | t-dev-null: Permission denied
>> | E: Cannot install into target
>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs'
>> | mounted with noexec or nodev
> 
> Hmm, there are more strange errors. Could it be that you forgot the "--isar"
> after "kas-docker"? That would mean you are trying to run unprivileged which is doomed to fail.
> 
>> | WARNING: exit code 1 from a shell command.
>> | ERROR: Function failed: do_bootstrap (log file is located at
>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.d
>> | o_bootstrap.77)
>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1'
>> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed:
>> do_bootstrap (log file is located at
>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
>> -amd64/temp/log.do_bootstrap.79)
>> ERROR: Logfile of failure stored in:
>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
>> -amd64/temp/log.do_bootstrap.79
>> Log data follows:
>> | DEBUG: Executing shell function do_bootstrap
>> | W: Target architecture is the same as host architecture; disabling
>> | QEMU support
>> | I: Running command: debootstrap --arch amd64 --verbose
>> | --variant=minbase --include=locales
>> | --components=main,contrib,non-free stretch
>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
>> | ch-amd64/rootfs http://ftp.de.debian.org/debian
>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
>> | ch-amd64/rootfs/test-dev-null: Permission denied
>> | E: Cannot install into target
>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stre
>> | tch-amd64/rootfs' mounted with noexec or nodev
>> | WARNING: exit code 1 from a shell command.
>> | ERROR: Function failed: do_bootstrap (log file is located at
>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
>> | ch-amd64/temp/log.do_bootstrap.79)
>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed with exit code '1'
>>
>> $ systemctl status systemd-binfmt.service ● systemd-binfmt.service -
>> Set Up Additional Binary Formats
>>      Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
>>      Active: inactive (dead)
>> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
>>              none of the trigger conditions were met
>>        Docs: man:systemd-binfmt.service(8)
>>              man:binfmt.d(5)
>>              https://www.kernel.org/doc/Documentation/binfmt_misc.txt
>>
> 
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [cip-dev] isar error log

[Ben Hutchings]

On Wed, 2019-04-03 at 00:56 +0000, daniel.sangorrin@toshiba.co.jp wrote:
> Hi Jan,
> 
> Yes I am following the Readme.
> 
> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am using
> kas-docker --isar build kas.yml:board-qemu-amd64.yml
> 
> Yes, it seems there are multiple errors:
> /proc/sys/fs/binfmt_misc/register: Invalid argument
> /test-dev-null: Permission denied
> 
> Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.

Might this be restricted by an AppArmor profile?  aa-logprof should
show you AppArmor denials and change the relevant profile(s) to allow
the operations.

Ben.

-- 
Ben Hutchings, Software Developer                         Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom

RE: [cip-dev] isar error log

[daniel.sangorrin]

> -----Original Message-----
> From: Ben Hutchings <ben.hutchings@codethink.co.uk>
> Sent: Wednesday, April 3, 2019 10:53 PM
> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ)
> <daniel.sangorrin@toshiba.co.jp>; jan.kiszka@siemens.com
> Cc: isar-users@googlegroups.com; cip-dev@lists.cip-project.org
> Subject: Re: [cip-dev] isar error log
> 
> On Wed, 2019-04-03 at 00:56 +0000, daniel.sangorrin@toshiba.co.jp wrote:
> > Hi Jan,
> >
> > Yes I am following the Readme.
> >
> > The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am
> > using kas-docker script). And I am using kas-docker --isar build
> > kas.yml:board-qemu-amd64.yml
> >
> > Yes, it seems there are multiple errors:
> > /proc/sys/fs/binfmt_misc/register: Invalid argument
> > /test-dev-null: Permission denied
> >
> > Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it
> out.
> 
> Might this be restricted by an AppArmor profile?  aa-logprof should show
> you AppArmor denials and change the relevant profile(s) to allow the
> operations.

Thanks for the advice Ben.
I checked with aa-logprof but there was nothing related.
# interestingly, for some reason AppArmor was restricting some functionality in Evince

Then, I tried to update the kernel to a new version (i was using about 4.4.0-84 and the latest was around 4.4.0-145) to see if something improved but I found that the new kernel wouldn't boot properly because my disk is encrypted. So I have possibly found a regression on the 4.4 kernel, I will have to check when I get back to my desk. And there is a possibility that the reason for my problem is in the binfmt kernel module.

Thanks,
Daniel

> 
> Ben.
> 
> --
> Ben Hutchings, Software Developer                         Codethink Ltd
> https://www.codethink.co.uk/                 Dale House, 35 Dale Street
>                                      Manchester, M1 2HF, United Kingdom

RE: isar error log

[daniel.sangorrin]

> From: Jan Kiszka <jan.kiszka@siemens.com>
> On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:
> > Hi Jan,
> >
> > Yes I am following the Readme.
> >
> > The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am using
> 
> That's the same version I have, and it works for me.
> 
> > kas-docker --isar build kas.yml:board-qemu-amd64.yml
> >
> > Yes, it seems there are multiple errors:
> > /proc/sys/fs/binfmt_misc/register: Invalid argument
> > /test-dev-null: Permission denied
> >
> > Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.
> >
> 
> Adding kas-devel: Would be interesting to understand this so that we can fix it
> or at least document a workaround. Maybe you need to be in some privileged group
> in order to use privileged docker containers? That's how things work on SUSE.
> 
> No one around is using Ubuntu, so that distro is likely a blind spot for our
> testing.

OK, so I figured out that Ubuntu 16.04 was using the service "binfmt-support", not "systemd-binfmt".
For that reason, the "systemd-binfmt" service was inactive. However, "binfmt-support" is working fine.
I also checked that the post-install script of the qemu-user-static package had run correctly
and there were binfmt configuration files at /var/lib/binfmts/

I am going to try with Ubuntu 18.04 (which worked fine for Suzuki-san) and then try to compare both.

Thanks,
Daniel






> 
> Thanks
> Jan
> 
> > -----Original Message-----
> > From: Jan Kiszka <jan.kiszka@siemens.com>
> > Sent: Tuesday, April 2, 2019 3:14 PM
> > To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
> > Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com
> > Subject: Re: isar error log
> >
> > On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
> >> Hi Jan,
> >>
> >> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.
> >
> > It's supposed to be fixed in the :latest kas-isar image. Are you using that?
> >
> >> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
> >> I already installed qemu-user-static. The system service for binfmt seems to fail.
> >>
> >> Thanks,
> >> Daniel
> >>
> >> update-binfmts: warning: unable to close
> >> /proc/sys/fs/binfmt_misc/register: Invalid argument
> >> update-binfmts: warning: unable to enable binary format qemu-aarch64
> >> update-binfmts: exiting due to previous errors
> >
> > That indeed indicate broken binfmt settings.
> >
> >> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t
> ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> >> 2019-04-01 15:37:22 - INFO     - Repository isar already contains
> ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
> >> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
> >> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> >> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec. Nothing
> to do.
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build cip-core-image
> >> Loading cache: 100%
> >>
> |#################################################################
> #################################################################
> ####| Time: 0:00:00 Loaded 19 entries from dependency cache.
> >> NOTE: Resolving any missing task queue dependencies Initialising
> >> tasks: 100%
> >>
> |#################################################################
> ####
> >> ############################################################|
> Time:
> >> 0:00:00
> >> NOTE: Executing RunQueue Tasks
> >> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed:
> >> do_bootstrap (log file is located at
> >> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> >> bootstrap.77)
> >> ERROR: Logfile of failure stored in:
> >> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> >> bootstrap.77
> >> Log data follows:
> >> | DEBUG: Executing shell function do_bootstrap
> >> | W: Target architecture is the same as host architecture; disabling
> >> | QEMU support
> >> | I: Running command: debootstrap --arch amd64 --verbose
> >> | --variant=minbase --include=locales
> >> | --components=main,contrib,non-free stretch
> >> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs
> >> | http://ftp.de.debian.org/debian
> >> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
> >> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/tes
> >> | t-dev-null: Permission denied
> >> | E: Cannot install into target
> >> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs'
> >> | mounted with noexec or nodev
> >
> > Hmm, there are more strange errors. Could it be that you forgot the "--isar"
> > after "kas-docker"? That would mean you are trying to run unprivileged which is doomed to fail.
> >
> >> | WARNING: exit code 1 from a shell command.
> >> | ERROR: Function failed: do_bootstrap (log file is located at
> >> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.d
> >> | o_bootstrap.77)
> >> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed
> with exit code '1'
> >> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed:
> >> do_bootstrap (log file is located at
> >> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> >> -amd64/temp/log.do_bootstrap.79)
> >> ERROR: Logfile of failure stored in:
> >> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> >> -amd64/temp/log.do_bootstrap.79
> >> Log data follows:
> >> | DEBUG: Executing shell function do_bootstrap
> >> | W: Target architecture is the same as host architecture; disabling
> >> | QEMU support
> >> | I: Running command: debootstrap --arch amd64 --verbose
> >> | --variant=minbase --include=locales
> >> | --components=main,contrib,non-free stretch
> >> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> >> | ch-amd64/rootfs http://ftp.de.debian.org/debian
> >> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
> >> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> >> | ch-amd64/rootfs/test-dev-null: Permission denied
> >> | E: Cannot install into target
> >> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stre
> >> | tch-amd64/rootfs' mounted with noexec or nodev
> >> | WARNING: exit code 1 from a shell command.
> >> | ERROR: Function failed: do_bootstrap (log file is located at
> >> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> >> | ch-amd64/temp/log.do_bootstrap.79)
> >> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed
> with exit code '1'
> >>
> >> $ systemctl status systemd-binfmt.service ● systemd-binfmt.service -
> >> Set Up Additional Binary Formats
> >>      Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
> >>      Active: inactive (dead)
> >> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
> >>              none of the trigger conditions were met
> >>        Docs: man:systemd-binfmt.service(8)
> >>              man:binfmt.d(5)
> >>              https://www.kernel.org/doc/Documentation/binfmt_misc.txt
> >>
> >
> > Jan
> >
> > --
> > Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
> >
> 
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux

Re: isar error log

[Jan Kiszka]

On 11.04.19 11:08, daniel.sangorrin@toshiba.co.jp wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>> On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:
>>> Hi Jan,
>>>
>>> Yes I am following the Readme.
>>>
>>> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am using
>>
>> That's the same version I have, and it works for me.
>>
>>> kas-docker --isar build kas.yml:board-qemu-amd64.yml
>>>
>>> Yes, it seems there are multiple errors:
>>> /proc/sys/fs/binfmt_misc/register: Invalid argument
>>> /test-dev-null: Permission denied
>>>
>>> Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.
>>>
>>
>> Adding kas-devel: Would be interesting to understand this so that we can fix it
>> or at least document a workaround. Maybe you need to be in some privileged group
>> in order to use privileged docker containers? That's how things work on SUSE.
>>
>> No one around is using Ubuntu, so that distro is likely a blind spot for our
>> testing.
> 
> OK, so I figured out that Ubuntu 16.04 was using the service "binfmt-support", not "systemd-binfmt".
> For that reason, the "systemd-binfmt" service was inactive. However, "binfmt-support" is working fine.
> I also checked that the post-install script of the qemu-user-static package had run correctly
> and there were binfmt configuration files at /var/lib/binfmts/

But what prevented the container to tune the host kernel according to its needs? 
That is the intended resolution for the host not having the same settings as the 
container (or any at all).

Jan

> 
> I am going to try with Ubuntu 18.04 (which worked fine for Suzuki-san) and then try to compare both.
> 
> Thanks,
> Daniel
> 
> 
> 
> 
> 
> 
>>
>> Thanks
>> Jan
>>
>>> -----Original Message-----
>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>> Sent: Tuesday, April 2, 2019 3:14 PM
>>> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
>>> Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com
>>> Subject: Re: isar error log
>>>
>>> On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
>>>> Hi Jan,
>>>>
>>>> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.
>>>
>>> It's supposed to be fixed in the :latest kas-isar image. Are you using that?
>>>
>>>> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
>>>> I already installed qemu-user-static. The system service for binfmt seems to fail.
>>>>
>>>> Thanks,
>>>> Daniel
>>>>
>>>> update-binfmts: warning: unable to close
>>>> /proc/sys/fs/binfmt_misc/register: Invalid argument
>>>> update-binfmts: warning: unable to enable binary format qemu-aarch64
>>>> update-binfmts: exiting due to previous errors
>>>
>>> That indeed indicate broken binfmt settings.
>>>
>>>> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t
>> ee5518cc6ecdade53dcafabfbb40ba4b0c505777
>>>> 2019-04-01 15:37:22 - INFO     - Repository isar already contains
>> ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
>>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
>>>> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
>>>> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec. Nothing
>> to do.
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
>>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
>>>> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build cip-core-image
>>>> Loading cache: 100%
>>>>
>> |#################################################################
>> #################################################################
>> ####| Time: 0:00:00 Loaded 19 entries from dependency cache.
>>>> NOTE: Resolving any missing task queue dependencies Initialising
>>>> tasks: 100%
>>>>
>> |#################################################################
>> ####
>>>> ############################################################|
>> Time:
>>>> 0:00:00
>>>> NOTE: Executing RunQueue Tasks
>>>> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed:
>>>> do_bootstrap (log file is located at
>>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
>>>> bootstrap.77)
>>>> ERROR: Logfile of failure stored in:
>>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
>>>> bootstrap.77
>>>> Log data follows:
>>>> | DEBUG: Executing shell function do_bootstrap
>>>> | W: Target architecture is the same as host architecture; disabling
>>>> | QEMU support
>>>> | I: Running command: debootstrap --arch amd64 --verbose
>>>> | --variant=minbase --include=locales
>>>> | --components=main,contrib,non-free stretch
>>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs
>>>> | http://ftp.de.debian.org/debian
>>>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
>>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/tes
>>>> | t-dev-null: Permission denied
>>>> | E: Cannot install into target
>>>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs'
>>>> | mounted with noexec or nodev
>>>
>>> Hmm, there are more strange errors. Could it be that you forgot the "--isar"
>>> after "kas-docker"? That would mean you are trying to run unprivileged which is doomed to fail.
>>>
>>>> | WARNING: exit code 1 from a shell command.
>>>> | ERROR: Function failed: do_bootstrap (log file is located at
>>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.d
>>>> | o_bootstrap.77)
>>>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed
>> with exit code '1'
>>>> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed:
>>>> do_bootstrap (log file is located at
>>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
>>>> -amd64/temp/log.do_bootstrap.79)
>>>> ERROR: Logfile of failure stored in:
>>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
>>>> -amd64/temp/log.do_bootstrap.79
>>>> Log data follows:
>>>> | DEBUG: Executing shell function do_bootstrap
>>>> | W: Target architecture is the same as host architecture; disabling
>>>> | QEMU support
>>>> | I: Running command: debootstrap --arch amd64 --verbose
>>>> | --variant=minbase --include=locales
>>>> | --components=main,contrib,non-free stretch
>>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
>>>> | ch-amd64/rootfs http://ftp.de.debian.org/debian
>>>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
>>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
>>>> | ch-amd64/rootfs/test-dev-null: Permission denied
>>>> | E: Cannot install into target
>>>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stre
>>>> | tch-amd64/rootfs' mounted with noexec or nodev
>>>> | WARNING: exit code 1 from a shell command.
>>>> | ERROR: Function failed: do_bootstrap (log file is located at
>>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
>>>> | ch-amd64/temp/log.do_bootstrap.79)
>>>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed
>> with exit code '1'
>>>>
>>>> $ systemctl status systemd-binfmt.service ● systemd-binfmt.service -
>>>> Set Up Additional Binary Formats
>>>>       Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
>>>>       Active: inactive (dead)
>>>> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
>>>>               none of the trigger conditions were met
>>>>         Docs: man:systemd-binfmt.service(8)
>>>>               man:binfmt.d(5)
>>>>               https://www.kernel.org/doc/Documentation/binfmt_misc.txt
>>>>
>>>
>>> Jan
>>>
>>> --
>>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
>>>
>>
>> --
>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
>> Corporate Competence Center Embedded Linux

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

RE: isar error log

[daniel.sangorrin]

> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: Thursday, April 11, 2019 6:16 PM
> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
> Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com; kas-devel@googlegroups.com
> Subject: Re: isar error log
> 
> On 11.04.19 11:08, daniel.sangorrin@toshiba.co.jp wrote:
> >> From: Jan Kiszka <jan.kiszka@siemens.com>
> >> On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:
> >>> Hi Jan,
> >>>
> >>> Yes I am following the Readme.
> >>>
> >>> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am using
> >>
> >> That's the same version I have, and it works for me.
> >>
> >>> kas-docker --isar build kas.yml:board-qemu-amd64.yml
> >>>
> >>> Yes, it seems there are multiple errors:
> >>> /proc/sys/fs/binfmt_misc/register: Invalid argument
> >>> /test-dev-null: Permission denied
> >>>
> >>> Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.
> >>>
> >>
> >> Adding kas-devel: Would be interesting to understand this so that we can fix it
> >> or at least document a workaround. Maybe you need to be in some privileged group
> >> in order to use privileged docker containers? That's how things work on SUSE.
> >>
> >> No one around is using Ubuntu, so that distro is likely a blind spot for our
> >> testing.
> >
> > OK, so I figured out that Ubuntu 16.04 was using the service "binfmt-support", not "systemd-binfmt".
> > For that reason, the "systemd-binfmt" service was inactive. However, "binfmt-support" is working fine.
> > I also checked that the post-install script of the qemu-user-static package had run correctly
> > and there were binfmt configuration files at /var/lib/binfmts/
> 
> But what prevented the container to tune the host kernel according to its needs?
> That is the intended resolution for the host not having the same settings as the
> container (or any at all).

To be honest I don't really know what binfmt is all about yet, so I can't even understand what you mean about "tuning the kernel".

By the way, I managed to update to the latest Ubuntu 4.4 kernel (145) and the problem persists. I tried to reinstall qemu-user-static etc but still fails humm
.
Also i just tested on Ubuntu 18.04 and there I can build with ISAR without problems. So I might just leave it unsolved and update to 18.04...

Thanks,
Daniel





> 
> Jan
> 
> >
> > I am going to try with Ubuntu 18.04 (which worked fine for Suzuki-san) and then try to compare both.
> >
> > Thanks,
> > Daniel
> >
> >
> >
> >
> >
> >
> >>
> >> Thanks
> >> Jan
> >>
> >>> -----Original Message-----
> >>> From: Jan Kiszka <jan.kiszka@siemens.com>
> >>> Sent: Tuesday, April 2, 2019 3:14 PM
> >>> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
> >>> Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com
> >>> Subject: Re: isar error log
> >>>
> >>> On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
> >>>> Hi Jan,
> >>>>
> >>>> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.
> >>>
> >>> It's supposed to be fixed in the :latest kas-isar image. Are you using that?
> >>>
> >>>> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
> >>>> I already installed qemu-user-static. The system service for binfmt seems to fail.
> >>>>
> >>>> Thanks,
> >>>> Daniel
> >>>>
> >>>> update-binfmts: warning: unable to close
> >>>> /proc/sys/fs/binfmt_misc/register: Invalid argument
> >>>> update-binfmts: warning: unable to enable binary format qemu-aarch64
> >>>> update-binfmts: exiting due to previous errors
> >>>
> >>> That indeed indicate broken binfmt settings.
> >>>
> >>>> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t
> >> ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> >>>> 2019-04-01 15:37:22 - INFO     - Repository isar already contains
> >> ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
> >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
> >>>> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> >>>> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec.
> Nothing
> >> to do.
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> >>>> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build
> cip-core-image
> >>>> Loading cache: 100%
> >>>>
> >>
> |#################################################################
> >>
> #################################################################
> >> ####| Time: 0:00:00 Loaded 19 entries from dependency cache.
> >>>> NOTE: Resolving any missing task queue dependencies Initialising
> >>>> tasks: 100%
> >>>>
> >>
> |#################################################################
> >> ####
> >>>> ############################################################|
> >> Time:
> >>>> 0:00:00
> >>>> NOTE: Executing RunQueue Tasks
> >>>> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed:
> >>>> do_bootstrap (log file is located at
> >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> >>>> bootstrap.77)
> >>>> ERROR: Logfile of failure stored in:
> >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> >>>> bootstrap.77
> >>>> Log data follows:
> >>>> | DEBUG: Executing shell function do_bootstrap
> >>>> | W: Target architecture is the same as host architecture; disabling
> >>>> | QEMU support
> >>>> | I: Running command: debootstrap --arch amd64 --verbose
> >>>> | --variant=minbase --include=locales
> >>>> | --components=main,contrib,non-free stretch
> >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs
> >>>> | http://ftp.de.debian.org/debian
> >>>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
> >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/tes
> >>>> | t-dev-null: Permission denied
> >>>> | E: Cannot install into target
> >>>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs'
> >>>> | mounted with noexec or nodev
> >>>
> >>> Hmm, there are more strange errors. Could it be that you forgot the "--isar"
> >>> after "kas-docker"? That would mean you are trying to run unprivileged which is doomed to fail.
> >>>
> >>>> | WARNING: exit code 1 from a shell command.
> >>>> | ERROR: Function failed: do_bootstrap (log file is located at
> >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.d
> >>>> | o_bootstrap.77)
> >>>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap)
> failed
> >> with exit code '1'
> >>>> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed:
> >>>> do_bootstrap (log file is located at
> >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> >>>> -amd64/temp/log.do_bootstrap.79)
> >>>> ERROR: Logfile of failure stored in:
> >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> >>>> -amd64/temp/log.do_bootstrap.79
> >>>> Log data follows:
> >>>> | DEBUG: Executing shell function do_bootstrap
> >>>> | W: Target architecture is the same as host architecture; disabling
> >>>> | QEMU support
> >>>> | I: Running command: debootstrap --arch amd64 --verbose
> >>>> | --variant=minbase --include=locales
> >>>> | --components=main,contrib,non-free stretch
> >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> >>>> | ch-amd64/rootfs http://ftp.de.debian.org/debian
> >>>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
> >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> >>>> | ch-amd64/rootfs/test-dev-null: Permission denied
> >>>> | E: Cannot install into target
> >>>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stre
> >>>> | tch-amd64/rootfs' mounted with noexec or nodev
> >>>> | WARNING: exit code 1 from a shell command.
> >>>> | ERROR: Function failed: do_bootstrap (log file is located at
> >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> >>>> | ch-amd64/temp/log.do_bootstrap.79)
> >>>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap) failed
> >> with exit code '1'
> >>>>
> >>>> $ systemctl status systemd-binfmt.service ● systemd-binfmt.service -
> >>>> Set Up Additional Binary Formats
> >>>>       Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
> >>>>       Active: inactive (dead)
> >>>> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
> >>>>               none of the trigger conditions were met
> >>>>         Docs: man:systemd-binfmt.service(8)
> >>>>               man:binfmt.d(5)
> >>>>               https://www.kernel.org/doc/Documentation/binfmt_misc.txt
> >>>>
> >>>
> >>> Jan
> >>>
> >>> --
> >>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
> >>>
> >>
> >> --
> >> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> >> Corporate Competence Center Embedded Linux
> 
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux

RE: isar error log

[daniel.sangorrin]

> -----Original Message-----
> From: cip-dev-bounces@lists.cip-project.org <cip-dev-bounces@lists.cip-project.org> On Behalf Of
> daniel.sangorrin@toshiba.co.jp
> Sent: Thursday, April 11, 2019 7:06 PM
> To: jan.kiszka@siemens.com
> Cc: kas-devel@googlegroups.com; isar-users@googlegroups.com; cip-dev@lists.cip-project.org
> Subject: Re: [cip-dev] isar error log
> 
> > -----Original Message-----
> > From: Jan Kiszka <jan.kiszka@siemens.com>
> > Sent: Thursday, April 11, 2019 6:16 PM
> > To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
> > Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com; kas-devel@googlegroups.com
> > Subject: Re: isar error log
> >
> > On 11.04.19 11:08, daniel.sangorrin@toshiba.co.jp wrote:
> > >> From: Jan Kiszka <jan.kiszka@siemens.com>
> > >> On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:
> > >>> Hi Jan,
> > >>>
> > >>> Yes I am following the Readme.
> > >>>
> > >>> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am
> using
> > >>
> > >> That's the same version I have, and it works for me.
> > >>
> > >>> kas-docker --isar build kas.yml:board-qemu-amd64.yml
> > >>>
> > >>> Yes, it seems there are multiple errors:
> > >>> /proc/sys/fs/binfmt_misc/register: Invalid argument
> > >>> /test-dev-null: Permission denied
> > >>>
> > >>> Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.
> > >>>
> > >>
> > >> Adding kas-devel: Would be interesting to understand this so that we can fix it
> > >> or at least document a workaround. Maybe you need to be in some privileged group
> > >> in order to use privileged docker containers? That's how things work on SUSE.
> > >>
> > >> No one around is using Ubuntu, so that distro is likely a blind spot for our
> > >> testing.
> > >
> > > OK, so I figured out that Ubuntu 16.04 was using the service "binfmt-support", not "systemd-binfmt".
> > > For that reason, the "systemd-binfmt" service was inactive. However, "binfmt-support" is working fine.
> > > I also checked that the post-install script of the qemu-user-static package had run correctly
> > > and there were binfmt configuration files at /var/lib/binfmts/
> >
> > But what prevented the container to tune the host kernel according to its needs?
> > That is the intended resolution for the host not having the same settings as the
> > container (or any at all).
> 
> To be honest I don't really know what binfmt is all about yet, so I can't even understand what you mean about
> "tuning the kernel".
> 
> By the way, I managed to update to the latest Ubuntu 4.4 kernel (145) and the problem persists. I tried to reinstall
> qemu-user-static etc but still fails humm
> .
> Also i just tested on Ubuntu 18.04 and there I can build with ISAR without problems. So I might just leave it
> unsolved and update to 18.04...

This might be a hint.
On Ubuntu 18.04 I can see "qemu-alpha" on /proc/sys/fs/binfmt_misc, but on Ubuntu 16.04 it isn't there.
When running kas-docker, the first error is about not being able to close /proc/sys/fs/binfmt_misc/register (invalid argument)
and the second is about qemu-alpha.

Thanks,
Daniel


> >
> > Jan
> >
> > >
> > > I am going to try with Ubuntu 18.04 (which worked fine for Suzuki-san) and then try to compare both.
> > >
> > > Thanks,
> > > Daniel
> > >
> > >
> > >
> > >
> > >
> > >
> > >>
> > >> Thanks
> > >> Jan
> > >>
> > >>> -----Original Message-----
> > >>> From: Jan Kiszka <jan.kiszka@siemens.com>
> > >>> Sent: Tuesday, April 2, 2019 3:14 PM
> > >>> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
> > >>> Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com
> > >>> Subject: Re: isar error log
> > >>>
> > >>> On 02.04.19 05:17, daniel.sangorrin@toshiba.co.jp wrote:
> > >>>> Hi Jan,
> > >>>>
> > >>>> I am trying to show CIP Core ISAR to some guys in Linaro Connect but I also met a binfmt related error.
> > >>>
> > >>> It's supposed to be fixed in the :latest kas-isar image. Are you using that?
> > >>>
> > >>>> I am using Ubuntu 16.04. Do you have a hint on how to solve this problem?
> > >>>> I already installed qemu-user-static. The system service for binfmt seems to fail.
> > >>>>
> > >>>> Thanks,
> > >>>> Daniel
> > >>>>
> > >>>> update-binfmts: warning: unable to close
> > >>>> /proc/sys/fs/binfmt_misc/register: Invalid argument
> > >>>> update-binfmts: warning: unable to enable binary format qemu-aarch64
> > >>>> update-binfmts: exiting due to previous errors
> > >>>
> > >>> That indeed indicate broken binfmt settings.
> > >>>
> > >>>> 2019-04-01 15:37:22 - INFO     - kas 1.0 started
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> > >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git cat-file -t
> > >> ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> > >>>> 2019-04-01 15:37:22 - INFO     - Repository isar already contains
> > >> ee5518cc6ecdade53dcafabfbb40ba4b0c505777 as commit
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> > >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git status -s
> > >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ git rev-parse --verify HEAD
> > >>>> 2019-04-01 15:37:22 - INFO     - ee5518cc6ecdade53dcafabfbb40ba4b0c505777
> > >>>> 2019-04-01 15:37:22 - INFO     - Repo isar has already been checked out with correct refspec.
> > Nothing
> > >> to do.
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> > >>>> 2019-04-01 15:37:22 - INFO     - /work/isar$ /tmp/tmp9u5m9ths /work/build
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> > >>>> 2019-04-01 15:37:22 - INFO     - /repo$ git rev-parse --show-toplevel
> > >>>> 2019-04-01 15:37:22 - INFO     - Using /repo as root for repository cip-core
> > >>>> 2019-04-01 15:37:22 - INFO     - /work/build$ /work/isar/bitbake/bin/bitbake -k -c build
> > cip-core-image
> > >>>> Loading cache: 100%
> > >>>>
> > >>
> >
> |#################################################################
> > >>
> >
> #################################################################
> > >> ####| Time: 0:00:00 Loaded 19 entries from dependency cache.
> > >>>> NOTE: Resolving any missing task queue dependencies Initialising
> > >>>> tasks: 100%
> > >>>>
> > >>
> >
> |#################################################################
> > >> ####
> > >>>>
> ############################################################|
> > >> Time:
> > >>>> 0:00:00
> > >>>> NOTE: Executing RunQueue Tasks
> > >>>> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: Function failed:
> > >>>> do_bootstrap (log file is located at
> > >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> > >>>> bootstrap.77)
> > >>>> ERROR: Logfile of failure stored in:
> > >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.do_
> > >>>> bootstrap.77
> > >>>> Log data follows:
> > >>>> | DEBUG: Executing shell function do_bootstrap
> > >>>> | W: Target architecture is the same as host architecture; disabling
> > >>>> | QEMU support
> > >>>> | I: Running command: debootstrap --arch amd64 --verbose
> > >>>> | --variant=minbase --include=locales
> > >>>> | --components=main,contrib,non-free stretch
> > >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs
> > >>>> | http://ftp.de.debian.org/debian
> > >>>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
> > >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs/tes
> > >>>> | t-dev-null: Permission denied
> > >>>> | E: Cannot install into target
> > >>>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/rootfs'
> > >>>> | mounted with noexec or nodev
> > >>>
> > >>> Hmm, there are more strange errors. Could it be that you forgot the "--isar"
> > >>> after "kas-docker"? That would mean you are trying to run unprivileged which is doomed to fail.
> > >>>
> > >>>> | WARNING: exit code 1 from a shell command.
> > >>>> | ERROR: Function failed: do_bootstrap (log file is located at
> > >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-target/temp/log.d
> > >>>> | o_bootstrap.77)
> > >>>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap)
> > failed
> > >> with exit code '1'
> > >>>> ERROR: isar-bootstrap-host-1.0-r0 do_bootstrap: Function failed:
> > >>>> do_bootstrap (log file is located at
> > >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> > >>>> -amd64/temp/log.do_bootstrap.79)
> > >>>> ERROR: Logfile of failure stored in:
> > >>>> /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stretch
> > >>>> -amd64/temp/log.do_bootstrap.79
> > >>>> Log data follows:
> > >>>> | DEBUG: Executing shell function do_bootstrap
> > >>>> | W: Target architecture is the same as host architecture; disabling
> > >>>> | QEMU support
> > >>>> | I: Running command: debootstrap --arch amd64 --verbose
> > >>>> | --variant=minbase --include=locales
> > >>>> | --components=main,contrib,non-free stretch
> > >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> > >>>> | ch-amd64/rootfs http://ftp.de.debian.org/debian
> > >>>> | /usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create
> > >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> > >>>> | ch-amd64/rootfs/test-dev-null: Permission denied
> > >>>> | E: Cannot install into target
> > >>>> | '/work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stre
> > >>>> | tch-amd64/rootfs' mounted with noexec or nodev
> > >>>> | WARNING: exit code 1 from a shell command.
> > >>>> | ERROR: Function failed: do_bootstrap (log file is located at
> > >>>> | /work/build/tmp/work/cip-core-amd64/isar-bootstrap-host-debian-stret
> > >>>> | ch-amd64/temp/log.do_bootstrap.79)
> > >>>> ERROR: Task (/work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb:do_bootstrap)
> failed
> > >> with exit code '1'
> > >>>>
> > >>>> $ systemctl status systemd-binfmt.service ● systemd-binfmt.service -
> > >>>> Set Up Additional Binary Formats
> > >>>>       Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
> > >>>>       Active: inactive (dead)
> > >>>> Condition: start condition failed at Tue 2019-04-02 08:04:31 +07; 2h 1min ago
> > >>>>               none of the trigger conditions were met
> > >>>>         Docs: man:systemd-binfmt.service(8)
> > >>>>               man:binfmt.d(5)
> > >>>>               https://www.kernel.org/doc/Documentation/binfmt_misc.txt
> > >>>>
> > >>>
> > >>> Jan
> > >>>
> > >>> --
> > >>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
> > >>>
> > >>
> > >> --
> > >> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> > >> Corporate Competence Center Embedded Linux
> >
> > --
> > Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> > Corporate Competence Center Embedded Linux
> _______________________________________________
> cip-dev mailing list
> cip-dev@lists.cip-project.org
> https://lists.cip-project.org/mailman/listinfo/cip-dev

Re: isar error log

[Jan Kiszka]

On 11.04.19 12:13, daniel.sangorrin@toshiba.co.jp wrote:
>> -----Original Message-----
>> From: cip-dev-bounces@lists.cip-project.org <cip-dev-bounces@lists.cip-project.org> On Behalf Of
>> daniel.sangorrin@toshiba.co.jp
>> Sent: Thursday, April 11, 2019 7:06 PM
>> To: jan.kiszka@siemens.com
>> Cc: kas-devel@googlegroups.com; isar-users@googlegroups.com; cip-dev@lists.cip-project.org
>> Subject: Re: [cip-dev] isar error log
>>
>>> -----Original Message-----
>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>> Sent: Thursday, April 11, 2019 6:16 PM
>>> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ) <daniel.sangorrin@toshiba.co.jp>
>>> Cc: cip-dev@lists.cip-project.org; isar-users@googlegroups.com; kas-devel@googlegroups.com
>>> Subject: Re: isar error log
>>>
>>> On 11.04.19 11:08, daniel.sangorrin@toshiba.co.jp wrote:
>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>> On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:
>>>>>> Hi Jan,
>>>>>>
>>>>>> Yes I am following the Readme.
>>>>>>
>>>>>> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I am using kas-docker script). And I am
>> using
>>>>>
>>>>> That's the same version I have, and it works for me.
>>>>>
>>>>>> kas-docker --isar build kas.yml:board-qemu-amd64.yml
>>>>>>
>>>>>> Yes, it seems there are multiple errors:
>>>>>> /proc/sys/fs/binfmt_misc/register: Invalid argument
>>>>>> /test-dev-null: Permission denied
>>>>>>
>>>>>> Probably this is a bug or misconfiguration in Ubuntu. I will try to figure it out.
>>>>>>
>>>>>
>>>>> Adding kas-devel: Would be interesting to understand this so that we can fix it
>>>>> or at least document a workaround. Maybe you need to be in some privileged group
>>>>> in order to use privileged docker containers? That's how things work on SUSE.
>>>>>
>>>>> No one around is using Ubuntu, so that distro is likely a blind spot for our
>>>>> testing.
>>>>
>>>> OK, so I figured out that Ubuntu 16.04 was using the service "binfmt-support", not "systemd-binfmt".
>>>> For that reason, the "systemd-binfmt" service was inactive. However, "binfmt-support" is working fine.
>>>> I also checked that the post-install script of the qemu-user-static package had run correctly
>>>> and there were binfmt configuration files at /var/lib/binfmts/
>>>
>>> But what prevented the container to tune the host kernel according to its needs?
>>> That is the intended resolution for the host not having the same settings as the
>>> container (or any at all).
>>
>> To be honest I don't really know what binfmt is all about yet, so I can't even understand what you mean about
>> "tuning the kernel".

https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html

>>
>> By the way, I managed to update to the latest Ubuntu 4.4 kernel (145) and the problem persists. I tried to reinstall
>> qemu-user-static etc but still fails humm
>> .
>> Also i just tested on Ubuntu 18.04 and there I can build with ISAR without problems. So I might just leave it
>> unsolved and update to 18.04...
> 
> This might be a hint.
> On Ubuntu 18.04 I can see "qemu-alpha" on /proc/sys/fs/binfmt_misc, but on Ubuntu 16.04 it isn't there.
> When running kas-docker, the first error is about not being able to close /proc/sys/fs/binfmt_misc/register (invalid argument)
> and the second is about qemu-alpha.
> 

Maybe /proc/sys/fs/binfmt_misc is not mounted into the container under Ubuntu 
16.04, even if that is privileged.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: isar error log

[Henning Schild]

Am Thu, 11 Apr 2019 14:42:03 +0200
schrieb "[ext] Jan Kiszka" <jan.kiszka@siemens.com>:

> On 11.04.19 12:13, daniel.sangorrin@toshiba.co.jp wrote:
> >> -----Original Message-----
> >> From: cip-dev-bounces@lists.cip-project.org
> >> <cip-dev-bounces@lists.cip-project.org> On Behalf Of
> >> daniel.sangorrin@toshiba.co.jp Sent: Thursday, April 11, 2019 7:06
> >> PM To: jan.kiszka@siemens.com
> >> Cc: kas-devel@googlegroups.com; isar-users@googlegroups.com;
> >> cip-dev@lists.cip-project.org Subject: Re: [cip-dev] isar error log
> >>  
> >>> -----Original Message-----
> >>> From: Jan Kiszka <jan.kiszka@siemens.com>
> >>> Sent: Thursday, April 11, 2019 6:16 PM
> >>> To: sangorrin daniel(サンゴリン ダニエル ○ＳＷＣ□ＯＳＴ)
> >>> <daniel.sangorrin@toshiba.co.jp> Cc:
> >>> cip-dev@lists.cip-project.org; isar-users@googlegroups.com;
> >>> kas-devel@googlegroups.com Subject: Re: isar error log
> >>>
> >>> On 11.04.19 11:08, daniel.sangorrin@toshiba.co.jp wrote:  
> >>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
> >>>>> On 03.04.19 02:56, daniel.sangorrin@toshiba.co.jp wrote:  
> >>>>>> Hi Jan,
> >>>>>>
> >>>>>> Yes I am following the Readme.
> >>>>>>
> >>>>>> The docker image id for kasproject/kas-isar is 9e9fe44e68d3 (I
> >>>>>> am using kas-docker script). And I am  
> >> using  
> >>>>>
> >>>>> That's the same version I have, and it works for me.
> >>>>>  
> >>>>>> kas-docker --isar build kas.yml:board-qemu-amd64.yml
> >>>>>>
> >>>>>> Yes, it seems there are multiple errors:
> >>>>>> /proc/sys/fs/binfmt_misc/register: Invalid argument
> >>>>>> /test-dev-null: Permission denied
> >>>>>>
> >>>>>> Probably this is a bug or misconfiguration in Ubuntu. I will
> >>>>>> try to figure it out. 
> >>>>>
> >>>>> Adding kas-devel: Would be interesting to understand this so
> >>>>> that we can fix it or at least document a workaround. Maybe you
> >>>>> need to be in some privileged group in order to use privileged
> >>>>> docker containers? That's how things work on SUSE.
> >>>>>
> >>>>> No one around is using Ubuntu, so that distro is likely a blind
> >>>>> spot for our testing.  
> >>>>
> >>>> OK, so I figured out that Ubuntu 16.04 was using the service
> >>>> "binfmt-support", not "systemd-binfmt". For that reason, the
> >>>> "systemd-binfmt" service was inactive. However, "binfmt-support"
> >>>> is working fine. I also checked that the post-install script of
> >>>> the qemu-user-static package had run correctly and there were
> >>>> binfmt configuration files at /var/lib/binfmts/  
> >>>
> >>> But what prevented the container to tune the host kernel
> >>> according to its needs? That is the intended resolution for the
> >>> host not having the same settings as the container (or any at
> >>> all).  
> >>
> >> To be honest I don't really know what binfmt is all about yet, so
> >> I can't even understand what you mean about "tuning the kernel".  
> 
> https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
> 
> >>
> >> By the way, I managed to update to the latest Ubuntu 4.4 kernel
> >> (145) and the problem persists. I tried to reinstall
> >> qemu-user-static etc but still fails humm .
> >> Also i just tested on Ubuntu 18.04 and there I can build with ISAR
> >> without problems. So I might just leave it unsolved and update to
> >> 18.04...  
> > 
> > This might be a hint.
> > On Ubuntu 18.04 I can see "qemu-alpha" on /proc/sys/fs/binfmt_misc,
> > but on Ubuntu 16.04 it isn't there. When running kas-docker, the
> > first error is about not being able to
> > close /proc/sys/fs/binfmt_misc/register (invalid argument) and the
> > second is about qemu-alpha. 
> 
> Maybe /proc/sys/fs/binfmt_misc is not mounted into the container
> under Ubuntu 16.04, even if that is privileged.

Just had a look at a ubuntu 16.04. It does support the fs but it is
not mounted. So a pseudo-code patch for the docker entry could be:

if not mounted
 if grep binfmt_misc /proc/filesystems
  mount
 else
  echo "Get a real and recent distro"
 fi
fi

Henning

> Jan
> 

Re: [cip-dev] isar error log

[Ben Hutchings]

On Fri, 2019-04-12 at 13:37 +0200, Henning Schild wrote:
[...]
> Just had a look at a ubuntu 16.04. It does support the fs but it is
> not mounted. So a pseudo-code patch for the docker entry could be:
> 
> if not mounted
>  if grep binfmt_misc /proc/filesystems
>   mount
>  else
>   echo "Get a real and recent distro"
>  fi
> fi

Actual code should be something like:

if ! mountpoint -q /proc/sys/fs/binfmt_misc \
   && ! mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc; then
    echo >&2 "E: Unable to mount binfmt_misc"
    exit 1
fi

Ben.

-- 
Ben Hutchings, Software Developer                         Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom

RE: [cip-dev] isar error log

[daniel.sangorrin]

Henning, Ben, Jan: thanks for your answers.

Actually, in my PC (Ubuntu 16.04.6) binfmt_misc is mounted, both outside and inside the container.
# Henning: not sure why yours wasn't but it could be because I have also qemu-user-static installed on the host OS or because of something I installed in the past.

I tried to dig a bit deeper.

- The error:
update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument
- This error appears when /kas/docker-entrypoint calls "dpkg-configure qemu-user-static"
- I added a set -x to qemu-user-static's postinstall script (/var/lib/dpkg/info/qemu-user-static.postinst) and the error comes from this line:
update-binfmts --package qemu-user-static --install qemu-aarch64 ... --credential yes --fix-binary yes
- I tried to run update-binfmts (with the same arguments) and I could reproduce the problem
- If I remove the "--fix-binary yes" part and execute it twice, then the first time fails with the same error but the second time it succeeds
- I straced update-binfmts with --fix-binary yes
  - it opens /proc/sys/fs/binfmt_misc/register
  - writes :qemu-aarch64:M:0:x7fx45x4cx46x0"
  - returns EINVAL

And then I found the real reason of the problem:
- The qemu-user-static on the host side was conflicting with the one inside docker.
- If I remove qemu-user-static from the host then, it works.

Thanks,
Daniel

> -----Original Message-----
> From: cip-dev-bounces@lists.cip-project.org <cip-dev-bounces@lists.cip-
> project.org> On Behalf Of Ben Hutchings
> Sent: Friday, April 12, 2019 9:00 PM
> To: Henning Schild <henning.schild@siemens.com>; [ext] Jan Kiszka
> <jan.kiszka@siemens.com>
> Cc: kas-devel@googlegroups.com; isar-users@googlegroups.com; cip-
> dev@lists.cip-project.org
> Subject: Re: [cip-dev] isar error log
> 
> On Fri, 2019-04-12 at 13:37 +0200, Henning Schild wrote:
> [...]
> > Just had a look at a ubuntu 16.04. It does support the fs but it is
> > not mounted. So a pseudo-code patch for the docker entry could be:
> >
> > if not mounted
> >  if grep binfmt_misc /proc/filesystems
> >   mount
> >  else
> >   echo "Get a real and recent distro"
> >  fi
> > fi
> 
> Actual code should be something like:
> 
> if ! mountpoint -q /proc/sys/fs/binfmt_misc \
>    && ! mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc; then
>     echo >&2 "E: Unable to mount binfmt_misc"
>     exit 1
> fi
> 
> Ben.
> 
> --
> Ben Hutchings, Software Developer                         Codethink Ltd
> https://www.codethink.co.uk/                 Dale House, 35 Dale Street
>                                      Manchester, M1 2HF, United Kingdom
> _______________________________________________
> cip-dev mailing list
> cip-dev@lists.cip-project.org
> https://lists.cip-project.org/mailman/listinfo/cip-dev

RE: [cip-dev] isar error log

[daniel.sangorrin]

> -----Original Message-----
> From: cip-dev-bounces@lists.cip-project.org <cip-dev-bounces@lists.cip-
> project.org> On Behalf Of daniel.sangorrin@toshiba.co.jp
> Sent: Tuesday, April 16, 2019 11:47 AM
> To: ben.hutchings@codethink.co.uk; henning.schild@siemens.com;
> jan.kiszka@siemens.com
> Cc: kas-devel@googlegroups.com; isar-users@googlegroups.com; cip-
> dev@lists.cip-project.org
> Subject: Re: [cip-dev] isar error log
> 
> Henning, Ben, Jan: thanks for your answers.
> 
> Actually, in my PC (Ubuntu 16.04.6) binfmt_misc is mounted, both outside
> and inside the container.
> # Henning: not sure why yours wasn't but it could be because I have also
> qemu-user-static installed on the host OS or because of something I installed
> in the past.
> 
> I tried to dig a bit deeper.
> 
> - The error:
> update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register:
> Invalid argument
> - This error appears when /kas/docker-entrypoint calls "dpkg-configure
> qemu-user-static"
> - I added a set -x to qemu-user-static's postinstall script
> (/var/lib/dpkg/info/qemu-user-static.postinst) and the error comes from this
> line:
> update-binfmts --package qemu-user-static --install qemu-aarch64 ... --
> credential yes --fix-binary yes
> - I tried to run update-binfmts (with the same arguments) and I could
> reproduce the problem
> - If I remove the "--fix-binary yes" part and execute it twice, then the first
> time fails with the same error but the second time it succeeds
> - I straced update-binfmts with --fix-binary yes
>   - it opens /proc/sys/fs/binfmt_misc/register
>   - writes :qemu-aarch64:M:0:x7fx45x4cx46x0"
>   - returns EINVAL
> 
> And then I found the real reason of the problem:
> - The qemu-user-static on the host side was conflicting with the one inside
> docker.
> - If I remove qemu-user-static from the host then, it works.

Oh no, I thought it was working but I was looking at the wrong screen. This doesn't seem to be the reason. I will dig a bit more.

Thanks,
Daniel

> 
> Thanks,
> Daniel
> 
> > -----Original Message-----
> > From: cip-dev-bounces@lists.cip-project.org
> > <cip-dev-bounces@lists.cip- project.org> On Behalf Of Ben Hutchings
> > Sent: Friday, April 12, 2019 9:00 PM
> > To: Henning Schild <henning.schild@siemens.com>; [ext] Jan Kiszka
> > <jan.kiszka@siemens.com>
> > Cc: kas-devel@googlegroups.com; isar-users@googlegroups.com; cip-
> > dev@lists.cip-project.org
> > Subject: Re: [cip-dev] isar error log
> >
> > On Fri, 2019-04-12 at 13:37 +0200, Henning Schild wrote:
> > [...]
> > > Just had a look at a ubuntu 16.04. It does support the fs but it is
> > > not mounted. So a pseudo-code patch for the docker entry could be:
> > >
> > > if not mounted
> > >  if grep binfmt_misc /proc/filesystems
> > >   mount
> > >  else
> > >   echo "Get a real and recent distro"
> > >  fi
> > > fi
> >
> > Actual code should be something like:
> >
> > if ! mountpoint -q /proc/sys/fs/binfmt_misc \
> >    && ! mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc; then
> >     echo >&2 "E: Unable to mount binfmt_misc"
> >     exit 1
> > fi
> >
> > Ben.
> >
> > --
> > Ben Hutchings, Software Developer                         Codethink Ltd
> > https://www.codethink.co.uk/                 Dale House, 35 Dale Street
> >                                      Manchester, M1 2HF, United
> > Kingdom _______________________________________________
> > cip-dev mailing list
> > cip-dev@lists.cip-project.org
> > https://lists.cip-project.org/mailman/listinfo/cip-dev
> _______________________________________________
> cip-dev mailing list
> cip-dev@lists.cip-project.org
> https://lists.cip-project.org/mailman/listinfo/cip-dev