[PATCH v7 0/5] Fix usage of additional apt keys and repos

[PATCH v7 0/5] Fix usage of additional apt keys and repos

[Andreas J. Reichel]

From: Andreas Reichel <andreas.reichel.ext@siemens.com>

Diff to v6: There was a call to 'apt-key update' that was always
executed. But apt-key needs gnupg, which is only installed if we install
apt-keys. Second, this call is deprecated anyway, so it does not make
sense and we don't need it. Removed it.

Sorry for this.

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Andreas Reichel (5):
  Revert "isar-bootstrap: Allow to set local keys in DISTRO_APT_KEYS"
  Remove duplicate code from apt-keyring generation
  Fix fetched key location in apt-keyring generator
  Use apt-key to generate apt-keyring
  If we use a custom keyring debootstrap may fall to https

 .../conf/multiconfig/qemuamd64-buster.conf    |  1 -
 .../conf/multiconfig/qemuamd64-jessie.conf    |  1 -
 meta/conf/bitbake.conf                        |  1 +
 .../isar-bootstrap/isar-bootstrap.inc         | 62 +++++++++++++------
 4 files changed, 45 insertions(+), 20 deletions(-)

-- 
2.21.0

[PATCH v7 1/5] Revert "isar-bootstrap: Allow to set local keys in DISTRO_APT_KEYS"

[Andreas J. Reichel]

From: Andreas Reichel <andreas.reichel.ext@siemens.com>

This reverts commit af983a13b6f4cee5d4af5e5cf6318231e02775c9.

This commit broke usage of remote keys, where they usually come from.
If fetching "http://example.com/dir1/dir2/key", the file is fetched
into the subdir WORKDIR/dir1/dir2/, which breaks with this code.
However it succeeds with absolute paths.
We do not want to guess where the downloaded file will be. This does
not work anymore if the key is downloaded from remote with a URL.
Furthermore, a user could specify "subdir" as fetcher parameter
or other things, which break this.

This is really fixed in a follow-up commit.

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index c1b571a..2910eea 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -40,8 +40,9 @@ python () {
         d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
         for key in distro_apt_keys.split():
             url = urlparse(key)
-            d.appendVar("SRC_URI", " " + key)
-            d.appendVar("APTKEYFILES", " " + wd + url.path)
+            filename = os.path.basename(url.path)
+            d.appendVar("SRC_URI", " %s" % key)
+            d.appendVar("APTKEYFILES", " %s" % filename)
     if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')):
         own_pub_key = d.getVar("BASE_REPO_KEY", False)
         if own_pub_key:
-- 
2.21.0

[PATCH v7 2/5] Remove duplicate code from apt-keyring generation

[Andreas J. Reichel]

From: Andreas Reichel <andreas.reichel.ext@siemens.com>

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
---
 .../isar-bootstrap/isar-bootstrap.inc         | 23 +++++++++----------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 2910eea..4526aa7 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -35,22 +35,21 @@ inherit base-apt-helper
 python () {
     from urllib.parse import urlparse
     distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False)
-    wd = d.getVar("WORKDIR", True)
+    aptkeys = []
+
     if distro_apt_keys:
-        d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
-        for key in distro_apt_keys.split():
-            url = urlparse(key)
-            filename = os.path.basename(url.path)
-            d.appendVar("SRC_URI", " %s" % key)
-            d.appendVar("APTKEYFILES", " %s" % filename)
+        aptkeys += distro_apt_keys.split()
     if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')):
         own_pub_key = d.getVar("BASE_REPO_KEY", False)
         if own_pub_key:
-            d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
-            for key in own_pub_key.split():
-                url = urlparse(key)
-                d.appendVar("SRC_URI", " " + key)
-                d.appendVar("APTKEYFILES", " " + wd + url.path)
+            aptkeys += own_pub_key.split()
+
+    d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
+    for key in aptkeys:
+        url = urlparse(key)
+        filename = os.path.basename(url.path)
+        d.appendVar("SRC_URI", " %s" % key)
+        d.appendVar("APTKEYFILES", " %s" % filename)
 }
 
 def aggregate_files(d, file_list, file_out):
-- 
2.21.0

[PATCH v7 3/5] Fix fetched key location in apt-keyring generator

[Andreas J. Reichel]

From: Andreas Reichel <andreas.reichel.ext@siemens.com>

Use bb.fetch2.Fetch to retrieve the final filename after it is
downloaded. This way we don't have to guess (wrongly), and also
additional SRC_URI parameters like subdir or filename are usable now.

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 4526aa7..1f8f178 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -33,7 +33,6 @@ DISTRO_APT_PREMIRRORS ?= "${@ "http://ftp\.(\S+\.)?debian.org  file:///${REPO_BA
 inherit base-apt-helper
 
 python () {
-    from urllib.parse import urlparse
     distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False)
     aptkeys = []
 
@@ -46,9 +45,9 @@ python () {
 
     d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
     for key in aptkeys:
-        url = urlparse(key)
-        filename = os.path.basename(url.path)
         d.appendVar("SRC_URI", " %s" % key)
+        fetcher = bb.fetch2.Fetch([key], d)
+        filename = fetcher.localpath(key)
         d.appendVar("APTKEYFILES", " %s" % filename)
 }
 
-- 
2.21.0

[PATCH v7 4/5] Use apt-key to generate apt-keyring

[Andreas J. Reichel]

From: Andreas Reichel <andreas.reichel.ext@siemens.com>

Use apt-key instead of manually calling gpg.

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
---
 meta/conf/bitbake.conf                        |  1 +
 .../isar-bootstrap/isar-bootstrap.inc         | 30 ++++++++++++++-----
 2 files changed, 24 insertions(+), 7 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0e521bb..769ec9a 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -62,6 +62,7 @@ DEBDISTRONAME = "isar"
 # Isar apt repository paths
 REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt/apt"
 REPO_ISAR_DB_DIR = "${DEPLOY_DIR}/isar-apt/db"
+ISARKEYRING = "/etc/apt/trusted.gpg.d/isar.gpg"
 
 # Base apt repository paths
 REPO_BASE_DIR = "${DL_DIR}/base-apt/apt"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 1f8f178..455a4a1 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -23,10 +23,9 @@ APTSRCS = "${WORKDIR}/apt-sources"
 APTSRCS_INIT = "${WORKDIR}/apt-sources-init"
 BASEAPTSRCS = "${WORKDIR}/base-apt-sources"
 APTKEYFILES = ""
-APTKEYRING = "${WORKDIR}/apt-keyring.gpg"
-DEBOOTSTRAP_KEYRING = ""
 DEPLOY_ISAR_BOOTSTRAP ?= ""
 DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
+DISTRO_BOOTSTRAP_BASE_PACKAGES_append_gnupg = ",gnupg2"
 
 DISTRO_APT_PREMIRRORS ?= "${@ "http://ftp\.(\S+\.)?debian.org  file:///${REPO_BASE_DIR} \n" if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else "" }"
 
@@ -43,7 +42,6 @@ python () {
         if own_pub_key:
             aptkeys += own_pub_key.split()
 
-    d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
     for key in aptkeys:
         d.appendVar("SRC_URI", " %s" % key)
         fetcher = bb.fetch2.Fetch([key], d)
@@ -158,6 +156,14 @@ def get_distro_needs_https_support(d, is_host=False):
     else:
         return ""
 
+def get_distro_needs_gpg_support(d):
+    apt_keys = d.getVar("HAVE_CUSTOM_APT_KEYS", False)
+    if apt_keys:
+        return "gnupg"
+    return ""
+
+OVERRIDES_append = ":${@get_distro_needs_gpg_support(d)}"
+
 def get_distro_source(d, is_host):
     return get_distro_primary_source_entry(d, is_host)[0]
 
@@ -171,13 +177,17 @@ def get_distro_components_argument(d, is_host):
     else:
         return ""
 
+APT_KEYS_DIR := "${WORKDIR}/aptkeys"
+
+do_generate_keyring[cleandirs] = "${APT_KEYS_DIR}"
 do_generate_keyring[dirs] = "${DL_DIR}"
 do_generate_keyring[vardeps] += "DISTRO_APT_KEYS"
 do_generate_keyring() {
     if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then
+        chmod 777 "${APT_KEYS_DIR}"
         for keyfile in ${@d.getVar("APTKEYFILES", True)}; do
-           gpg --no-default-keyring --keyring "${APTKEYRING}" \
-               --no-tty --homedir "${DL_DIR}"  --import "$keyfile"
+           cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")"
+           sudo apt-key --keyring "${ISARKEYRING}" add "$keyfile"
         done
     fi
 }
@@ -221,7 +231,6 @@ isar_bootstrap() {
             if [ ${IS_HOST} ]; then
                 ${DEBOOTSTRAP} $debootstrap_args \
                                ${@get_distro_components_argument(d, True)} \
-                               ${DEBOOTSTRAP_KEYRING} \
                                "${@get_distro_suite(d, True)}" \
                                "${ROOTFSDIR}" \
                                "${@get_distro_source(d, True)}"
@@ -230,7 +239,6 @@ isar_bootstrap() {
                  "${DEBOOTSTRAP}" $debootstrap_args \
                                   --arch="${DISTRO_ARCH}" \
                                   ${@get_distro_components_argument(d, False)} \
-                                  ${DEBOOTSTRAP_KEYRING} \
                                   "${@get_distro_suite(d, False)}" \
                                   "${ROOTFSDIR}" \
                                   "${@get_distro_source(d, False)}"
@@ -259,6 +267,14 @@ isar_bootstrap() {
             mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d"
             install -v -m644 "${WORKDIR}/isar-apt.conf" \
                              "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf"
+            find ${APT_KEYS_DIR}/ -type f | while read keyfile
+            do
+                kfn="$(basename $keyfile)"
+                cp $keyfile "${ROOTFSDIR}/tmp/$kfn"
+                sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-key \
+                   --keyring ${ISARKEYRING} add "/tmp/$kfn"
+                rm "${ROOTFSDIR}/tmp/$kfn"
+            done
 
             if [ "${@get_distro_suite(d, True)}" = "stretch" ] && [ "${@get_host_release().split('.')[0]}" -lt "4" ]; then
                 install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \
-- 
2.21.0

[PATCH v7 5/5] If we use a custom keyring debootstrap may fall to https

[Andreas J. Reichel]

From: Andreas Reichel <andreas.reichel.ext@siemens.com>

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891009

So if we have something in aptkeyring, append https-support to
OVERRIDES.

Furthermore, the conditional append for https-support was missing
in qemuamd64-stretch.conf, thus, remove this from all the distros
and put it into the isar-bootstrap.inc.

Furthermore, packages are comma-, not space-separated.

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
---
 meta-isar/conf/multiconfig/qemuamd64-buster.conf    |  1 -
 meta-isar/conf/multiconfig/qemuamd64-jessie.conf    |  1 -
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 11 +++++++++++
 3 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
index 63df75c..da90993 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
@@ -18,4 +18,3 @@ QEMU_MACHINE ?= "q35"
 QEMU_CPU ?= ""
 QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE## -bios /usr/local/share/ovmf/OVMF.fd"
 
-DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
index d1335ff..42c71df 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
@@ -15,4 +15,3 @@ QEMU_MACHINE ?= "pc"
 QEMU_CPU ?= ""
 QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE##"
 
-DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 455a4a1..4e6d1f1 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -26,6 +26,7 @@ APTKEYFILES = ""
 DEPLOY_ISAR_BOOTSTRAP ?= ""
 DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_gnupg = ",gnupg2"
+DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = ",apt-transport-https,ca-certificates"
 
 DISTRO_APT_PREMIRRORS ?= "${@ "http://ftp\.(\S+\.)?debian.org  file:///${REPO_BASE_DIR} \n" if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else "" }"
 
@@ -42,6 +43,12 @@ python () {
         if own_pub_key:
             aptkeys += own_pub_key.split()
 
+    if len(aptkeys) > 0:
+        # debootstrap falls back to https if there is no
+        # 'reliable' keyring, whatever that means, but it happened
+        # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891009
+        d.setVar("HAVE_CUSTOM_APT_KEYS", "True")
+
     for key in aptkeys:
         d.appendVar("SRC_URI", " %s" % key)
         fetcher = bb.fetch2.Fetch([key], d)
@@ -151,6 +158,10 @@ def get_distro_have_https_source(d, is_host=False):
     return any(source[2].startswith("https://") for source in generate_distro_sources(d, is_host))
 
 def get_distro_needs_https_support(d, is_host=False):
+    apt_keys = d.getVar("HAVE_CUSTOM_APT_KEYS", False)
+    if apt_keys:
+        return "https-support"
+
     if get_distro_have_https_source(d, is_host):
         return "https-support"
     else:
-- 
2.21.0

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Andreas Reichel]

Hi Maxim,

this succeeded both on siemens CI without additional repos and locally
with additional repo. Hope this is fine now.

vG
Andreas
-- 
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

Hi Andreas,

Ok. Great.

Have you tried to run my example  with apt repo caching feature enabled 
'bitbake -c cache_base_repo multiconfig:qemuamd64-stretch:isar-image-base'
(with signing and w/o signing (default) the repo)?

Thanks,
Maxim.


On 3/19/19 2:36 PM, Andreas Reichel wrote:
> Hi Maxim,
> 
> this succeeded both on siemens CI without additional repos and locally
> with additional repo. Hope this is fine now.
> 
> vG
> Andreas
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Andreas Reichel]

On Tue, Mar 19, 2019 at 03:26:34PM +0100, Maxim Yu. Osipov wrote:
> Hi Andreas,
> 
> Ok. Great.
> 
> Have you tried to run my example  with apt repo caching feature enabled
> 'bitbake -c cache_base_repo multiconfig:qemuamd64-stretch:isar-image-base'
> (with signing and w/o signing (default) the repo)?

I have successfully run your command with and without BASE_REPO_KEY set.

> 
> Thanks,
> Maxim.
> 
> 
> On 3/19/19 2:36 PM, Andreas Reichel wrote:
> > Hi Maxim,
> > 
> > this succeeded both on siemens CI without additional repos and locally
> > with additional repo. Hope this is fine now.
> > 
> > vG
> > Andreas
> > 
> 
> 
> -- 
> Maxim Osipov
> ilbers GmbH
> Maria-Merian-Str. 8
> 85521 Ottobrunn
> Germany
> +49 (151) 6517 6917
> mosipov@ilbers.de
> http://ilbers.de/
> Commercial register Munich, HRB 214197
> General Manager: Baurzhan Ismagulov

-- 
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

Hi Andreas,

CI build ('scripts/ci_build.sh -q') failed. See
http://isar-build.org:8080/job/isar_mosipov_next/165/console

We may create project for you on isar-build.org so you can run CI build 
on isar-build.org before submitting patches to the mailing list.
We just need your github repo name (like
https://github.com/henning-schild-work/isar/)

Thanks,
Maxim.

On 3/19/19 2:35 PM, Andreas J. Reichel wrote:
> From: Andreas Reichel <andreas.reichel.ext@siemens.com>
> 
> Diff to v6: There was a call to 'apt-key update' that was always
> executed. But apt-key needs gnupg, which is only installed if we install
> apt-keys. Second, this call is deprecated anyway, so it does not make
> sense and we don't need it. Removed it.
> 
> Sorry for this.
> 
> Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
> 
> Andreas Reichel (5):
>    Revert "isar-bootstrap: Allow to set local keys in DISTRO_APT_KEYS"
>    Remove duplicate code from apt-keyring generation
>    Fix fetched key location in apt-keyring generator
>    Use apt-key to generate apt-keyring
>    If we use a custom keyring debootstrap may fall to https
> 
>   .../conf/multiconfig/qemuamd64-buster.conf    |  1 -
>   .../conf/multiconfig/qemuamd64-jessie.conf    |  1 -
>   meta/conf/bitbake.conf                        |  1 +
>   .../isar-bootstrap/isar-bootstrap.inc         | 62 +++++++++++++------
>   4 files changed, 45 insertions(+), 20 deletions(-)
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Jan Kiszka]

On 19.03.19 22:58, Maxim Yu. Osipov wrote:
> Hi Andreas,
> 
> CI build ('scripts/ci_build.sh -q') failed. See
> http://isar-build.org:8080/job/isar_mosipov_next/165/console
> 

This is the same error I saw yesterday ("..isar-bootstrap-target/rootfs/dev: not 
mounted"). And given that this and my build had no overlap in our own patches, 
it must be something else you should examine because it may affect your patches 
(next..e13be9cefd87).

> We may create project for you on isar-build.org so you can run CI build on 
> isar-build.org before submitting patches to the mailing list.
> We just need your github repo name (like
> https://github.com/henning-schild-work/isar/)

We rather need to find a way to make CI public, the current state is rather 
suboptimal. It's also not very well integrated with github.

In fact, we need to find a way to build Isar in the cloud in general, also for 
use cases such as https://gitlab.com/cip-project/cip-core/isar-cip-core. And our 
internal CI is not scaling either. We do know how to do that better 
(container-based builds, controlled e.g. by gitlab-ci), but not yet how to 
deploy economically.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

Hi Jan,

On 3/20/19 7:26 AM, Jan Kiszka wrote:
> On 19.03.19 22:58, Maxim Yu. Osipov wrote:
>> Hi Andreas,
>>
>> CI build ('scripts/ci_build.sh -q') failed. See
>> http://isar-build.org:8080/job/isar_mosipov_next/165/console
>>
> 
> This is the same error I saw yesterday 
> ("..isar-bootstrap-target/rootfs/dev: not mounted"). And given that this 
> and my build had no overlap in our own patches, it must be something 
> else you should examine because it may affect your patches 
> (next..e13be9cefd87).

Yep, when I tested my patches CI build passed OK,
but now I see the same error in overnight build of 'next'.


>> We may create project for you on isar-build.org so you can run CI 
>> build on isar-build.org before submitting patches to the mailing list.
>> We just need your github repo name (like
>> https://github.com/henning-schild-work/isar/)
> 
> We rather need to find a way to make CI public, the current state is 
> rather suboptimal. It's also not very well integrated with github.
> 
> In fact, we need to find a way to build Isar in the cloud in general, 
> also for use cases such as 
> https://gitlab.com/cip-project/cip-core/isar-cip-core. And our internal 
> CI is not scaling either. We do know how to do that better 
> (container-based builds, controlled e.g. by gitlab-ci), but not yet how 
> to deploy economically.

Agreed. To be honest I'm not a big fan of gitlab - it's rather slow.
Do you have any suggestions regarding other alternatives to gitlab/jenkins?

Maxim.

> 
> Jan
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Jan Kiszka]

On 20.03.19 07:48, Maxim Yu. Osipov wrote:
> Hi Jan,
> 
> On 3/20/19 7:26 AM, Jan Kiszka wrote:
>> On 19.03.19 22:58, Maxim Yu. Osipov wrote:
>>> Hi Andreas,
>>>
>>> CI build ('scripts/ci_build.sh -q') failed. See
>>> http://isar-build.org:8080/job/isar_mosipov_next/165/console
>>>
>>
>> This is the same error I saw yesterday ("..isar-bootstrap-target/rootfs/dev:
>> not mounted"). And given that this and my build had no overlap in our own
>> patches, it must be something else you should examine because it may affect
>> your patches (next..e13be9cefd87).
> 
> Yep, when I tested my patches CI build passed OK,
> but now I see the same error in overnight build of 'next'.
> 

OK, that makes more sense again.

> 
>>> We may create project for you on isar-build.org so you can run CI build on
>>> isar-build.org before submitting patches to the mailing list.
>>> We just need your github repo name (like
>>> https://github.com/henning-schild-work/isar/)
>>
>> We rather need to find a way to make CI public, the current state is rather
>> suboptimal. It's also not very well integrated with github.
>>
>> In fact, we need to find a way to build Isar in the cloud in general, also for
>> use cases such as https://gitlab.com/cip-project/cip-core/isar-cip-core. And
>> our internal CI is not scaling either. We do know how to do that better
>> (container-based builds, controlled e.g. by gitlab-ci), but not yet how to
>> deploy economically.
> 
> Agreed. To be honest I'm not a big fan of gitlab - it's rather slow.
> Do you have any suggestions regarding other alternatives to gitlab/jenkins?

The speed of gitlab is primarily a matter of how you deploy it (ok, when you
have >25K users and >40K projects like we do at Siemens, there are also other
limits). But this is about where to run the CI builds and tests and how to make
those results available.

Unfortunately, our builds take long and require root permissions. So we cannot
simply use any of the public CI services, that scale well but are limited in
those regards. We need hosting in VMs that we control (and likely also pay for),
but that can also be cloned by other users (copy&paste of deployment recipes).
We played with AWS as well as Google Compute and Kubernetes clusters that spawn
up powerful runners on demand (you can afford running them 24/7). These recipes
aren't complete yet, unfortunately, and tuning them takes some time. That's why
this isn't ready yet. I'll try to prioritize that topic as it is becoming a
bottleneck in many projects.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Claudius Heine]

On Wed, 2019-03-20 at 07:26 +0100, [ext] Jan Kiszka wrote:
> On 19.03.19 22:58, Maxim Yu. Osipov wrote:
> > Hi Andreas,
> > 
> > CI build ('scripts/ci_build.sh -q') failed. See
> > http://isar-build.org:8080/job/isar_mosipov_next/165/console
> > 
> 
> This is the same error I saw yesterday ("..isar-bootstrap-
> target/rootfs/dev: not 
> mounted"). And given that this and my build had no overlap in our own
> patches, 
> it must be something else you should examine because it may affect
> your patches 
> (next..e13be9cefd87).

I saw this error pretty often as well. AFAIK this is one of the errors
that can be resolved by triggering the build multiple times until the
build is green.

I guess that this is a concurrency issue and a lock is missing
somewhere.

Claudius

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

Hi Andreas, Jan,

On 3/20/19 8:12 AM, Jan Kiszka wrote:
> On 20.03.19 07:48, Maxim Yu. Osipov wrote:
>> Hi Jan,
>>
>> On 3/20/19 7:26 AM, Jan Kiszka wrote:
>>> On 19.03.19 22:58, Maxim Yu. Osipov wrote:
>>>> Hi Andreas,
>>>>
>>>> CI build ('scripts/ci_build.sh -q') failed. See
>>>> http://isar-build.org:8080/job/isar_mosipov_next/165/console
>>>>
>>>
>>> This is the same error I saw yesterday ("..isar-bootstrap-target/rootfs/dev:
>>> not mounted"). And given that this and my build had no overlap in our own
>>> patches, it must be something else you should examine because it may affect
>>> your patches (next..e13be9cefd87).
>>
>> Yep, when I tested my patches CI build passed OK,
>> but now I see the same error in overnight build of 'next'.
>>
> 
> OK, that makes more sense again.


1) I've analyzed the two last build failures in 'next' branch - the 
reason is quite stupid - some of the packages can't be downloaded:

For http://isar-build.org:8080/job/isar_next/258/console - libprocps3,
for http://isar-build.org:8080/job/isar_next/259/console - ncurses-bin.

2) As for build with Andreas v7 patch seriesthe failure is related to 
keys stuff (raspbian-jessie-armhf):

ERROR: mc:rpi-jessie:isar-bootstrap-target-1.0-r0 do_bootstrap: Function 
failed: do_bootstrap (log file is located at 
/workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
ERROR: Logfile of failure stored in: 
/workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981
Log data follows:
| DEBUG: Executing shell function do_bootstrap
| I: Running command: debootstrap --arch armhf --foreign --verbose 
--variant=minbase 
--include=locales,gnupg2,apt-transport-https,ca-certificates 
--components=main,contrib,non-free,firmware jessie 
/workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/rootfs 
http://archive.raspbian.org/raspbian
| I: Retrieving InRelease
| I: Checking Release signature
| E: Release signed by unknown key (key id 9165938D90FDDD2E)
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_bootstrap (log file is located at 
/workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed

So, Andreas, please have a look on that.

Regards,
Maxim.

>>
>>>> We may create project for you on isar-build.org so you can run CI build on
>>>> isar-build.org before submitting patches to the mailing list.
>>>> We just need your github repo name (like
>>>> https://github.com/henning-schild-work/isar/)
>>>
>>> We rather need to find a way to make CI public, the current state is rather
>>> suboptimal. It's also not very well integrated with github.
>>>
>>> In fact, we need to find a way to build Isar in the cloud in general, also for
>>> use cases such as https://gitlab.com/cip-project/cip-core/isar-cip-core. And
>>> our internal CI is not scaling either. We do know how to do that better
>>> (container-based builds, controlled e.g. by gitlab-ci), but not yet how to
>>> deploy economically.
>>
>> Agreed. To be honest I'm not a big fan of gitlab - it's rather slow.
>> Do you have any suggestions regarding other alternatives to gitlab/jenkins?
> 
> The speed of gitlab is primarily a matter of how you deploy it (ok, when you
> have >25K users and >40K projects like we do at Siemens, there are also other
> limits). But this is about where to run the CI builds and tests and how to make
> those results available.
> 
> Unfortunately, our builds take long and require root permissions. So we cannot
> simply use any of the public CI services, that scale well but are limited in
> those regards. We need hosting in VMs that we control (and likely also pay for),
> but that can also be cloned by other users (copy&paste of deployment recipes).
> We played with AWS as well as Google Compute and Kubernetes clusters that spawn
> up powerful runners on demand (you can afford running them 24/7). These recipes
> aren't complete yet, unfortunately, and tuning them takes some time. That's why
> this isn't ready yet. I'll try to prioritize that topic as it is becoming a
> bottleneck in many projects.
> 
> Jan
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Jan Kiszka]

On 20.03.19 08:55, Maxim Yu. Osipov wrote:
> Hi Andreas, Jan,
> 
> On 3/20/19 8:12 AM, Jan Kiszka wrote:
>> On 20.03.19 07:48, Maxim Yu. Osipov wrote:
>>> Hi Jan,
>>>
>>> On 3/20/19 7:26 AM, Jan Kiszka wrote:
>>>> On 19.03.19 22:58, Maxim Yu. Osipov wrote:
>>>>> Hi Andreas,
>>>>>
>>>>> CI build ('scripts/ci_build.sh -q') failed. See
>>>>> http://isar-build.org:8080/job/isar_mosipov_next/165/console
>>>>>
>>>>
>>>> This is the same error I saw yesterday ("..isar-bootstrap-target/rootfs/dev:
>>>> not mounted"). And given that this and my build had no overlap in our own
>>>> patches, it must be something else you should examine because it may affect
>>>> your patches (next..e13be9cefd87).
>>>
>>> Yep, when I tested my patches CI build passed OK,
>>> but now I see the same error in overnight build of 'next'.
>>>
>>
>> OK, that makes more sense again.
> 
> 
> 1) I've analyzed the two last build failures in 'next' branch - the reason is
> quite stupid - some of the packages can't be downloaded:
> 
> For http://isar-build.org:8080/job/isar_next/258/console - libprocps3,
> for http://isar-build.org:8080/job/isar_next/259/console - ncurses-bin.

Any idea why these downloads may fail?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

On 3/20/19 9:22 AM, Jan Kiszka wrote:
> On 20.03.19 08:55, Maxim Yu. Osipov wrote:
>> Hi Andreas, Jan,
>>
>> On 3/20/19 8:12 AM, Jan Kiszka wrote:
>>> On 20.03.19 07:48, Maxim Yu. Osipov wrote:
>>>> Hi Jan,
>>>>
>>>> On 3/20/19 7:26 AM, Jan Kiszka wrote:
>>>>> On 19.03.19 22:58, Maxim Yu. Osipov wrote:
>>>>>> Hi Andreas,
>>>>>>
>>>>>> CI build ('scripts/ci_build.sh -q') failed. See
>>>>>> http://isar-build.org:8080/job/isar_mosipov_next/165/console
>>>>>>
>>>>>
>>>>> This is the same error I saw yesterday ("..isar-bootstrap-target/rootfs/dev:
>>>>> not mounted"). And given that this and my build had no overlap in our own
>>>>> patches, it must be something else you should examine because it may affect
>>>>> your patches (next..e13be9cefd87).
>>>>
>>>> Yep, when I tested my patches CI build passed OK,
>>>> but now I see the same error in overnight build of 'next'.
>>>>
>>>
>>> OK, that makes more sense again.
>>
>>
>> 1) I've analyzed the two last build failures in 'next' branch - the reason is
>> quite stupid - some of the packages can't be downloaded:
>>
>> For http://isar-build.org:8080/job/isar_next/258/console - libprocps3,
>> for http://isar-build.org:8080/job/isar_next/259/console - ncurses-bin.
> 
> Any idea why these downloads may fail?

No idea at the moment.

debootstrap is called with '--verbose' flag to produce more info about 
downloading.

Unfortunately such extra information doesn't provide detailed (wget?) 
error codes:

I: Running command: debootstrap --arch armhf --foreign --verbose 
--variant=minbase --include=locales --components=main,contrib,non-free 
jessie 
/workspace/build/isar_next/259/build/tmp/work/debian-jessie-armhf/isar-bootstrap-target/rootfs 
http://ftp.debian.org/debian

<snip>

W: Couldn't download package ncurses-bin (ver 5.9+20140913-1+deb8u3 arch 
armhf)

<snip>

E: Couldn't download packages: ncurses-bin
WARNING: exit code 1 from a shell command.


Maxim.

-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Andreas Reichel]

On Wed, Mar 20, 2019 at 08:55:05AM +0100, Maxim Yu. Osipov wrote:
> Hi Andreas, Jan,
> 
> | I: Retrieving InRelease
> | I: Checking Release signature
> | E: Release signed by unknown key (key id 9165938D90FDDD2E)
> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_bootstrap (log file is located at /workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
> NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed
> 
> So, Andreas, please have a look on that.
> 
@Jan, seems the build on siemens CI is NOT identical to the build on
ilbers CI.

@Maxim, may I please get access to ilbers CI (again)? I had a login a
few years ago but never used it, so I don't have the login details on
the current setup.

Regards,
Andreas

> Regards,
> Maxim.
> 
> -- 
> Maxim Osipov
> ilbers GmbH
> Maria-Merian-Str. 8
> 85521 Ottobrunn
> Germany
> +49 (151) 6517 6917
> mosipov@ilbers.de
> http://ilbers.de/
> Commercial register Munich, HRB 214197
> General Manager: Baurzhan Ismagulov

-- 
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

On 3/20/19 11:25 AM, Andreas Reichel wrote:
> On Wed, Mar 20, 2019 at 08:55:05AM +0100, Maxim Yu. Osipov wrote:
>> Hi Andreas, Jan,
>>
>> | I: Retrieving InRelease
>> | I: Checking Release signature
>> | E: Release signed by unknown key (key id 9165938D90FDDD2E)
>> | WARNING: exit code 1 from a shell command.
>> | ERROR: Function failed: do_bootstrap (log file is located at /workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
>> NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed
>>
>> So, Andreas, please have a look on that.
>>
> @Jan, seems the build on siemens CI is NOT identical to the build on
> ilbers CI.
> 
> @Maxim, may I please get access to ilbers CI (again)? I had a login a
> few years ago but never used it, so I don't have the login details on
> the current setup.

Have you received yesterday the email from Bauzrzhan?

-------- Forwarded Message --------
Subject: Isar CI account
Date: Tue, 19 Mar 2019 12:03:08 +0100
From: Baurzhan Ismagulov <ibr@ilbers.de>
To: andreas.reichel@tngtech.com
CC: mosipov@ilbers.de

Hello Andreas,

I've created the user "ar" at http://ci.isar-build.org:8080/ and sent 
the password via SMS. Please change it soon.

If you'd like to build stuff, please
send me your github repo name (like
https://github.com/henning-schild-work/isar/).

With kind regards,
-- 
Baurzhan Ismagulov
ilbers GmbH
Maria-Merian-Str. 8
D-85521 Ottobrunn
+49 (89) 122 67 24-0
ibr@ilbers.de
http://ilbers.de/
Amtsgericht München, HRB 214197
Geschäftsführer: Baurzhan Ismagulov



> Regards,
> Andreas
> 
>> Regards,
>> Maxim.
>>
>> -- 
>> Maxim Osipov
>> ilbers GmbH
>> Maria-Merian-Str. 8
>> 85521 Ottobrunn
>> Germany
>> +49 (151) 6517 6917
>> mosipov@ilbers.de
>> http://ilbers.de/
>> Commercial register Munich, HRB 214197
>> General Manager: Baurzhan Ismagulov
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Jan Kiszka]

On 20.03.19 11:25, Andreas Reichel wrote:
> On Wed, Mar 20, 2019 at 08:55:05AM +0100, Maxim Yu. Osipov wrote:
>> Hi Andreas, Jan,
>>
>> | I: Retrieving InRelease
>> | I: Checking Release signature
>> | E: Release signed by unknown key (key id 9165938D90FDDD2E)
>> | WARNING: exit code 1 from a shell command.
>> | ERROR: Function failed: do_bootstrap (log file is located at /workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
>> NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed
>>
>> So, Andreas, please have a look on that.
>>
> @Jan, seems the build on siemens CI is NOT identical to the build on
> ilbers CI.
> 

We run the same script with the same parameters. But: We have different servers
(#number of cores) which can affect concurrency issues. And we have different
Internet connectivity, which can affect downloads (and also trigger concurrency
issues).

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Maxim Yu. Osipov]

On 3/20/19 11:37 AM, Jan Kiszka wrote:
> On 20.03.19 11:25, Andreas Reichel wrote:
>> On Wed, Mar 20, 2019 at 08:55:05AM +0100, Maxim Yu. Osipov wrote:
>>> Hi Andreas, Jan,
>>>
>>> | I: Retrieving InRelease
>>> | I: Checking Release signature
>>> | E: Release signed by unknown key (key id 9165938D90FDDD2E)
>>> | WARNING: exit code 1 from a shell command.
>>> | ERROR: Function failed: do_bootstrap (log file is located at /workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
>>> NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed
>>>
>>> So, Andreas, please have a look on that.
>>>
>> @Jan, seems the build on siemens CI is NOT identical to the build on
>> ilbers CI.
>>
> 
> We run the same script with the same parameters. But: We have different servers
> (#number of cores) which can affect concurrency issues. And we have different
> Internet connectivity, which can affect downloads (and also trigger concurrency
> issues).

We have debian-stretch-amd64 on isar-build.org.

Which host system do you use in Siemens CI?

Maxim.

-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Andreas Reichel]

On Wed, Mar 20, 2019 at 11:32:07AM +0100, Maxim Yu. Osipov wrote:
> On 3/20/19 11:25 AM, Andreas Reichel wrote:
> > On Wed, Mar 20, 2019 at 08:55:05AM +0100, Maxim Yu. Osipov wrote:
> > > Hi Andreas, Jan,
> > > 
> > > | I: Retrieving InRelease
> > > | I: Checking Release signature
> > > | E: Release signed by unknown key (key id 9165938D90FDDD2E)
> > > | WARNING: exit code 1 from a shell command.
> > > | ERROR: Function failed: do_bootstrap (log file is located at /workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
> > > NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed
> > > 
> > > So, Andreas, please have a look on that.
> > > 
> > @Jan, seems the build on siemens CI is NOT identical to the build on
> > ilbers CI.
> > 
> > @Maxim, may I please get access to ilbers CI (again)? I had a login a
> > few years ago but never used it, so I don't have the login details on
> > the current setup.
> 
> Have you received yesterday the email from Bauzrzhan?

Ah, on my other e-mail... yes, thank you!
> 
> -------- Forwarded Message --------
> Subject: Isar CI account
> Date: Tue, 19 Mar 2019 12:03:08 +0100
> From: Baurzhan Ismagulov <ibr@ilbers.de>
> To: andreas.reichel@tngtech.com
> CC: mosipov@ilbers.de
> 
> Hello Andreas,
> 
> I've created the user "ar" at http://ci.isar-build.org:8080/ and sent the
> password via SMS. Please change it soon.
> 
> If you'd like to build stuff, please
> send me your github repo name (like
> https://github.com/henning-schild-work/isar/).
> 
> With kind regards,
> -- 
> Baurzhan Ismagulov
> ilbers GmbH
> Maria-Merian-Str. 8
> D-85521 Ottobrunn
> +49 (89) 122 67 24-0
> ibr@ilbers.de
> http://ilbers.de/
> Amtsgericht M�nchen, HRB 214197
> Gesch�ftsf�hrer: Baurzhan Ismagulov
> 
> 
> 
> > Regards,
> > Andreas
> > 
> > > Regards,
> > > Maxim.
> > > 
> > > -- 
> > > Maxim Osipov
> > > ilbers GmbH
> > > Maria-Merian-Str. 8
> > > 85521 Ottobrunn
> > > Germany
> > > +49 (151) 6517 6917
> > > mosipov@ilbers.de
> > > http://ilbers.de/
> > > Commercial register Munich, HRB 214197
> > > General Manager: Baurzhan Ismagulov
> > 
> 
> 
> -- 
> Maxim Osipov
> ilbers GmbH
> Maria-Merian-Str. 8
> 85521 Ottobrunn
> Germany
> +49 (151) 6517 6917
> mosipov@ilbers.de
> http://ilbers.de/
> Commercial register Munich, HRB 214197
> General Manager: Baurzhan Ismagulov

-- 
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

Re: [PATCH v7 0/5] Fix usage of additional apt keys and repos

[Jan Kiszka]

On 20.03.19 12:05, Maxim Yu. Osipov wrote:
> On 3/20/19 11:37 AM, Jan Kiszka wrote:
>> On 20.03.19 11:25, Andreas Reichel wrote:
>>> On Wed, Mar 20, 2019 at 08:55:05AM +0100, Maxim Yu. Osipov wrote:
>>>> Hi Andreas, Jan,
>>>>
>>>> | I: Retrieving InRelease
>>>> | I: Checking Release signature
>>>> | E: Release signed by unknown key (key id 9165938D90FDDD2E)
>>>> | WARNING: exit code 1 from a shell command.
>>>> | ERROR: Function failed: do_bootstrap (log file is located at
>>>> /workspace/build/isar_mosipov_next/165/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap-target/temp/log.do_bootstrap.12981)
>>>>
>>>> NOTE: recipe isar-bootstrap-target-1.0-r0: task do_bootstrap: Failed
>>>>
>>>> So, Andreas, please have a look on that.
>>>>
>>> @Jan, seems the build on siemens CI is NOT identical to the build on
>>> ilbers CI.
>>>
>>
>> We run the same script with the same parameters. But: We have different servers
>> (#number of cores) which can affect concurrency issues. And we have different
>> Internet connectivity, which can affect downloads (and also trigger concurrency
>> issues).
> 
> We have debian-stretch-amd64 on isar-build.org.
> 
> Which host system do you use in Siemens CI?

The kasproject/kas-isar container, the default for Isar builds by now. The host
is a Debian 9 as well, but that plays no significant role (except for kernel
issues).

The server is currently on 16-core machine. We will try to scale up in a couple
of weeks, adding a 32-core box to the cluster. And there will be a 128-core ARM
server available, but that likely only late summer.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux