[PATCH 1/2] ci: always test signed and unsigned repo builds

[PATCH 1/2] ci: always test signed and unsigned repo builds

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Just split the target set we had into two. Test the first signed and the
other one unsigned.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 scripts/ci_build.sh | 38 +++++++++++++++++++++++---------------
 1 file changed, 23 insertions(+), 15 deletions(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ecaa88a..8295c43 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -50,13 +50,14 @@ CROSS_TARGETS_SET="\
                   mc:de0-nano-soc-stretch:isar-image-base \
                   mc:rpi-stretch:isar-image-base"
 
-REPRO_TARGETS_SET="\
+REPRO_TARGETS_SET_SIGNED="\
             mc:qemuarm-stretch:isar-image-base \
-            mc:qemuarm64-stretch:isar-image-base \
+            mc:qemuarm64-stretch:isar-image-base"
+
+REPRO_TARGETS_SET="\
             mc:qemuamd64-stretch:isar-image-base \
             mc:qemuarm-buster:isar-image-base"
 
-
 show_help() {
     echo "This script builds the default Isar images."
     echo
@@ -70,7 +71,7 @@ show_help() {
     echo "    -d, --debug              enable debug bitbake output."
     echo "    -f, --fast               cross build reduced set of configurations."
     echo "    -q, --quiet              suppress verbose bitbake output."
-    echo "    -r, --repro [-s, --sign] enable use of cached base repository with optional signing."
+    echo "    -r, --repro              enable use of cached base repository."
     echo "    --help                   display this message and exit."
     echo
     echo "Exit status:"
@@ -109,9 +110,6 @@ do
         ;;
     -r|--repro)
         REPRO_BUILD="1"
-        case "$2" in
-        -s|--sign) SIGN_REPO='1'; shift ;;
-        esac
         ;;
     *)
         echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
@@ -133,13 +131,24 @@ if [ -n "$CROSS_BUILD" ]; then
 fi
 
 if [ -n "$REPRO_BUILD" ]; then
-    if [ -n "$SIGN_REPO" ]; then
-        ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
-        ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
-        export GNUPGHOME=$(mktemp -d)
-        gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
-        echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf
-    fi
+    ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
+    ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
+    export GNUPGHOME=$(mktemp -d)
+    gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
+    
+    # Enable use of cached base repository
+    echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf
+    bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET_SIGNED
+    while [ -e bitbake.sock ]; do sleep 1; done
+    sudo rm -rf tmp
+    sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+    bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
+    while [ -e bitbake.sock ]; do sleep 1; done
+    # Cleanup and disable use of cached base repository
+    sudo rm -rf tmp
+    sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
+
     # Enable use of cached base repository
     bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET
     while [ -e bitbake.sock ]; do sleep 1; done
@@ -150,7 +159,6 @@ if [ -n "$REPRO_BUILD" ]; then
     # Cleanup and disable use of cached base repository
     sudo rm -rf tmp
     sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
-    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
 fi
 
 sed -i -e 's/#IMAGE_INSTALL += "isar-disable-apt-cache"/IMAGE_INSTALL += "isar-disable-apt-cache"/g' conf/local.conf
-- 
2.23.0

[PATCH 2/2] bootstrap: really fix gpg bootstrap when not signed

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

ef5255a8f8006e6f4 fixed one case but broke another. The existance of the
key file does not mean that it actually contains our bootstrap keys. It
contains keys from multiple sources.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index b506f30..7353abf 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -233,7 +233,9 @@ isar_bootstrap() {
     debootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}"
     if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then
         debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}"
-    else
+    fi
+    if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ] && [ -z "${BASE_REPO_KEY}" ]
+    then
         debootstrap_args="$debootstrap_args --no-check-gpg"
     fi
     E="${@ isar_export_proxies(d)}"
-- 
2.23.0

Re: [PATCH 1/2] ci: always test signed and unsigned repo builds

[Henning Schild]

This is not the full test-it all i imagined. But in order to send the
fix ASAP and bring at least some testing, that is what i came up with.

Note if your CI uses "-r -s" you will have to change it! Or this patch
needs a legacy option ignore for the "-s".

Whatever we come up with for this one, the second one is a fix that
probably solves Jans issue. It could potentially be applied before the
testing gets "repaired", but i would like both to be applied. Would
mean we test signed+unsigned but not yet the extra key sources
(DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS)

Henning

On Fri, 15 Nov 2019 18:00:43 +0100
Henning Schild <henning.schild@siemens.com> wrote:

> From: Henning Schild <henning.schild@siemens.com>
> 
> Just split the target set we had into two. Test the first signed and
> the other one unsigned.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  scripts/ci_build.sh | 38 +++++++++++++++++++++++---------------
>  1 file changed, 23 insertions(+), 15 deletions(-)
> 
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index ecaa88a..8295c43 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -50,13 +50,14 @@ CROSS_TARGETS_SET="\
>                    mc:de0-nano-soc-stretch:isar-image-base \
>                    mc:rpi-stretch:isar-image-base"
>  
> -REPRO_TARGETS_SET="\
> +REPRO_TARGETS_SET_SIGNED="\
>              mc:qemuarm-stretch:isar-image-base \
> -            mc:qemuarm64-stretch:isar-image-base \
> +            mc:qemuarm64-stretch:isar-image-base"
> +
> +REPRO_TARGETS_SET="\
>              mc:qemuamd64-stretch:isar-image-base \
>              mc:qemuarm-buster:isar-image-base"
>  
> -
>  show_help() {
>      echo "This script builds the default Isar images."
>      echo
> @@ -70,7 +71,7 @@ show_help() {
>      echo "    -d, --debug              enable debug bitbake output."
>      echo "    -f, --fast               cross build reduced set of
> configurations." echo "    -q, --quiet              suppress verbose
> bitbake output."
> -    echo "    -r, --repro [-s, --sign] enable use of cached base
> repository with optional signing."
> +    echo "    -r, --repro              enable use of cached base
> repository." echo "    --help                   display this message
> and exit." echo
>      echo "Exit status:"
> @@ -109,9 +110,6 @@ do
>          ;;
>      -r|--repro)
>          REPRO_BUILD="1"
> -        case "$2" in
> -        -s|--sign) SIGN_REPO='1'; shift ;;
> -        esac
>          ;;
>      *)
>          echo "error: invalid parameter '$key', please try '--help'
> to get list of supported parameters" @@ -133,13 +131,24 @@ if [ -n
> "$CROSS_BUILD" ]; then fi
>  
>  if [ -n "$REPRO_BUILD" ]; then
> -    if [ -n "$SIGN_REPO" ]; then
> -
> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
> -
> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
> -        export GNUPGHOME=$(mktemp -d)
> -        gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE
> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
> -        echo
> BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >>
> conf/local.conf
> -    fi
> +
> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
> +
> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
> +    export GNUPGHOME=$(mktemp -d)
> +    gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE
> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
> +    
> +    # Enable use of cached base repository
> +    echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\"
> >> conf/local.conf
> +    bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET_SIGNED
> +    while [ -e bitbake.sock ]; do sleep 1; done
> +    sudo rm -rf tmp
> +    sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?=
> "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
> +    bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
> +    while [ -e bitbake.sock ]; do sleep 1; done
> +    # Cleanup and disable use of cached base repository
> +    sudo rm -rf tmp
> +    sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?=
> "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
> +    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
> +
>      # Enable use of cached base repository
>      bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET
>      while [ -e bitbake.sock ]; do sleep 1; done
> @@ -150,7 +159,6 @@ if [ -n "$REPRO_BUILD" ]; then
>      # Cleanup and disable use of cached base repository
>      sudo rm -rf tmp
>      sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?=
> "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
> -    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
>  fi
>  
>  sed -i -e 's/#IMAGE_INSTALL +=
> "isar-disable-apt-cache"/IMAGE_INSTALL += "isar-disable-apt-cache"/g'
> conf/local.conf

Re: [PATCH 1/2] ci: always test signed and unsigned repo builds

[Jan Kiszka]

On 15.11.19 18:06, Henning Schild wrote:
> This is not the full test-it all i imagined. But in order to send the
> fix ASAP and bring at least some testing, that is what i came up with.
> 
> Note if your CI uses "-r -s" you will have to change it! Or this patch
> needs a legacy option ignore for the "-s".

Then please also update in-tree .gitlab-ci.yml accordingly. This would 
otherwise break the night CI.

Jan

> 
> Whatever we come up with for this one, the second one is a fix that
> probably solves Jans issue. It could potentially be applied before the
> testing gets "repaired", but i would like both to be applied. Would
> mean we test signed+unsigned but not yet the extra key sources
> (DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS)
> 
> Henning
> 
> On Fri, 15 Nov 2019 18:00:43 +0100
> Henning Schild <henning.schild@siemens.com> wrote:
> 
>> From: Henning Schild <henning.schild@siemens.com>
>>
>> Just split the target set we had into two. Test the first signed and
>> the other one unsigned.
>>
>> Signed-off-by: Henning Schild <henning.schild@siemens.com>
>> ---
>>   scripts/ci_build.sh | 38 +++++++++++++++++++++++---------------
>>   1 file changed, 23 insertions(+), 15 deletions(-)
>>
>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
>> index ecaa88a..8295c43 100755
>> --- a/scripts/ci_build.sh
>> +++ b/scripts/ci_build.sh
>> @@ -50,13 +50,14 @@ CROSS_TARGETS_SET="\
>>                     mc:de0-nano-soc-stretch:isar-image-base \
>>                     mc:rpi-stretch:isar-image-base"
>>   
>> -REPRO_TARGETS_SET="\
>> +REPRO_TARGETS_SET_SIGNED="\
>>               mc:qemuarm-stretch:isar-image-base \
>> -            mc:qemuarm64-stretch:isar-image-base \
>> +            mc:qemuarm64-stretch:isar-image-base"
>> +
>> +REPRO_TARGETS_SET="\
>>               mc:qemuamd64-stretch:isar-image-base \
>>               mc:qemuarm-buster:isar-image-base"
>>   
>> -
>>   show_help() {
>>       echo "This script builds the default Isar images."
>>       echo
>> @@ -70,7 +71,7 @@ show_help() {
>>       echo "    -d, --debug              enable debug bitbake output."
>>       echo "    -f, --fast               cross build reduced set of
>> configurations." echo "    -q, --quiet              suppress verbose
>> bitbake output."
>> -    echo "    -r, --repro [-s, --sign] enable use of cached base
>> repository with optional signing."
>> +    echo "    -r, --repro              enable use of cached base
>> repository." echo "    --help                   display this message
>> and exit." echo
>>       echo "Exit status:"
>> @@ -109,9 +110,6 @@ do
>>           ;;
>>       -r|--repro)
>>           REPRO_BUILD="1"
>> -        case "$2" in
>> -        -s|--sign) SIGN_REPO='1'; shift ;;
>> -        esac
>>           ;;
>>       *)
>>           echo "error: invalid parameter '$key', please try '--help'
>> to get list of supported parameters" @@ -133,13 +131,24 @@ if [ -n
>> "$CROSS_BUILD" ]; then fi
>>   
>>   if [ -n "$REPRO_BUILD" ]; then
>> -    if [ -n "$SIGN_REPO" ]; then
>> -
>> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
>> -
>> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
>> -        export GNUPGHOME=$(mktemp -d)
>> -        gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE
>> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
>> -        echo
>> BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >>
>> conf/local.conf
>> -    fi
>> +
>> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
>> +
>> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
>> +    export GNUPGHOME=$(mktemp -d)
>> +    gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE
>> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
>> +
>> +    # Enable use of cached base repository
>> +    echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\"
>>>> conf/local.conf
>> +    bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET_SIGNED
>> +    while [ -e bitbake.sock ]; do sleep 1; done
>> +    sudo rm -rf tmp
>> +    sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?=
>> "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
>> +    bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
>> +    while [ -e bitbake.sock ]; do sleep 1; done
>> +    # Cleanup and disable use of cached base repository
>> +    sudo rm -rf tmp
>> +    sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?=
>> "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
>> +    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
>> +
>>       # Enable use of cached base repository
>>       bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET
>>       while [ -e bitbake.sock ]; do sleep 1; done
>> @@ -150,7 +159,6 @@ if [ -n "$REPRO_BUILD" ]; then
>>       # Cleanup and disable use of cached base repository
>>       sudo rm -rf tmp
>>       sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?=
>> "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
>> -    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
>>   fi
>>   
>>   sed -i -e 's/#IMAGE_INSTALL +=
>> "isar-disable-apt-cache"/IMAGE_INSTALL += "isar-disable-apt-cache"/g'
>> conf/local.conf
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[PATCHv2 1/2] ci: always test signed and unsigned repo builds

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Just split the target set we had into two. Test the first signed and the
other one unsigned.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 .gitlab-ci.yml      |  2 +-
 scripts/ci_build.sh | 42 +++++++++++++++++++++++++++---------------
 2 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index dd6e12b..1437bd7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -32,7 +32,7 @@ full-ci:
     - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
     - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
           echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
-          scripts/ci_build.sh -q -c -r -s;
+          scripts/ci_build.sh -q -c -r;
       fi
   cache:
     key: "$CI_COMMIT_REF_SLUG"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ecaa88a..8707a71 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -50,13 +50,14 @@ CROSS_TARGETS_SET="\
                   mc:de0-nano-soc-stretch:isar-image-base \
                   mc:rpi-stretch:isar-image-base"
 
-REPRO_TARGETS_SET="\
+REPRO_TARGETS_SET_SIGNED="\
             mc:qemuarm-stretch:isar-image-base \
-            mc:qemuarm64-stretch:isar-image-base \
+            mc:qemuarm64-stretch:isar-image-base"
+
+REPRO_TARGETS_SET="\
             mc:qemuamd64-stretch:isar-image-base \
             mc:qemuarm-buster:isar-image-base"
 
-
 show_help() {
     echo "This script builds the default Isar images."
     echo
@@ -70,7 +71,7 @@ show_help() {
     echo "    -d, --debug              enable debug bitbake output."
     echo "    -f, --fast               cross build reduced set of configurations."
     echo "    -q, --quiet              suppress verbose bitbake output."
-    echo "    -r, --repro [-s, --sign] enable use of cached base repository with optional signing."
+    echo "    -r, --repro              enable use of cached base repository."
     echo "    --help                   display this message and exit."
     echo
     echo "Exit status:"
@@ -109,9 +110,10 @@ do
         ;;
     -r|--repro)
         REPRO_BUILD="1"
-        case "$2" in
-        -s|--sign) SIGN_REPO='1'; shift ;;
-        esac
+	# this switch is deprecated, just here to not cause failing CI on legacy configs
+	case "$2" in
+	-s|--sign) shift ;;
+	esac
         ;;
     *)
         echo "error: invalid parameter '$key', please try '--help' to get list of supported parameters"
@@ -133,13 +135,24 @@ if [ -n "$CROSS_BUILD" ]; then
 fi
 
 if [ -n "$REPRO_BUILD" ]; then
-    if [ -n "$SIGN_REPO" ]; then
-        ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
-        ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
-        export GNUPGHOME=$(mktemp -d)
-        gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
-        echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf
-    fi
+    ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
+    ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
+    export GNUPGHOME=$(mktemp -d)
+    gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
+    
+    # Enable use of cached base repository
+    echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >> conf/local.conf
+    bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET_SIGNED
+    while [ -e bitbake.sock ]; do sleep 1; done
+    sudo rm -rf tmp
+    sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+    bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
+    while [ -e bitbake.sock ]; do sleep 1; done
+    # Cleanup and disable use of cached base repository
+    sudo rm -rf tmp
+    sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
+    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
+
     # Enable use of cached base repository
     bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET
     while [ -e bitbake.sock ]; do sleep 1; done
@@ -150,7 +163,6 @@ if [ -n "$REPRO_BUILD" ]; then
     # Cleanup and disable use of cached base repository
     sudo rm -rf tmp
     sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?= "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
-    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
 fi
 
 sed -i -e 's/#IMAGE_INSTALL += "isar-disable-apt-cache"/IMAGE_INSTALL += "isar-disable-apt-cache"/g' conf/local.conf
-- 
2.23.0

Re: [PATCHv2 1/2] ci: always test signed and unsigned repo builds

[Henning Schild]

diff to v1:
 - just ignore the "-s" for "legacy" CI
 - update gitlab-ci config

Henning

On Fri, 15 Nov 2019 18:21:38 +0100
Henning Schild <henning.schild@siemens.com> wrote:

> From: Henning Schild <henning.schild@siemens.com>
> 
> Just split the target set we had into two. Test the first signed and
> the other one unsigned.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  .gitlab-ci.yml      |  2 +-
>  scripts/ci_build.sh | 42 +++++++++++++++++++++++++++---------------
>  2 files changed, 28 insertions(+), 16 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index dd6e12b..1437bd7 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -32,7 +32,7 @@ full-ci:
>      - PREVIOUS_SHA="$(cat .CI_COMMIT_SHA || true)"
>      - if [ "$CI_COMMIT_SHA" != "$PREVIOUS_SHA" ]; then
>            echo "$CI_COMMIT_SHA" > .CI_COMMIT_SHA;
> -          scripts/ci_build.sh -q -c -r -s;
> +          scripts/ci_build.sh -q -c -r;
>        fi
>    cache:
>      key: "$CI_COMMIT_REF_SLUG"
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index ecaa88a..8707a71 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -50,13 +50,14 @@ CROSS_TARGETS_SET="\
>                    mc:de0-nano-soc-stretch:isar-image-base \
>                    mc:rpi-stretch:isar-image-base"
>  
> -REPRO_TARGETS_SET="\
> +REPRO_TARGETS_SET_SIGNED="\
>              mc:qemuarm-stretch:isar-image-base \
> -            mc:qemuarm64-stretch:isar-image-base \
> +            mc:qemuarm64-stretch:isar-image-base"
> +
> +REPRO_TARGETS_SET="\
>              mc:qemuamd64-stretch:isar-image-base \
>              mc:qemuarm-buster:isar-image-base"
>  
> -
>  show_help() {
>      echo "This script builds the default Isar images."
>      echo
> @@ -70,7 +71,7 @@ show_help() {
>      echo "    -d, --debug              enable debug bitbake output."
>      echo "    -f, --fast               cross build reduced set of
> configurations." echo "    -q, --quiet              suppress verbose
> bitbake output."
> -    echo "    -r, --repro [-s, --sign] enable use of cached base
> repository with optional signing."
> +    echo "    -r, --repro              enable use of cached base
> repository." echo "    --help                   display this message
> and exit." echo
>      echo "Exit status:"
> @@ -109,9 +110,10 @@ do
>          ;;
>      -r|--repro)
>          REPRO_BUILD="1"
> -        case "$2" in
> -        -s|--sign) SIGN_REPO='1'; shift ;;
> -        esac
> +	# this switch is deprecated, just here to not cause failing
> CI on legacy configs
> +	case "$2" in
> +	-s|--sign) shift ;;
> +	esac
>          ;;
>      *)
>          echo "error: invalid parameter '$key', please try '--help'
> to get list of supported parameters" @@ -133,13 +135,24 @@ if [ -n
> "$CROSS_BUILD" ]; then fi
>  
>  if [ -n "$REPRO_BUILD" ]; then
> -    if [ -n "$SIGN_REPO" ]; then
> -
> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
> -
> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
> -        export GNUPGHOME=$(mktemp -d)
> -        gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE
> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
> -        echo
> BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\" >>
> conf/local.conf
> -    fi
> +
> ISAR_TESTSUITE_GPG_PUB_KEY_FILE="$ISARROOT/testsuite/base-apt/test_pub.key"
> +
> ISAR_TESTSUITE_GPG_PRIV_KEY_FILE="$ISARROOT/testsuite/base-apt/test_priv.key"
> +    export GNUPGHOME=$(mktemp -d)
> +    gpg --import $ISAR_TESTSUITE_GPG_PUB_KEY_FILE
> $ISAR_TESTSUITE_GPG_PRIV_KEY_FILE
> +    
> +    # Enable use of cached base repository
> +    echo BASE_REPO_KEY=\"file://$ISAR_TESTSUITE_GPG_PUB_KEY_FILE\"
> >> conf/local.conf
> +    bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET_SIGNED
> +    while [ -e bitbake.sock ]; do sleep 1; done
> +    sudo rm -rf tmp
> +    sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?=
> "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
> +    bitbake $BB_ARGS $REPRO_TARGETS_SET_SIGNED
> +    while [ -e bitbake.sock ]; do sleep 1; done
> +    # Cleanup and disable use of cached base repository
> +    sudo rm -rf tmp
> +    sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?=
> "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
> +    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
> +
>      # Enable use of cached base repository
>      bitbake $BB_ARGS -c cache_base_repo $REPRO_TARGETS_SET
>      while [ -e bitbake.sock ]; do sleep 1; done
> @@ -150,7 +163,6 @@ if [ -n "$REPRO_BUILD" ]; then
>      # Cleanup and disable use of cached base repository
>      sudo rm -rf tmp
>      sed -i -e 's/ISAR_USE_CACHED_BASE_REPO ?=
> "1"/#ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
> -    sed -i -e 's/^BASE_REPO_KEY/#BASE_REPO_KEY/g' conf/local.conf
>  fi
>  
>  sed -i -e 's/#IMAGE_INSTALL +=
> "isar-disable-apt-cache"/IMAGE_INSTALL += "isar-disable-apt-cache"/g'
> conf/local.conf

Re: [PATCH 1/2] ci: always test signed and unsigned repo builds

[Baurzhan Ismagulov]

On Fri, Nov 15, 2019 at 06:06:04PM +0100, Henning Schild wrote:
> This is not the full test-it all i imagined. But in order to send the
> fix ASAP and bring at least some testing, that is what i came up with.

Yes, I see. Looks good to me. There is trailing white space in the first patch.
I've removed it and made other minor changes on the way. The modified version
is applied to ibr/next. I'll send it to the list, please let me know if the
changes are ok for you.

With kind regards,
Baurzhan.

Re: [PATCH 1/2] ci: always test signed and unsigned repo builds

[Henning Schild]

Am Mon, 18 Nov 2019 18:05:07 +0100
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Fri, Nov 15, 2019 at 06:06:04PM +0100, Henning Schild wrote:
> > This is not the full test-it all i imagined. But in order to send
> > the fix ASAP and bring at least some testing, that is what i came
> > up with.  
> 
> Yes, I see. Looks good to me. There is trailing white space in the
> first patch. I've removed it and made other minor changes on the way.
> The modified version is applied to ibr/next. I'll send it to the
> list, please let me know if the changes are ok for you.

LGTM.

Henning

> With kind regards,
> Baurzhan.
> 

Re: [PATCH 1/2] ci: always test signed and unsigned repo builds

[Baurzhan Ismagulov]

On Mon, Nov 18, 2019 at 06:52:56PM +0100, Henning Schild wrote:
> > Yes, I see. Looks good to me. There is trailing white space in the
> > first patch. I've removed it and made other minor changes on the way.
> > The modified version is applied to ibr/next. I'll send it to the
> > list, please let me know if the changes are ok for you.
> 
> LGTM.

Applied to next, thanks.

With kind regards,
Baurzhan.