[PATCH 0/3] Cleanups, gitlab-ci update

[PATCH 0/3] Cleanups, gitlab-ci update

[Jan Kiszka]

See patches for details.

Jan

Jan Kiszka (3):
  scripts: Remove build_parallel
  doc: Remove references to wheezy
  gitlab-ci: Update targets, add cross-compilation test

 .gitlab-ci.yml         | 12 +++++--
 doc/user_manual.md     | 40 ++---------------------
 scripts/build_parallel | 89 --------------------------------------------------
 3 files changed, 12 insertions(+), 129 deletions(-)
 delete mode 100755 scripts/build_parallel

-- 
2.16.4

[PATCH 1/3] scripts: Remove build_parallel

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

We are happily building multiple images in parallel by now, without any
indication that things are needlessly serialized. This likely dates back
to the situation, possibly both in bitbake and Isar, at the time the
script was merged.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 doc/user_manual.md     | 29 ----------------
 scripts/build_parallel | 89 --------------------------------------------------
 2 files changed, 118 deletions(-)
 delete mode 100755 scripts/build_parallel

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 427f83d..c8f8ad9 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -62,7 +62,6 @@ python3
 python3-distutils           # in case your host is host > debian 9
 qemu
 qemu-user-static
-rxvt-unicode                # build_parallel
 binfmt-support
 sudo
 reprepro
@@ -175,34 +174,6 @@ tmp/deploy/images/isar-image-base-debian-stretch-qemuamd64.ext4.img
 tmp/deploy/images/isar-image-base.rpi-sdimg
 ```
 
-The BitBake revision included with Isar seems to serialize multiconfig builds.
-The following script may be used from the project directory (`isar`) to build
-multiple configurations in different build directories faster:
-
-```
-scripts/build_parallel ../build multiconfig:qemuarm-wheezy:isar-image-base \
-    multiconfig:qemuarm-jessie:isar-image-base \
-    multiconfig:qemuarm-stretch:isar-image-base \
-    multiconfig:qemui386-jessie:isar-image-base \
-    multiconfig:qemui386-stretch:isar-image-base \
-    multiconfig:qemuamd64-jessie:isar-image-base \
-    multiconfig:qemuamd64-stretch:isar-image-base \
-    multiconfig:rpi-jessie:isar-image-base
-```
-
-Created images are:
-
-```
-../build-1/tmp/deploy/images/isar-image-base-debian-wheezy-qemuarm.ext4.img
-../build-2/tmp/deploy/images/isar-image-base-debian-jessie-qemuarm.ext4.img
-../build-3/tmp/deploy/images/isar-image-base-debian-stretch-qemuarm.ext4.img
-../build-4/tmp/deploy/images/isar-image-base-debian-jessie-qemui386.ext4.img
-../build-5/tmp/deploy/images/isar-image-base-debian-stretch-qemui386.wic.img
-../build-6/tmp/deploy/images/isar-image-base-debian-jessie-qemuamd64.ext4.img
-../build-7/tmp/deploy/images/isar-image-base-debian-stretch-qemuamd64.wic.img
-../build-8/tmp/deploy/images/isar-image-base.rpi-sdimg
-```
-
 ### Generate full disk image
 
 A bootable disk image is generated if you set IMAGE_TYPE to 'wic-img'. Behind the scenes a tool called `wic` is used to assemble the images. It is controlled by a `.wks` file which you can choose with changing WKS_FILE. Some examples in the tree use that feature already.
diff --git a/scripts/build_parallel b/scripts/build_parallel
deleted file mode 100755
index 20c8401..0000000
--- a/scripts/build_parallel
+++ /dev/null
@@ -1,89 +0,0 @@
-#!/bin/bash
-#
-# Build several multiconfig targets in parallel
-#
-# Bitbake seems to serialize multiconfig builds.
-#
-# Example usage:
-# scripts/build_parallel ../build multiconfig:qemuarm-wheezy:isar-image-base multiconfig:qemuarm-jessie:isar-image-base multiconfig:rpi-jessie:isar-image-base
-#
-# This software is a part of Isar.
-# Copyright (C) 2017 ilbers GmbH
-#
-# Test:
-# - As is
-# - With opt_exit=1
-# - With opt_debug=11
-
-ES_OK=0
-ES_SYNTAX=1
-ES_RUNTIME=2
-ES_NOTREACHED=255
-
-dbg() {
-    local cat=$1
-    local lvl=$2
-    shift 2
-    if [ "$lvl" -lt $opt_debug ]; then
-	echo $* >&2
-    fi
-}
-
-if [ $# -lt 2 ]; then
-    echo "$0: ERROR: Too few arguments" >&2
-    echo "Usage: $0 BUILD_DIR_TEMPLATE TARGET..." >&2
-    exit $ES_SYNTAX
-fi
-
-bdirt=$1
-shift
-targets="$*"
-
-opt_exit=0
-opt_debug=0
-
-XTERM=urxvt
-
-n=`echo $targets |wc -w`
-ret=$ES_OK
-i=1
-
-time (
-    for t in $targets; do
-	# urxvt seems to always return zero, use files to return exit status.
-	# Provide a value if subprocess fails to write one.
-	echo $ES_NOTREACHED >${bdirt}-$i-ret
-	# Exit on success, remain in $XTERM on failure to ease debugging
-	$XTERM -e bash --init-file <(echo " \
-            echo -n \`pwd\`'$ '; \
-            tput bold; \
-            echo . isar-init-build-env ${bdirt}-$i; \
-            tput sgr0; \
-            . isar-init-build-env ${bdirt}-$i; \
-            echo -n \`pwd\`'$ '; \
-            tput bold; \
-            echo time bitbake $t; \
-            tput sgr0; \
-            time bitbake $t; \
-            r=\$?; \
-            echo \$r >${bdirt}-$i-ret; \
-            if [ $opt_exit != 0 -a \$r = $ES_OK ]; then \
-                exit $ES_OK; \
-            fi") &
-	pid[$i]=$!
-	dbg CHLD 10 pid ${pid[$i]} dir ${bdirt}-$i target $t
-	i=$((i+1))
-    done
-    for i in `seq 1 $n`; do
-	wait ${pid[$i]}
-	r=`cat ${bdirt}-$i-ret`
-	# Don't use -ne to prevent [ errors if $r contains non-numeric garbage
-	if [ "$r" != $ES_OK ]; then
-	    t=`echo $targets |awk '{ print $'$i' }'`
-	    echo "ERROR: Target $t failed in ${bdirt}-$i with $r" >&2
-	    rm -f ${bdirt}-$i-ret
-	    ret=$ES_RUNTIME
-	fi
-    done)
-
-exit $ret
-- 
2.16.4

[PATCH 2/3] doc: Remove references to wheezy

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

The next logical step to phase it out.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 doc/user_manual.md | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index c8f8ad9..d99f1f0 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -38,7 +38,6 @@ Isar provides:
 For demonstration purposes, Isar provides support for the following
 configurations:
 
- - QEMU ARM with Debian Wheezy
  - QEMU ARM with Debian Jessie
  - QEMU ARM with Debian Stretch (builds but fails to run, see #22)
  - QEMU i386 with Debian Jessie
@@ -72,7 +71,6 @@ Notes:
 * BitBake requires Python 3.4+.
 * The python3 package is required for the correct `alternatives` setting.
 * QEMU 2.8+ is required to run Stretch images locally in QEMU. This version is available in Stretch and Jessie backports apts.
-* e2fsprogs v1.42- required for building Wheezy images.
 
 ### Setup Sudo
 
@@ -137,7 +135,6 @@ a single call. List all configurations in `conf/local.conf`:
 
 ```
 BBMULTICONFIG = " \
-    qemuarm-wheezy \
     qemuarm-jessie \
     qemuarm-stretch \
     qemui386-jessie \
@@ -151,7 +148,7 @@ BBMULTICONFIG = " \
 The following command will produce `isar-image-base` images for all targets:
 
 ```
-$ bitbake multiconfig:qemuarm-wheezy:isar-image-base \
+$ bitbake \
     multiconfig:qemuarm-jessie:isar-image-base \
     multiconfig:qemuarm-stretch:isar-image-base \
     multiconfig:qemui386-jessie:isar-image-base \
@@ -164,7 +161,6 @@ $ bitbake multiconfig:qemuarm-wheezy:isar-image-base \
 Created images are:
 
 ```
-tmp/deploy/images/isar-image-base-debian-wheezy-qemuarm.ext4.img
 tmp/deploy/images/isar-image-base-debian-jessie-qemuarm.ext4.img
 tmp/deploy/images/isar-image-base-debian-stretch-qemuarm.ext4.img
 tmp/deploy/images/isar-image-base-debian-jessie-qemui386.ext4.img
@@ -277,7 +273,7 @@ following variables define the default configuration to build for:
  - `MACHINE` - The board to build for (e.g., `qemuarm`, `rpi`). BitBake looks
    for conf/multiconfig/${MACHINE}.conf in every layer.
 
- - `DISTRO` - The distro to use (e.g., `debian-wheezy`, `raspbian-jessie`).
+ - `DISTRO` - The distro to use (e.g. `raspbian-jessie`, `debian-stretch`).
    BitBake looks for conf/distro/${DISTRO}.conf in every layer.
 
  - `DISTRO_ARCH` - The Debian architecture to build for (e.g., `armhf`).
@@ -307,7 +303,6 @@ Some other variables include:
 
 In Isar, each machine can use its specific Linux distro to generate `buildchroot` and target filesystem. By default, Isar provides configuration files for the following distros:
 
- - debian-wheezy
  - debian-jessie
  - debian-stretch
  - raspbian-jessie
@@ -367,8 +362,8 @@ To add new distro, user should perform the following steps:
  - Create the `.conf` file in distro folder with the name of your distribution. We recommend to name distribution in the following format: `name`-`suite`, for example:
 
     ```
-    debian-wheezy
     debian-jessie
+    debian-stretch
     ```
 
  - In this file, define the variables described above.
-- 
2.16.4

[PATCH 3/3] gitlab-ci: Update targets, add cross-compilation test

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

We dropped wheezy but added buster by now. Moreover, we should check the
cross-build feature by running the qemuarm/64-stretch targets in that
mode. And to avoid stumbling over artifacts from previous builds, we
need to set GIT_STRATEGY to "clone".

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .gitlab-ci.yml | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 79a6710..da73826 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,5 +1,8 @@
 image: kasproject/kas-isar:latest
 
+variables:
+  GIT_STRATEGY: clone
+
 all:
   stage: build
   script:
@@ -9,12 +12,15 @@ all:
     - export no_proxy=$NO_PROXY
     - source isar-init-build-env out
     - bitbake
-        multiconfig:qemuarm-wheezy:isar-image-base
         multiconfig:qemuarm-jessie:isar-image-base
-        multiconfig:qemuarm-stretch:isar-image-base
-        multiconfig:qemuarm64-stretch:isar-image-base
         multiconfig:qemui386-jessie:isar-image-base
         multiconfig:qemui386-stretch:isar-image-base
+        multiconfig:qemui386-buster:isar-image-base
         multiconfig:qemuamd64-jessie:isar-image-base
         multiconfig:qemuamd64-stretch:isar-image-base
+        multiconfig:qemuamd64-buster:isar-image-base
         multiconfig:rpi-jessie:isar-image-base
+    - sed -i -e 's/ISAR_CROSS_COMPILE ?= "0"/ISAR_CROSS_COMPILE ?= "1"/g' conf/local.conf
+    - bitbake
+        multiconfig:qemuarm-stretch:isar-image-base
+        multiconfig:qemuarm64-stretch:isar-image-base
-- 
2.16.4

Re: [PATCH 0/3] Cleanups, gitlab-ci update

[Maxim Yu. Osipov]

On 08/20/2018 09:20 AM, Jan Kiszka wrote:
> See patches for details.

Applied to the 'next',

Thanks,
Maxim.

> Jan
> 
> Jan Kiszka (3):
>    scripts: Remove build_parallel
>    doc: Remove references to wheezy
>    gitlab-ci: Update targets, add cross-compilation test
> 
>   .gitlab-ci.yml         | 12 +++++--
>   doc/user_manual.md     | 40 ++---------------------
>   scripts/build_parallel | 89 --------------------------------------------------
>   3 files changed, 12 insertions(+), 129 deletions(-)
>   delete mode 100755 scripts/build_parallel
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov