[PATCH v2 0/6] Enable U-Boot-based images, add Banana Pi demo

[PATCH v2 0/6] Enable U-Boot-based images, add Banana Pi demo

[Jan Kiszka]

Changes in v2:
 - use booti on aarch64 in generated boot.scr
 - simplify boot script generator internally (fstab parsing, ramdisk
   size evaluation)
 - use options for setup_root_file_system()

Original cover:

This provides missing bits to build SD-card (or also eMMC) images for
boards that use U-Boot as bootloader. The images can be described via
well-known wks files.

For U-Boot, we rely on scan_dev_for_scripts from the default
environment. The reason to generate our own boot script is documented in
the related patch. While that generator may start its life inside Isar,
I could imagine selling it also to Debian as a seperate package.

As an example, I've added the necessary configs to build a basic image
for a Banana Pi board.

This is surely not the smoothest solution. E.g., I dislike that we still
need those ugly ROOTFS_TYPE and ROOTFS_DEV variables - the information
is encoded in the wks file already. But those files are evaluated by wic
only AFTER we build the rootfs. So, extracting that information from wks
would require logic like in u-boot-script_1.0.bb, i.e. a certain level
wic duplication so that we can continue to generate /etc/fstab ahead of
the package installation. Better suggestions welcome.

We should probably also convert BUILDCHROOT_PREINSTALL_WIC to
WIC_FILE_DEPENDS* as indicated in patch 2. But I didn't study the
implications in details yet.

Looking forward to comments and tests!

Jan

Jan Kiszka (6):
  Add DISTRO and DISTRO_ARCH as OVERRIDES suffixes
  buildchroot: Break up wic build deps into common and arch-specific
    ones
  isar-bootstrap: Use options for setup_root_file_system()
  Set up target image fstab prior to installing any packages
  Add u-boot script generator
  Add Banana Pi SD-card image

 .../conf/distro/debian-stretch-backports.list      |  1 +
 meta-isar/conf/local.conf.sample                   |  1 +
 meta-isar/conf/multiconfig/bananapi-stretch.conf   | 24 +++++++++
 .../conf/multiconfig/preferences.bananapi.conf     |  7 +++
 .../images/files/debian-configscript.sh            | 14 ------
 .../images/files/raspbian-configscript.sh          | 14 ------
 meta-isar/recipes-core/images/isar-image-base.bb   | 20 ++++++--
 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks  |  5 ++
 meta/classes/isar-bootstrap-helper.bbclass         | 18 +++++--
 meta/conf/isar-bitbake.conf                        |  2 +
 meta/recipes-bsp/u-boot-script/files/u-boot-script |  9 ++++
 .../u-boot-script/files/update-u-boot-script       | 57 ++++++++++++++++++++++
 .../u-boot-script/files/zz-u-boot-script           |  3 ++
 .../recipes-bsp/u-boot-script/u-boot-script_1.0.bb | 53 ++++++++++++++++++++
 meta/recipes-devtools/buildchroot/buildchroot.bb   | 23 ++++++---
 15 files changed, 208 insertions(+), 43 deletions(-)
 create mode 100644 meta-isar/conf/distro/debian-stretch-backports.list
 create mode 100644 meta-isar/conf/multiconfig/bananapi-stretch.conf
 create mode 100644 meta-isar/conf/multiconfig/preferences.bananapi.conf
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks
 create mode 100644 meta/recipes-bsp/u-boot-script/files/u-boot-script
 create mode 100755 meta/recipes-bsp/u-boot-script/files/update-u-boot-script
 create mode 100755 meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
 create mode 100644 meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb

-- 
2.13.7

[PATCH v2 1/6] Add DISTRO and DISTRO_ARCH as OVERRIDES suffixes

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

By default, bitbake offers MACHINE, TARGET_OS and TARGET_ARCH. But we
are using DISTRO and DISTRO_ARCH in Isar. Adding them will allow to let
variables depend on these parameters as well.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/conf/isar-bitbake.conf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index 8a1d86b..ead7798 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -24,6 +24,8 @@ SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
 BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
 CACHE = "${TMPDIR}/cache"
 
+OVERRIDES_append = ":${DISTRO}:${DISTRO_ARCH}"
+
 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DIR \
     SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
     USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
-- 
2.13.7

[PATCH v2 2/6] buildchroot: Break up wic build deps into common and arch-specific ones

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

wic is currently not usable on non-x86 archs because of naturally
unresolvable dependencies on x86 grub-efi packages. Fix this by
factoring our a common dependency core and a set of arch-specific ones.
This should also allow to build EFI images for armhf and arm64
(Tianocore firmware or U-boot in EFI mode). But primarily, it allows to
build non-EFI images on non-x86 archs via wic.

Note that we should probably consider replacing the Isar-proprietary
BUILDCHROOT_PREINSTALL_WIC variable with the OE standard
WIC_FILE_DEPENDS*.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/recipes-devtools/buildchroot/buildchroot.bb | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index cec42c3..e405509 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -32,15 +32,27 @@ BUILDCHROOT_PREINSTALL_WIC = " \
                              parted \
                              gdisk \
                              util-linux \
-                             syslinux \
-                             syslinux-common \
                              dosfstools \
                              mtools \
                              e2fsprogs \
-                             grub-efi-amd64-bin \
-                             grub-efi-ia32-bin \
                              python3"
 
+BUILDCHROOT_PREINSTALL_WIC_append_amd64 = " \
+                             syslinux \
+                             syslinux-common \
+                             grub-efi-amd64-bin"
+
+BUILDCHROOT_PREINSTALL_WIC_append_armhf = " \
+                             grub-efi-arm-bin"
+
+BUILDCHROOT_PREINSTALL_WIC_append_arm64 = " \
+                             grub-efi-arm64-bin"
+
+BUILDCHROOT_PREINSTALL_WIC_append_i386 = " \
+                             syslinux \
+                             syslinux-common \
+                             grub-efi-ia32-bin"
+
 python () {
     if d.getVar('IMAGE_TYPE', True) == 'wic-img':
         d.appendVar('BUILDCHROOT_PREINSTALL',
-- 
2.13.7

[PATCH v2 3/6] isar-bootstrap: Use options for setup_root_file_system()

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Convert the clean/noclean control into an option --clean. We can expand
this pattern later on.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta-isar/recipes-core/images/isar-image-base.bb |  2 +-
 meta/classes/isar-bootstrap-helper.bbclass       | 15 +++++++++++----
 meta/recipes-devtools/buildchroot/buildchroot.bb |  3 +--
 3 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index f91dfb0..891f80a 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -28,7 +28,7 @@ do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
                              ${IMAGE_ROOTFS}/isar-apt"
 
 do_rootfs() {
-    setup_root_file_system "${IMAGE_ROOTFS}" "clean" \
+    setup_root_file_system --clean "${IMAGE_ROOTFS}" \
         ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
 
     # Configure root filesystem
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index a3f1686..91dfc1a 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -15,9 +15,16 @@ def reverse_bb_array(d, varname):
     return " ".join(i for i in array)
 
 setup_root_file_system() {
+    CLEAN=""
+    while true; do
+        case "$1" in
+        --clean) CLEAN=1 ;;
+        -*) bbfatal "$0: invalid option specified: $1" ;;
+        *) break ;;
+        esac
+        shift
+    done
     ROOTFSDIR="$1"
-    CLEAN="$2"
-    shift
     shift
     PACKAGES="$@"
     APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
@@ -51,7 +58,7 @@ setup_root_file_system() {
     sudo -E chroot "$ROOTFSDIR" \
         /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
             ${IMAGE_TRANSIENT_PACKAGES}
-    [ "clean" = ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
+    [ ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
     sudo -E chroot "$ROOTFSDIR" \
         /usr/bin/apt-get ${APT_ARGS} $PACKAGES
     for pkg in ${IMAGE_TRANSIENT_PACKAGES}; do
@@ -62,7 +69,7 @@ setup_root_file_system() {
         sudo -E chroot "$ROOTFSDIR" \
             /usr/bin/apt-get purge --yes $pkg
     done
-    if [ "clean" = ${CLEAN} ]; then
+    if [ ${CLEAN} ]; then
         sudo -E chroot "$ROOTFSDIR" \
             /usr/bin/apt-get autoremove --purge --yes
         sudo -E chroot "$ROOTFSDIR" \
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index e405509..538c577 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -69,8 +69,7 @@ do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
 do_build[depends] = "isar-apt:do_cache_config isar-bootstrap:do_deploy"
 
 do_build() {
-    setup_root_file_system "${BUILDCHROOT_DIR}" "noclean" \
-        ${BUILDCHROOT_PREINSTALL}
+    setup_root_file_system "${BUILDCHROOT_DIR}" ${BUILDCHROOT_PREINSTALL}
 
     # Install package builder script
     sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder"
-- 
2.13.7

[PATCH v2 4/6] Set up target image fstab prior to installing any packages

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Package postinst hooks may need the fstab or may want to modify it,
including customization packages or the upcoming u-boot-script
generator. So extract the already identical setup pattern from the
distro config scripts, generate the fstab prior to calling
setup_root_file_system and let the latter inject the file before
starting the package installation. By specifying "-" as fstab file, we
can skip the injection for the buildchroot which does not need it.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .../recipes-core/images/files/debian-configscript.sh | 14 --------------
 .../images/files/raspbian-configscript.sh            | 14 --------------
 meta-isar/recipes-core/images/isar-image-base.bb     | 20 +++++++++++++++-----
 meta/classes/isar-bootstrap-helper.bbclass           |  3 +++
 4 files changed, 18 insertions(+), 33 deletions(-)

diff --git a/meta-isar/recipes-core/images/files/debian-configscript.sh b/meta-isar/recipes-core/images/files/debian-configscript.sh
index 136918c..bcd5507 100755
--- a/meta-isar/recipes-core/images/files/debian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/debian-configscript.sh
@@ -7,20 +7,6 @@ set -e
 
 readonly MACHINE_SERIAL="$1"
 readonly BAUDRATE_TTY="$2"
-readonly ROOTFS_DEV="$3"
-readonly ROOTFS_TYPE="$4"
-
-cat > /etc/fstab << EOF
-# Begin /etc/fstab
-/dev/$ROOTFS_DEV	/		$ROOTFS_TYPE		defaults		1	1
-proc		/proc		proc		nosuid,noexec,nodev	0	0
-sysfs		/sys		sysfs		nosuid,noexec,nodev	0	0
-devpts		/dev/pts	devpts		gid=5,mode=620		0	0
-tmpfs		/run		tmpfs		defaults		0	0
-devtmpfs	/dev		devtmpfs	mode=0755,nosuid	0	0
-
-# End /etc/fstab
-EOF
 
 # Create console device
 if [ ! -e /dev/console ]; then
diff --git a/meta-isar/recipes-core/images/files/raspbian-configscript.sh b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
index 04ea571..3631c26 100644
--- a/meta-isar/recipes-core/images/files/raspbian-configscript.sh
+++ b/meta-isar/recipes-core/images/files/raspbian-configscript.sh
@@ -7,20 +7,6 @@ set -e
 
 readonly MACHINE_SERIAL="$1"
 readonly BAUDRATE_TTY="$2"
-readonly ROOTFS_DEV="$3"
-readonly ROOTFS_TYPE="$4"
-
-cat > /etc/fstab << EOF
-# Begin /etc/fstab
-/dev/$ROOTFS_DEV	/		$ROOTFS_TYPE		defaults		1	1
-proc		/proc		proc		nosuid,noexec,nodev	0	0
-sysfs		/sys		sysfs		nosuid,noexec,nodev	0	0
-devpts		/dev/pts	devpts		gid=5,mode=620		0	0
-tmpfs		/run		tmpfs		defaults		0	0
-devtmpfs	/dev		devtmpfs	mode=0755,nosuid	0	0
-
-# End /etc/fstab
-EOF
 
 # Enable tty conditionally, systemd does not have the file but its own magic
 if [ -f /etc/inittab ]; then
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 891f80a..6cdc6b2 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -28,15 +28,25 @@ do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \
                              ${IMAGE_ROOTFS}/isar-apt"
 
 do_rootfs() {
-    setup_root_file_system --clean "${IMAGE_ROOTFS}" \
-        ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
+    cat > ${WORKDIR}/fstab << EOF
+# Begin /etc/fstab
+/dev/${ROOTFS_DEV}	/		${ROOTFS_TYPE}		defaults		1	1
+proc		/proc		proc		nosuid,noexec,nodev	0	0
+sysfs		/sys		sysfs		nosuid,noexec,nodev	0	0
+devpts		/dev/pts	devpts		gid=5,mode=620		0	0
+tmpfs		/run		tmpfs		defaults		0	0
+devtmpfs	/dev		devtmpfs	mode=0755,nosuid	0	0
+
+# End /etc/fstab
+EOF
+
+    setup_root_file_system --clean --fstab "${WORKDIR}/fstab" \
+        "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
 
     # Configure root filesystem
     sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
     sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} ${MACHINE_SERIAL} \
-                                                         ${BAUDRATE_TTY} \
-                                                         ${ROOTFS_DEV} \
-                                                         ${ROOTFS_TYPE}
+                                                         ${BAUDRATE_TTY}
 
     # Cleanup
     sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}"
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 91dfc1a..3c10fc7 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -16,9 +16,11 @@ def reverse_bb_array(d, varname):
 
 setup_root_file_system() {
     CLEAN=""
+    FSTAB=""
     while true; do
         case "$1" in
         --clean) CLEAN=1 ;;
+        --fstab) FSTAB=$2; shift ;;
         -*) bbfatal "$0: invalid option specified: $1" ;;
         *) break ;;
         esac
@@ -33,6 +35,7 @@ setup_root_file_system() {
     sudo cp -Trpfx \
         "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}/" \
         "$ROOTFSDIR"
+    [ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee "$ROOTFSDIR/etc/fstab"
 
     echo "deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main" | \
         sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list" >/dev/null
-- 
2.13.7

[PATCH v2 5/6] Add u-boot script generator

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

In order to use upstream U-Boot default envs for booting Isar images, we
either need an extlinux script - can be provided by Debian u-boot-menu
from buster - or some boot.scr. The latter has the advantage of skipping
the interactive stage of a kernel selection, specifically when there is
only one kernel on an image, and allowing for injection of custom U-Boot
commands.

This package generates the needed boot.scr, using the installed kernel
package and the rootfs parameters found in the target's /etc/fstab. It
is able to update itself when the kernel package is changed on the
target. Further kernel parameters and custom U-Boot commands can be
configured on the target via /etc/default/u-boot-script. This also
allows to opt-out from using an initramfs when the kernel does not need
one to boot properly.

Furthermore, the package recipe can evaluate and bootloader --append=...
line from the WKS_FILE used for the target and inject those kernel
parameters into the deployed /etc/default/u-boot-script.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/recipes-bsp/u-boot-script/files/u-boot-script |  9 ++++
 .../u-boot-script/files/update-u-boot-script       | 57 ++++++++++++++++++++++
 .../u-boot-script/files/zz-u-boot-script           |  3 ++
 .../recipes-bsp/u-boot-script/u-boot-script_1.0.bb | 53 ++++++++++++++++++++
 4 files changed, 122 insertions(+)
 create mode 100644 meta/recipes-bsp/u-boot-script/files/u-boot-script
 create mode 100755 meta/recipes-bsp/u-boot-script/files/update-u-boot-script
 create mode 100755 meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
 create mode 100644 meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb

diff --git a/meta/recipes-bsp/u-boot-script/files/u-boot-script b/meta/recipes-bsp/u-boot-script/files/u-boot-script
new file mode 100644
index 0000000..53cc613
--- /dev/null
+++ b/meta/recipes-bsp/u-boot-script/files/u-boot-script
@@ -0,0 +1,9 @@
+# Arguments to append to kernel command line.
+# Make sure to escape $ in u-boot variables, e.g. "\${console}".
+KERNEL_ARGS_APPEND=""
+
+# Set to "yes" or "1" to leave out initrd, even if present
+NO_INITRD=""
+
+# U-boot commands to prepend to boot script
+SCRIPT_PREPEND=""
diff --git a/meta/recipes-bsp/u-boot-script/files/update-u-boot-script b/meta/recipes-bsp/u-boot-script/files/update-u-boot-script
new file mode 100755
index 0000000..ebdc0d5
--- /dev/null
+++ b/meta/recipes-bsp/u-boot-script/files/update-u-boot-script
@@ -0,0 +1,57 @@
+#!/bin/sh
+#
+# Boot script generator for U-Boot
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+set -e
+
+if [ -f /etc/default/u-boot-script ]; then
+	. /etc/default/u-boot-script
+fi
+
+ROOT_DEV=$(egrep '^[^# 	]+[ 	]+/[ 	]' /etc/fstab | awk '{print $1}') \
+	|| true
+
+ROOT_PART="\${distro_bootpart}"
+if [ -n "${ROOT_DEV}" ]; then
+	ROOT_PART=$(echo ${DEV} | sed 's/.*[^0-9]\([0-9]*\)$/\1/')
+fi
+
+BOOT_CMD=$(mktemp)
+
+KERNEL_VERSION=$(linux-version list | linux-version sort --reverse | head -1)
+
+echo "${SCRIPT_PREPEND}" >> ${BOOT_CMD}
+
+echo "setenv bootargs root=${ROOT_DEV} ${KERNEL_ARGS_APPEND}" >> ${BOOT_CMD}
+
+echo "load \${devtype} \${devnum}:${ROOT_PART} \${fdt_addr_r}" \
+     "/usr/lib/linux-image-${KERNEL_VERSION}/\${fdtfile}" >> ${BOOT_CMD}
+echo "load \${devtype} \${devnum}:\${distro_bootpart} \${kernel_addr_r}" \
+     "/boot/vmlinuz-${KERNEL_VERSION}" >> ${BOOT_CMD}
+
+case "${NO_INITRD}" in
+yes|1)
+	INITRD_ADDR="-"
+	;;
+*)
+	echo "load \${devtype} \${devnum}:\${distro_bootpart}" \
+	     "\${ramdisk_addr_r} /boot/initrd.img-${KERNEL_VERSION}" \
+	     >> ${BOOT_CMD}
+	INITRD_ADDR="\${ramdisk_addr_r}:\${filesize}"
+esac
+
+BOOT="bootz"
+if [ "$(uname -m)" == "aarch64" ]; then
+	BOOT="booti"
+fi
+
+echo "${BOOT} \${kernel_addr_r} ${INITRD_ADDR} \${fdt_addr_r}" >> ${BOOT_CMD}
+
+mkimage -T script -A invalid -C none -d ${BOOT_CMD} /boot/boot.scr > /dev/null
+
+rm ${BOOT_CMD}
diff --git a/meta/recipes-bsp/u-boot-script/files/zz-u-boot-script b/meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
new file mode 100755
index 0000000..cf4cc5a
--- /dev/null
+++ b/meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+update-u-boot-script
diff --git a/meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb b/meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb
new file mode 100644
index 0000000..490c306
--- /dev/null
+++ b/meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb
@@ -0,0 +1,53 @@
+# Boot script generator for U-Boot
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg-raw
+
+DESCRIPTION = "Boot script generator for U-Boot"
+
+SRC_URI = " \
+    file://update-u-boot-script \
+    file://u-boot-script \
+    file://zz-u-boot-script"
+
+DEBIAN_DEPENDS = "u-boot-tools, linux-image-${KERNEL_NAME}"
+
+do_install() {
+	# Find WKS_FILE specified for the current target.
+	WKS_DIRS=$(dirname $(which wic))/lib/wic/canned-wks
+	for LAYER in ${BBLAYERS}; do
+		WKS_DIRS="${WKS_DIRS} ${LAYER}/scripts/lib/wic/canned-wks"
+	done
+	for DIR in ${WKS_DIRS}; do
+		if [ -f ${DIR}/${WKS_FILE}.wks ]; then
+			WKS_PATH=${DIR}/${WKS_FILE}.wks
+			break
+		fi
+	done
+
+	# Transfer --append parameters from a bootloader entry in the wks file
+	# to the packaged /etc/default/u-boot-script.
+	if [ -n ${WKS_PATH} ]; then
+		APPEND=$(grep "^bootloader " ${WKS_PATH} | \
+			 sed 's/.* --append=\([^ $]*\).*/\1/')
+		sed -i 's|\(^KERNEL_ARGS_APPEND=\).*|\1'${APPEND}'|' \
+			${WORKDIR}/u-boot-script
+	fi
+
+	sudo rm -rf ${D}/etc ${D}/usr
+
+	install -v -d ${D}/usr/sbin
+	install -v -m 755 ${WORKDIR}/update-u-boot-script ${D}/usr/sbin/
+
+	install -v -d ${D}/etc/default
+	install -v -m 644 ${WORKDIR}/u-boot-script ${D}/etc/default/
+
+	install -v -d ${D}/etc/kernel/postinst.d
+	install -v -m 755 ${WORKDIR}/zz-u-boot-script ${D}/etc/kernel/postinst.d
+
+	sudo chown -R root:root ${D}/etc ${D}/usr
+}
-- 
2.13.7

[PATCH v2 6/6] Add Banana Pi SD-card image

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This generates a bootable SD-card image for the LeMaker Banana Pi board.
As the 4.9 stretch kernel does not work on the board, we need to pull
4.16 from backports. That also gives us a chance to document by example
how multiple external repos and apt preferences can be used with Isar.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .../conf/distro/debian-stretch-backports.list      |  1 +
 meta-isar/conf/local.conf.sample                   |  1 +
 meta-isar/conf/multiconfig/bananapi-stretch.conf   | 24 ++++++++++++++++++++++
 .../conf/multiconfig/preferences.bananapi.conf     |  7 +++++++
 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks  |  5 +++++
 5 files changed, 38 insertions(+)
 create mode 100644 meta-isar/conf/distro/debian-stretch-backports.list
 create mode 100644 meta-isar/conf/multiconfig/bananapi-stretch.conf
 create mode 100644 meta-isar/conf/multiconfig/preferences.bananapi.conf
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks

diff --git a/meta-isar/conf/distro/debian-stretch-backports.list b/meta-isar/conf/distro/debian-stretch-backports.list
new file mode 100644
index 0000000..727a835
--- /dev/null
+++ b/meta-isar/conf/distro/debian-stretch-backports.list
@@ -0,0 +1 @@
+deb	http://ftp.de.debian.org/debian stretch-backports main
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 390cc4c..16ef488 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -46,6 +46,7 @@ BBMULTICONFIG = " \
     qemui386-stretch \
     qemuamd64-jessie \
     qemuamd64-stretch \
+    bananapi-stretch \
     rpi-jessie \
 "
 
diff --git a/meta-isar/conf/multiconfig/bananapi-stretch.conf b/meta-isar/conf/multiconfig/bananapi-stretch.conf
new file mode 100644
index 0000000..a78300a
--- /dev/null
+++ b/meta-isar/conf/multiconfig/bananapi-stretch.conf
@@ -0,0 +1,24 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+MACHINE ?= "bananapi"
+
+DISTRO ?= "debian-stretch"
+DISTRO_ARCH ?= "armhf"
+
+KERNEL_NAME ?= "armmp"
+
+IMAGE_PREINSTALL += "init"
+
+DISTRO_APT_SOURCES_append = " conf/distro/debian-stretch-backports.list"
+DISTRO_APT_PREFERENCES += "conf/multiconfig/preferences.bananapi.conf"
+
+IMAGE_INSTALL += "u-boot-script"
+BUILDCHROOT_PREINSTALL_WIC_append = " u-boot-sunxi"
+
+IMAGE_TYPE ?= "wic-img"
+WKS_FILE ?= "bananapi"
+
+ROOTFS_DEV ?= "mmcblk0p1"
diff --git a/meta-isar/conf/multiconfig/preferences.bananapi.conf b/meta-isar/conf/multiconfig/preferences.bananapi.conf
new file mode 100644
index 0000000..9f5c196
--- /dev/null
+++ b/meta-isar/conf/multiconfig/preferences.bananapi.conf
@@ -0,0 +1,7 @@
+Package: linux-image* linux-headers* linux-kbuild*
+Pin: release n=stretch-backports
+Pin-Priority: 501
+
+Package: *
+Pin: release n=stretch-backports
+Pin-Priority: -1
diff --git a/meta-isar/scripts/lib/wic/canned-wks/bananapi.wks b/meta-isar/scripts/lib/wic/canned-wks/bananapi.wks
new file mode 100644
index 0000000..d60868a
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/bananapi.wks
@@ -0,0 +1,5 @@
+part u-boot --source rawcopy --sourceparams="file=/usr/lib/u-boot/Bananapi/u-boot-sunxi-with-spl.bin" --ondisk mmcblk --no-table --align 8
+
+part / --source rootfs --ondisk mmcblk --fstype=ext4 --label platform --align 1024
+
+bootloader --append="rootwait"
-- 
2.13.7

Re: [PATCH v2 0/6] Enable U-Boot-based images, add Banana Pi demo

[Maxim Yu. Osipov]

Hi Jan,

I've tested the resulting image on Banana Pi.
Banana Pi boots OK - I get login prompt.
The only complain during booting is from fsck:

 >>>>
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top 
... done.
Begin: Running /scripts/local-premount ... done.
[    7.536178] random: fast init done
Begin: Will now check root file system ... fsck from util-linux 2.29.2
fsck: error 2 (No such file or directory) while executing fsck.ext4 for 
/dev/mmcblk0p1
fsck exited with status code 8
done.
Warning: File system check failed but did not detect errors
[   12.782364] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data 
mode. Opts: (null)
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[   13.431139] systemd[1]: System time before build time, advancing clock.
[   13.582647] systemd[1]: systemd 232 running in system mode. (+PAM 
+AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP 
+GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[   13.614171] systemd[1]: Detected architecture arm.

Welcome to Debian GNU/Linux 9 (stretch)!
<<<<<

Kind regards,
Maxim.

On 06/07/2018 04:30 PM, Jan Kiszka wrote:
> Changes in v2:
>   - use booti on aarch64 in generated boot.scr
>   - simplify boot script generator internally (fstab parsing, ramdisk
>     size evaluation)
>   - use options for setup_root_file_system()
> 
> Original cover:
> 
> This provides missing bits to build SD-card (or also eMMC) images for
> boards that use U-Boot as bootloader. The images can be described via
> well-known wks files.
> 
> For U-Boot, we rely on scan_dev_for_scripts from the default
> environment. The reason to generate our own boot script is documented in
> the related patch. While that generator may start its life inside Isar,
> I could imagine selling it also to Debian as a seperate package.
> 
> As an example, I've added the necessary configs to build a basic image
> for a Banana Pi board.
> 
> This is surely not the smoothest solution. E.g., I dislike that we still
> need those ugly ROOTFS_TYPE and ROOTFS_DEV variables - the information
> is encoded in the wks file already. But those files are evaluated by wic
> only AFTER we build the rootfs. So, extracting that information from wks
> would require logic like in u-boot-script_1.0.bb, i.e. a certain level
> wic duplication so that we can continue to generate /etc/fstab ahead of
> the package installation. Better suggestions welcome.
> 
> We should probably also convert BUILDCHROOT_PREINSTALL_WIC to
> WIC_FILE_DEPENDS* as indicated in patch 2. But I didn't study the
> implications in details yet.
> 
> Looking forward to comments and tests!
> 
> Jan
> 
> Jan Kiszka (6):
>    Add DISTRO and DISTRO_ARCH as OVERRIDES suffixes
>    buildchroot: Break up wic build deps into common and arch-specific
>      ones
>    isar-bootstrap: Use options for setup_root_file_system()
>    Set up target image fstab prior to installing any packages
>    Add u-boot script generator
>    Add Banana Pi SD-card image
> 
>   .../conf/distro/debian-stretch-backports.list      |  1 +
>   meta-isar/conf/local.conf.sample                   |  1 +
>   meta-isar/conf/multiconfig/bananapi-stretch.conf   | 24 +++++++++
>   .../conf/multiconfig/preferences.bananapi.conf     |  7 +++
>   .../images/files/debian-configscript.sh            | 14 ------
>   .../images/files/raspbian-configscript.sh          | 14 ------
>   meta-isar/recipes-core/images/isar-image-base.bb   | 20 ++++++--
>   meta-isar/scripts/lib/wic/canned-wks/bananapi.wks  |  5 ++
>   meta/classes/isar-bootstrap-helper.bbclass         | 18 +++++--
>   meta/conf/isar-bitbake.conf                        |  2 +
>   meta/recipes-bsp/u-boot-script/files/u-boot-script |  9 ++++
>   .../u-boot-script/files/update-u-boot-script       | 57 ++++++++++++++++++++++
>   .../u-boot-script/files/zz-u-boot-script           |  3 ++
>   .../recipes-bsp/u-boot-script/u-boot-script_1.0.bb | 53 ++++++++++++++++++++
>   meta/recipes-devtools/buildchroot/buildchroot.bb   | 23 ++++++---
>   15 files changed, 208 insertions(+), 43 deletions(-)
>   create mode 100644 meta-isar/conf/distro/debian-stretch-backports.list
>   create mode 100644 meta-isar/conf/multiconfig/bananapi-stretch.conf
>   create mode 100644 meta-isar/conf/multiconfig/preferences.bananapi.conf
>   create mode 100644 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks
>   create mode 100644 meta/recipes-bsp/u-boot-script/files/u-boot-script
>   create mode 100755 meta/recipes-bsp/u-boot-script/files/update-u-boot-script
>   create mode 100755 meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
>   create mode 100644 meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 0/6] Enable U-Boot-based images, add Banana Pi demo

[Maxim Yu. Osipov]

On 06/07/2018 04:30 PM, Jan Kiszka wrote:
> Changes in v2:
>   - use booti on aarch64 in generated boot.scr
>   - simplify boot script generator internally (fstab parsing, ramdisk
>     size evaluation)
>   - use options for setup_root_file_system()
> 
> Original cover:
> 
> This provides missing bits to build SD-card (or also eMMC) images for
> boards that use U-Boot as bootloader. The images can be described via
> well-known wks files.
> 
> For U-Boot, we rely on scan_dev_for_scripts from the default
> environment. The reason to generate our own boot script is documented in
> the related patch. While that generator may start its life inside Isar,
> I could imagine selling it also to Debian as a seperate package.
> 
> As an example, I've added the necessary configs to build a basic image
> for a Banana Pi board.
> 
> This is surely not the smoothest solution. E.g., I dislike that we still
> need those ugly ROOTFS_TYPE and ROOTFS_DEV variables - the information
> is encoded in the wks file already. But those files are evaluated by wic
> only AFTER we build the rootfs. So, extracting that information from wks
> would require logic like in u-boot-script_1.0.bb, i.e. a certain level
> wic duplication so that we can continue to generate /etc/fstab ahead of
> the package installation. Better suggestions welcome.
> 
> We should probably also convert BUILDCHROOT_PREINSTALL_WIC to
> WIC_FILE_DEPENDS* as indicated in patch 2. But I didn't study the
> implications in details yet.
> 
> Looking forward to comments and tests!

Applied to the 'next',

Thanks,
Maxim.

> Jan
> 
> Jan Kiszka (6):
>    Add DISTRO and DISTRO_ARCH as OVERRIDES suffixes
>    buildchroot: Break up wic build deps into common and arch-specific
>      ones
>    isar-bootstrap: Use options for setup_root_file_system()
>    Set up target image fstab prior to installing any packages
>    Add u-boot script generator
>    Add Banana Pi SD-card image
> 
>   .../conf/distro/debian-stretch-backports.list      |  1 +
>   meta-isar/conf/local.conf.sample                   |  1 +
>   meta-isar/conf/multiconfig/bananapi-stretch.conf   | 24 +++++++++
>   .../conf/multiconfig/preferences.bananapi.conf     |  7 +++
>   .../images/files/debian-configscript.sh            | 14 ------
>   .../images/files/raspbian-configscript.sh          | 14 ------
>   meta-isar/recipes-core/images/isar-image-base.bb   | 20 ++++++--
>   meta-isar/scripts/lib/wic/canned-wks/bananapi.wks  |  5 ++
>   meta/classes/isar-bootstrap-helper.bbclass         | 18 +++++--
>   meta/conf/isar-bitbake.conf                        |  2 +
>   meta/recipes-bsp/u-boot-script/files/u-boot-script |  9 ++++
>   .../u-boot-script/files/update-u-boot-script       | 57 ++++++++++++++++++++++
>   .../u-boot-script/files/zz-u-boot-script           |  3 ++
>   .../recipes-bsp/u-boot-script/u-boot-script_1.0.bb | 53 ++++++++++++++++++++
>   meta/recipes-devtools/buildchroot/buildchroot.bb   | 23 ++++++---
>   15 files changed, 208 insertions(+), 43 deletions(-)
>   create mode 100644 meta-isar/conf/distro/debian-stretch-backports.list
>   create mode 100644 meta-isar/conf/multiconfig/bananapi-stretch.conf
>   create mode 100644 meta-isar/conf/multiconfig/preferences.bananapi.conf
>   create mode 100644 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks
>   create mode 100644 meta/recipes-bsp/u-boot-script/files/u-boot-script
>   create mode 100755 meta/recipes-bsp/u-boot-script/files/update-u-boot-script
>   create mode 100755 meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
>   create mode 100644 meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 0/6] Enable U-Boot-based images, add Banana Pi demo

[Jan Kiszka]

On 2018-06-07 19:59, Maxim Yu. Osipov wrote:
> Hi Jan,
> 
> I've tested the resulting image on Banana Pi.
> Banana Pi boots OK - I get login prompt.
> The only complain during booting is from fsck:
> 
>>>>>
> Begin: Loading essential drivers ... done.
> Begin: Running /scripts/init-premount ... done.
> Begin: Mounting root file system ... Begin: Running /scripts/local-top
> ... done.
> Begin: Running /scripts/local-premount ... done.
> [    7.536178] random: fast init done
> Begin: Will now check root file system ... fsck from util-linux 2.29.2
> fsck: error 2 (No such file or directory) while executing fsck.ext4 for
> /dev/mmcblk0p1
> fsck exited with status code 8
> done.
> Warning: File system check failed but did not detect errors
> [   12.782364] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data
> mode. Opts: (null)
> done.
> Begin: Running /scripts/local-bottom ... done.
> Begin: Running /scripts/init-bottom ... done.
> [   13.431139] systemd[1]: System time before build time, advancing clock.
> [   13.582647] systemd[1]: systemd 232 running in system mode. (+PAM
> +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
> +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
> [   13.614171] systemd[1]: Detected architecture arm.
> 
> Welcome to Debian GNU/Linux 9 (stretch)!
> <<<<<

I think I saw this flying by as well but didn't look into details. I
assume we are missing the corresponding fschk on the base image.

Jan

> 
> Kind regards,
> Maxim.
> 
> On 06/07/2018 04:30 PM, Jan Kiszka wrote:
>> Changes in v2:
>>   - use booti on aarch64 in generated boot.scr
>>   - simplify boot script generator internally (fstab parsing, ramdisk
>>     size evaluation)
>>   - use options for setup_root_file_system()
>>
>> Original cover:
>>
>> This provides missing bits to build SD-card (or also eMMC) images for
>> boards that use U-Boot as bootloader. The images can be described via
>> well-known wks files.
>>
>> For U-Boot, we rely on scan_dev_for_scripts from the default
>> environment. The reason to generate our own boot script is documented in
>> the related patch. While that generator may start its life inside Isar,
>> I could imagine selling it also to Debian as a seperate package.
>>
>> As an example, I've added the necessary configs to build a basic image
>> for a Banana Pi board.
>>
>> This is surely not the smoothest solution. E.g., I dislike that we still
>> need those ugly ROOTFS_TYPE and ROOTFS_DEV variables - the information
>> is encoded in the wks file already. But those files are evaluated by wic
>> only AFTER we build the rootfs. So, extracting that information from wks
>> would require logic like in u-boot-script_1.0.bb, i.e. a certain level
>> wic duplication so that we can continue to generate /etc/fstab ahead of
>> the package installation. Better suggestions welcome.
>>
>> We should probably also convert BUILDCHROOT_PREINSTALL_WIC to
>> WIC_FILE_DEPENDS* as indicated in patch 2. But I didn't study the
>> implications in details yet.
>>
>> Looking forward to comments and tests!
>>
>> Jan
>>
>> Jan Kiszka (6):
>>    Add DISTRO and DISTRO_ARCH as OVERRIDES suffixes
>>    buildchroot: Break up wic build deps into common and arch-specific
>>      ones
>>    isar-bootstrap: Use options for setup_root_file_system()
>>    Set up target image fstab prior to installing any packages
>>    Add u-boot script generator
>>    Add Banana Pi SD-card image
>>
>>   .../conf/distro/debian-stretch-backports.list      |  1 +
>>   meta-isar/conf/local.conf.sample                   |  1 +
>>   meta-isar/conf/multiconfig/bananapi-stretch.conf   | 24 +++++++++
>>   .../conf/multiconfig/preferences.bananapi.conf     |  7 +++
>>   .../images/files/debian-configscript.sh            | 14 ------
>>   .../images/files/raspbian-configscript.sh          | 14 ------
>>   meta-isar/recipes-core/images/isar-image-base.bb   | 20 ++++++--
>>   meta-isar/scripts/lib/wic/canned-wks/bananapi.wks  |  5 ++
>>   meta/classes/isar-bootstrap-helper.bbclass         | 18 +++++--
>>   meta/conf/isar-bitbake.conf                        |  2 +
>>   meta/recipes-bsp/u-boot-script/files/u-boot-script |  9 ++++
>>   .../u-boot-script/files/update-u-boot-script       | 57
>> ++++++++++++++++++++++
>>   .../u-boot-script/files/zz-u-boot-script           |  3 ++
>>   .../recipes-bsp/u-boot-script/u-boot-script_1.0.bb | 53
>> ++++++++++++++++++++
>>   meta/recipes-devtools/buildchroot/buildchroot.bb   | 23 ++++++---
>>   15 files changed, 208 insertions(+), 43 deletions(-)
>>   create mode 100644 meta-isar/conf/distro/debian-stretch-backports.list
>>   create mode 100644 meta-isar/conf/multiconfig/bananapi-stretch.conf
>>   create mode 100644 meta-isar/conf/multiconfig/preferences.bananapi.conf
>>   create mode 100644 meta-isar/scripts/lib/wic/canned-wks/bananapi.wks
>>   create mode 100644 meta/recipes-bsp/u-boot-script/files/u-boot-script
>>   create mode 100755
>> meta/recipes-bsp/u-boot-script/files/update-u-boot-script
>>   create mode 100755
>> meta/recipes-bsp/u-boot-script/files/zz-u-boot-script
>>   create mode 100644 meta/recipes-bsp/u-boot-script/u-boot-script_1.0.bb
>>
> 
> 


-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux