[PATCH v2 0/8] Cleanup rootfs creation

[PATCH v2 0/8] Cleanup rootfs creation

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Hi,

this is the fixed up version.

regards,
Claudius

changes from v1:
 - fixed typo in commit message

Claudius Heine (8):
  isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
    base.bbclass
  move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap
  buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
  isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
  bitbake.conf: remove unneeded and differently used variables
  image.bbclass: make IMAGE_ROOTFS overwritable
  bitbake.conf: set default QEMU_ARCH variables
  buildchroot/configscript: make creation of builder uid/gid idempotent

 meta/classes/base.bbclass                          |  9 +++++++++
 meta/classes/buildchroot.bbclass                   |  2 +-
 meta/classes/image-sdk-extension.bbclass           |  2 +-
 meta/classes/image.bbclass                         |  2 +-
 meta/classes/isar-bootstrap-helper.bbclass         | 14 --------------
 meta/conf/bitbake.conf                             | 13 ++++++++-----
 .../isar-bootstrap/isar-bootstrap-host.bb          | 11 +++--------
 .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
 .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
 .../buildchroot/buildchroot-host.bb                |  1 +
 meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
 .../buildchroot/files/configscript.sh              |  4 ++--
 meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
 13 files changed, 30 insertions(+), 41 deletions(-)

-- 
2.20.1

[PATCH v2 1/8] isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to base.bbclass

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

This also sets the 'HOST_DISTRO' to 'DISTRO' per default. (jessie
support was dropped.)

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/base.bbclass                  |  9 +++++++++
 meta/classes/isar-bootstrap-helper.bbclass | 12 ------------
 2 files changed, 9 insertions(+), 12 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 9a1837f..ca80bf8 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -20,6 +20,15 @@
 
 THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}"
 
+def get_deb_host_arch():
+    import subprocess
+    host_arch = subprocess.check_output(
+        ["dpkg", "--print-architecture"]
+    ).decode('utf-8').strip()
+    return host_arch
+HOST_ARCH ??= "${@get_deb_host_arch()}"
+HOST_DISTRO ??= "${DISTRO}"
+
 die() {
 	bbfatal "$*"
 }
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 38d90fb..59c447b 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -7,18 +7,6 @@
 
 IMAGE_TRANSIENT_PACKAGES ??= ""
 
-def get_deb_host_arch():
-    import subprocess
-    host_arch = subprocess.check_output(
-        ["dpkg", "--print-architecture"]
-    ).decode('utf-8').strip()
-    return host_arch
-
-#Debian Distribution for SDK host
-HOST_DISTRO ?= "debian-stretch"
-#Determine SDK host architecture if not explicitly set
-HOST_ARCH ?= "${@get_deb_host_arch()}"
-
 HOST_DISTRO_APT_SOURCES += "conf/distro/${HOST_DISTRO}.list"
 
 def reverse_bb_array(d, varname):
-- 
2.20.1

[PATCH v2 2/8] move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

This variable is only used in 'isar-bootstrap-host.bb' and mentioned in
'isar-bootstrap.inc'. So it does not make sense to have it available
anywhere else.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/isar-bootstrap-helper.bbclass          | 2 --
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 1 +
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 59c447b..0d77929 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -7,8 +7,6 @@
 
 IMAGE_TRANSIENT_PACKAGES ??= ""
 
-HOST_DISTRO_APT_SOURCES += "conf/distro/${HOST_DISTRO}.list"
-
 def reverse_bb_array(d, varname):
     array = d.getVar(varname, True)
     if array is None:
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index c1b571a..0ec2e2b 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -27,6 +27,7 @@ APTKEYRING = "${WORKDIR}/apt-keyring.gpg"
 DEBOOTSTRAP_KEYRING = ""
 DEPLOY_ISAR_BOOTSTRAP ?= ""
 DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
+HOST_DISTRO_APT_SOURCES += "conf/distro/${HOST_DISTRO}.list"
 
 DISTRO_APT_PREMIRRORS ?= "${@ "http://ftp\.(\S+\.)?debian.org  file:///${REPO_BASE_DIR} \n" if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else "" }"
 
-- 
2.20.1

[PATCH v2 3/8] buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/buildchroot.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass
index 1a87bf4..ea1538a 100644
--- a/meta/classes/buildchroot.bbclass
+++ b/meta/classes/buildchroot.bbclass
@@ -8,7 +8,7 @@ ISAR_CROSS_COMPILE ??= "0"
 # Add dependency from the correct buildchroot: host or target
 python __anonymous() {
     mode = d.getVar('ISAR_CROSS_COMPILE', True)
-    if mode == "0":
+    if mode == "0" or d.getVar('HOST_ARCH') == d.getVar('DISTRO_ARCH'):
         dep = "buildchroot-target:do_build"
         rootfs = d.getVar('BUILDCHROOT_TARGET_DIR', True)
     else:
-- 
2.20.1

[PATCH v2 4/8] isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Each of those are dependent on both HOST_ARCH and DISTRO_ARCH, so the
stamps etc. should not be shared.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/image-sdk-extension.bbclass              |  2 +-
 meta/conf/bitbake.conf                                |  4 ++--
 .../isar-bootstrap/isar-bootstrap-host.bb             | 11 +++--------
 .../isar-bootstrap/isar-bootstrap-target.bb           |  6 +-----
 meta/recipes-devtools/buildchroot/buildchroot-host.bb |  1 +
 meta/recipes-devtools/buildchroot/buildchroot.inc     |  3 +--
 meta/recipes-devtools/sdkchroot/sdkchroot.bb          |  3 +--
 7 files changed, 10 insertions(+), 20 deletions(-)

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 1838080..8df3ed1 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -5,7 +5,7 @@
 #
 # This class extends the image.bbclass to supply the creation of a sdk
 
-SDKCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/sdkchroot-${HOST_DISTRO}-${HOST_ARCH}"
+SDKCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/sdkchroot-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
 do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 do_populate_sdk[depends] = "sdkchroot:do_build"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0e521bb..7c1c89f 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -50,8 +50,8 @@ DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
 DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
 DL_DIR ?= "${TOPDIR}/downloads"
 SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
-BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
-BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
+BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host-${DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}/rootfs"
+BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target-${DISTRO}-${DISTRO_ARCH}/rootfs"
 CACHE = "${TMPDIR}/cache"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
index 08b068f..6d0f5f5 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
@@ -6,17 +6,13 @@
 # SPDX-License-Identifier: MIT
 
 Description = "Minimal host Debian root file system"
+PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${HOST_DISTRO}-${HOST_ARCH}"
-DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}"
-ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}.lock"
+DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
+ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}.lock"
 
 require isar-bootstrap.inc
-inherit isar-bootstrap-helper
 
-do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-
-do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_apt_config_prepare[dirs] = "${WORKDIR}"
 do_apt_config_prepare[vardeps] += "\
                                    APTPREFS \
@@ -47,7 +43,6 @@ addtask apt_config_prepare before do_bootstrap after do_unpack
 
 OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}"
 
-do_bootstrap[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}"
 do_bootstrap[vardeps] += "HOST_DISTRO_APT_SOURCES"
 do_bootstrap() {
     isar_bootstrap --host
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
index 79f3e34..552a317 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
@@ -6,16 +6,13 @@
 # SPDX-License-Identifier: MIT
 
 Description = "Minimal target Debian root file system"
+PF = "${PN}-${DISTRO}-${DISTRO_ARCH}"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}"
 ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}.lock"
 
 require isar-bootstrap.inc
 
-do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-
-do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_apt_config_prepare[dirs] = "${WORKDIR}"
 do_apt_config_prepare[vardeps] += "\
                                    APTPREFS \
@@ -44,7 +41,6 @@ addtask apt_config_prepare before do_bootstrap after do_unpack
 
 OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}"
 
-do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_bootstrap[vardeps] += "DISTRO_APT_SOURCES"
 do_bootstrap() {
     isar_bootstrap
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
index c8c10c0..e8f7f69 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
@@ -6,6 +6,7 @@
 DESCRIPTION = "Isar development filesystem for host"
 
 require buildchroot.inc
+PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
 BUILDCHROOT_PREINSTALL ?= "make \
                            debhelper \
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
index 764bc5e..cdb4c07 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
@@ -12,13 +12,12 @@ SRC_URI = "file://configscript.sh \
            file://common.sh \
            file://deps.sh"
 PV = "1.0"
+PF = "${PN}-${DISTRO}-${DISTRO_ARCH}"
 
 inherit isar-bootstrap-helper
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 BUILDCHROOT_DIR = "${WORKDIR}/rootfs"
 
-do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
                             ${BUILDCHROOT_DIR}/isar-apt \
                             ${BUILDCHROOT_DIR}/downloads \
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index 895a454..249fa94 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -14,6 +14,7 @@ SRC_URI = " \
 PV = "0.1"
 
 inherit isar-bootstrap-helper
+PF = "${PN}-${ROOTFS_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
 SDKCHROOT_PREINSTALL := "debhelper \
                            autotools-dev \
@@ -25,11 +26,9 @@ SDKCHROOT_PREINSTALL := "debhelper \
                            devscripts \
                            equivs"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${HOST_DISTRO}-${HOST_ARCH}"
 S = "${WORKDIR}/rootfs"
 
 do_build[dirs] = "${DEPLOY_DIR_IMAGE}"
-do_build[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}"
 do_build[root_cleandirs] = "${S} \
                             ${S}/isar-apt"
 
-- 
2.20.1

[PATCH v2 5/8] bitbake.conf: remove unneeded and differently used variables

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

IMAGE_CMD and TARGET_ARCH are not used in Isar or any bitbake code.

IMAGE_ROOTFS is used as the directory where image recipes store their
root file system. Pointing that to a global path like '${TMPDIR}/rootfs'
does not make sense for Isar.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/conf/bitbake.conf | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 7c1c89f..21d13ef 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -26,8 +26,6 @@ DEPLOY_DIR = "${TMPDIR}/deploy"
 FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
 FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
 GITDIR = "${DL_DIR}/git"
-IMAGE_CMD = "_NO_DEFINED_IMAGE_TYPES_"
-IMAGE_ROOTFS = "${TMPDIR}/rootfs"
 P = "${PN}-${PV}"
 PF = "${PN}-${PV}-${PR}"
 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
@@ -39,7 +37,6 @@ SRC_URI = "file://${FILE}"
 STAMP = "${TMPDIR}/stamps/${PF}"
 SVNDIR = "${DL_DIR}/svn"
 T = "${WORKDIR}/temp"
-TARGET_ARCH = "${BUILD_ARCH}"
 TMPDIR = "${TOPDIR}/tmp"
 PERSISTENT_DIR = "${TMPDIR}/cache"
 GITPKGV = "${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
-- 
2.20.1

[PATCH v2 6/8] image.bbclass: make IMAGE_ROOTFS overwritable

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

This way recipes that inherit image.bbclass can set this path
themselves. This was previously not possible, because the bitbake.conf
set this variable with '='.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index f9a3052..6881d37 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -5,7 +5,7 @@ PF = "${PN}-${DISTRO}-${MACHINE}"
 
 IMAGE_INSTALL ?= ""
 IMAGE_TYPE    ?= "ext4-img"
-IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
+IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
 
 IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
 
-- 
2.20.1

[PATCH v2 7/8] bitbake.conf: set default QEMU_ARCH variables

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/conf/bitbake.conf | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 21d13ef..37f1963 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -53,6 +53,12 @@ CACHE = "${TMPDIR}/cache"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
 
+# Setting default QEMU_ARCH variables for different DISTRO_ARCH:
+QEMU_ARCH_amd64 = "x86_64"
+QEMU_ARCH_i386 = "i386"
+QEMU_ARCH_armhf = "arm"
+QEMU_ARCH_arm64 = "aarch64"
+
 # Codename of the repository created by the caching class
 DEBDISTRONAME = "isar"
 
-- 
2.20.1

[PATCH v2 8/8] buildchroot/configscript: make creation of builder uid/gid idempotent

[claudius.heine.ext]

From: Claudius Heine <ch@denx.de>

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/recipes-devtools/buildchroot/files/configscript.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh
index 25a21ef..c4d8cf8 100644
--- a/meta/recipes-devtools/buildchroot/files/configscript.sh
+++ b/meta/recipes-devtools/buildchroot/files/configscript.sh
@@ -10,6 +10,6 @@ locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
 locales locales/default_environment_locale select en_US.UTF-8
 END
 
-groupadd --system builder -o --gid $2
-useradd --system -o --uid $1 --gid builder --no-create-home --home /home/builder --no-user-group --comment "Isar buildchroot build user" builder
+grep -q "builder:x:$2" /etc/group || groupadd -f --system builder -o --gid $2
+grep -q "builder:x:$1" /etc/passwd || useradd --system -o --uid $1 --gid builder --no-create-home --home /home/builder --no-user-group --comment "Isar buildchroot build user" builder
 chown -R builder:builder /home/builder
-- 
2.20.1

Re: [PATCH v2 0/8] Cleanup rootfs creation

[Maxim Yu. Osipov]

The same problem with SDK creation discovered in v1 series persists also 
in v2 series.

Just recognized that "quick" CI build fails too. See for details:
http://isar-build.org:8080/job/isar_mosipov_develop/91/console

Please run CI scripts before submitting patches to the mailing list - 
this will save maintainer's time.

Maxim.

On 4/17/19 1:11 PM, claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
> 
> Hi,
> 
> this is the fixed up version.
> 
> regards,
> Claudius
> 
> changes from v1:
>   - fixed typo in commit message
> 
> Claudius Heine (8):
>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>      base.bbclass
>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap
>    buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>    bitbake.conf: remove unneeded and differently used variables
>    image.bbclass: make IMAGE_ROOTFS overwritable
>    bitbake.conf: set default QEMU_ARCH variables
>    buildchroot/configscript: make creation of builder uid/gid idempotent
> 
>   meta/classes/base.bbclass                          |  9 +++++++++
>   meta/classes/buildchroot.bbclass                   |  2 +-
>   meta/classes/image-sdk-extension.bbclass           |  2 +-
>   meta/classes/image.bbclass                         |  2 +-
>   meta/classes/isar-bootstrap-helper.bbclass         | 14 --------------
>   meta/conf/bitbake.conf                             | 13 ++++++++-----
>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 +++--------
>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
>   .../buildchroot/buildchroot-host.bb                |  1 +
>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
>   .../buildchroot/files/configscript.sh              |  4 ++--
>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
>   13 files changed, 30 insertions(+), 41 deletions(-)
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 0/8] Cleanup rootfs creation

[Claudius Heine]

Hi Maxim,

On 22/04/2019 15.48, Maxim Yu. Osipov wrote:
> The same problem with SDK creation discovered in v1 series persists also 
> in v2 series.
> 
> Just recognized that "quick" CI build fails too. See for details:
> http://isar-build.org:8080/job/isar_mosipov_develop/91/console
> 
> Please run CI scripts before submitting patches to the mailing list - 

I did:

http://isar-build.org:8080/job/isar_claudius_ilbers-ci/46/consoleText

That build just fails because of the qemu boot where loading of modules 
fails for some reason, but you said that could be ignored.

> this will save maintainer's time.

Well if you are short on time, then I would suggest looking into better 
tooling.

There are solutions that could automate most of the job you do as a 
maintainer, meaning loading patchset from mailinglist, processing it in 
CI and reporting results back to the mailinglist.

Probably something like this:
https://that.guru/blog/patchwork-and-ci-in-a-tree/

As I suggested months ago that patchwork can help safe time and avoid 
leaving any patches behind. If you don't look into solutions like that 
or write your own scripting, then I don't see much reason for you to 
complain.

By doing that, you might even have enough time to start doing code 
reviews and taking part in design discussions etc. where it is much 
better invested that just pushing the "Run CI build" button and waiting 
for that to finish.

To be fair I might do you injustice and you are performing thorough 
review of every ones patches and every ones code is just that excellent 
that only grammar and spelling mistakes in commit messages are left to 
critic, but then it might just be a matter of improving communication.

Claudius

> 
> Maxim.
> 
> On 4/17/19 1:11 PM, claudius.heine.ext@siemens.com wrote:
>> From: Claudius Heine <ch@denx.de>
>>
>> Hi,
>>
>> this is the fixed up version.
>>
>> regards,
>> Claudius
>>
>> changes from v1:
>>   - fixed typo in commit message
>>
>> Claudius Heine (8):
>>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>>      base.bbclass
>>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap
>>    buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
>>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>>    bitbake.conf: remove unneeded and differently used variables
>>    image.bbclass: make IMAGE_ROOTFS overwritable
>>    bitbake.conf: set default QEMU_ARCH variables
>>    buildchroot/configscript: make creation of builder uid/gid idempotent
>>
>>   meta/classes/base.bbclass                          |  9 +++++++++
>>   meta/classes/buildchroot.bbclass                   |  2 +-
>>   meta/classes/image-sdk-extension.bbclass           |  2 +-
>>   meta/classes/image.bbclass                         |  2 +-
>>   meta/classes/isar-bootstrap-helper.bbclass         | 14 --------------
>>   meta/conf/bitbake.conf                             | 13 ++++++++-----
>>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 +++--------
>>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
>>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
>>   .../buildchroot/buildchroot-host.bb                |  1 +
>>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
>>   .../buildchroot/files/configscript.sh              |  4 ++--
>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
>>   13 files changed, 30 insertions(+), 41 deletions(-)
>>
> 
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH v2 0/8] Cleanup rootfs creation

[Maxim Yu. Osipov]

Hi Claudius,

On 4/24/19 8:53 AM, Claudius Heine wrote:
> Hi Maxim,
> 
> On 22/04/2019 15.48, Maxim Yu. Osipov wrote:
>> The same problem with SDK creation discovered in v1 series persists 
>> also in v2 series.
>>
>> Just recognized that "quick" CI build fails too. See for details:
>> http://isar-build.org:8080/job/isar_mosipov_develop/91/console
>>
>> Please run CI scripts before submitting patches to the mailing list - 
> 
> I did:
> 
> http://isar-build.org:8080/job/isar_claudius_ilbers-ci/46/consoleText
> 
> That build just fails because of the qemu boot where loading of modules 
> fails for some reason, but you said that could be ignored.

Well, I said that one may ignore problems with checking login prompt (CI 
tests need to be improved at this point), but I don't remember that I 
said that one may ignore problems with modules loading.

Actually, the problem with loading modules persists in the current 
'next' starting from the build #291 (when I applied your patch set 
Implement ROOTFS postprocess commands)
http://isar-build.org:8080/job/isar_next/291/console

I'm investigating the reason now - from the first glance the ARM stretch 
kernel version has been changed from 4.9.0-8 to 4.19.0.4.

> 
>> this will save maintainer's time.
> 
> Well if you are short on time, then I would suggest looking into better 
> tooling.
> 
> There are solutions that could automate most of the job you do as a 
> maintainer, meaning loading patchset from mailinglist, processing it in 
> CI and reporting results back to the mailinglist.
> 
> Probably something like this:
> https://that.guru/blog/patchwork-and-ci-in-a-tree/
> 
> As I suggested months ago that patchwork can help safe time and avoid 
> leaving any patches behind. If you don't look into solutions like that 
> or write your own scripting, then I don't see much reason for you to 
> complain.


> By doing that, you might even have enough time to start doing code 
> reviews and taking part in design discussions etc. where it is much 
> better invested that just pushing the "Run CI build" button and waiting 
> for that to finish.


> To be fair I might do you injustice and you are performing thorough 
> review of every ones patches and every ones code is just that excellent 
> that only grammar and spelling mistakes in commit messages are left to 
> critic, but then it might just be a matter of improving communication.

Thanks for the link/your opinion.

Maxim.

> Claudius
> 
>>
>> Maxim.
>>
>> On 4/17/19 1:11 PM, claudius.heine.ext@siemens.com wrote:
>>> From: Claudius Heine <ch@denx.de>
>>>
>>> Hi,
>>>
>>> this is the fixed up version.
>>>
>>> regards,
>>> Claudius
>>>
>>> changes from v1:
>>>   - fixed typo in commit message
>>>
>>> Claudius Heine (8):
>>>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>>>      base.bbclass
>>>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to 
>>> isar-bootstrap
>>>    buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
>>>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>>>    bitbake.conf: remove unneeded and differently used variables
>>>    image.bbclass: make IMAGE_ROOTFS overwritable
>>>    bitbake.conf: set default QEMU_ARCH variables
>>>    buildchroot/configscript: make creation of builder uid/gid idempotent
>>>
>>>   meta/classes/base.bbclass                          |  9 +++++++++
>>>   meta/classes/buildchroot.bbclass                   |  2 +-
>>>   meta/classes/image-sdk-extension.bbclass           |  2 +-
>>>   meta/classes/image.bbclass                         |  2 +-
>>>   meta/classes/isar-bootstrap-helper.bbclass         | 14 --------------
>>>   meta/conf/bitbake.conf                             | 13 ++++++++-----
>>>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 +++--------
>>>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
>>>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
>>>   .../buildchroot/buildchroot-host.bb                |  1 +
>>>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
>>>   .../buildchroot/files/configscript.sh              |  4 ++--
>>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
>>>   13 files changed, 30 insertions(+), 41 deletions(-)
>>>
>>
>>
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 0/8] Cleanup rootfs creation

[Claudius Heine]

Hi Maxim,

On 24/04/2019 10.00, Maxim Yu. Osipov wrote:
> Hi Claudius,
> 
> On 4/24/19 8:53 AM, Claudius Heine wrote:
>> Hi Maxim,
>>
>> On 22/04/2019 15.48, Maxim Yu. Osipov wrote:
>>> The same problem with SDK creation discovered in v1 series persists 
>>> also in v2 series.
>>>
>>> Just recognized that "quick" CI build fails too. See for details:
>>> http://isar-build.org:8080/job/isar_mosipov_develop/91/console
>>>
>>> Please run CI scripts before submitting patches to the mailing list - 
>>
>> I did:
>>
>> http://isar-build.org:8080/job/isar_claudius_ilbers-ci/46/consoleText
>>
>> That build just fails because of the qemu boot where loading of 
>> modules fails for some reason, but you said that could be ignored.
> 
> Well, I said that one may ignore problems with checking login prompt (CI 
> tests need to be improved at this point), but I don't remember that I 
> said that one may ignore problems with modules loading.

I thought you meant that issue, since that occurred on my 
post-processing patchset on my CI as well and you reviewed and merged 
that. It did not happen on your CI run as far as I could see so I 
thought it might be just something that occurred when I try to build 
something on that machine. (different user account, priorities, what ever)

Well those two problems occur pretty intermittent. From what I have seen 
there is no real pattern there. I also tried booting the resulting image 
in qemu at one point locally as well and I could load the module from 
the command line or was already loaded IIRC, so I thought it could be a 
timing issue.

Do we have a different CI configuration? If that is the case can you set 
my CI configuration to the one used when building next?

Claudius

> 
> Actually, the problem with loading modules persists in the current 
> 'next' starting from the build #291 (when I applied your patch set 
> Implement ROOTFS postprocess commands)
> http://isar-build.org:8080/job/isar_next/291/console
> 
> I'm investigating the reason now - from the first glance the ARM stretch 
> kernel version has been changed from 4.9.0-8 to 4.19.0.4.
> 
>>
>>> this will save maintainer's time.
>>
>> Well if you are short on time, then I would suggest looking into 
>> better tooling.
>>
>> There are solutions that could automate most of the job you do as a 
>> maintainer, meaning loading patchset from mailinglist, processing it 
>> in CI and reporting results back to the mailinglist.
>>
>> Probably something like this:
>> https://that.guru/blog/patchwork-and-ci-in-a-tree/
>>
>> As I suggested months ago that patchwork can help safe time and avoid 
>> leaving any patches behind. If you don't look into solutions like that 
>> or write your own scripting, then I don't see much reason for you to 
>> complain.
> 
> 
>> By doing that, you might even have enough time to start doing code 
>> reviews and taking part in design discussions etc. where it is much 
>> better invested that just pushing the "Run CI build" button and 
>> waiting for that to finish.
> 
> 
>> To be fair I might do you injustice and you are performing thorough 
>> review of every ones patches and every ones code is just that 
>> excellent that only grammar and spelling mistakes in commit messages 
>> are left to critic, but then it might just be a matter of improving 
>> communication.
> 
> Thanks for the link/your opinion.
> 
> Maxim.
> 
>> Claudius
>>
>>>
>>> Maxim.
>>>
>>> On 4/17/19 1:11 PM, claudius.heine.ext@siemens.com wrote:
>>>> From: Claudius Heine <ch@denx.de>
>>>>
>>>> Hi,
>>>>
>>>> this is the fixed up version.
>>>>
>>>> regards,
>>>> Claudius
>>>>
>>>> changes from v1:
>>>>   - fixed typo in commit message
>>>>
>>>> Claudius Heine (8):
>>>>    isar-boostrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>>>>      base.bbclass
>>>>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to 
>>>> isar-bootstrap
>>>>    buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
>>>>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>>>>    bitbake.conf: remove unneeded and differently used variables
>>>>    image.bbclass: make IMAGE_ROOTFS overwritable
>>>>    bitbake.conf: set default QEMU_ARCH variables
>>>>    buildchroot/configscript: make creation of builder uid/gid 
>>>> idempotent
>>>>
>>>>   meta/classes/base.bbclass                          |  9 +++++++++
>>>>   meta/classes/buildchroot.bbclass                   |  2 +-
>>>>   meta/classes/image-sdk-extension.bbclass           |  2 +-
>>>>   meta/classes/image.bbclass                         |  2 +-
>>>>   meta/classes/isar-bootstrap-helper.bbclass         | 14 
>>>> --------------
>>>>   meta/conf/bitbake.conf                             | 13 ++++++++-----
>>>>   .../isar-bootstrap/isar-bootstrap-host.bb          | 11 +++--------
>>>>   .../isar-bootstrap/isar-bootstrap-target.bb        |  6 +-----
>>>>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  1 +
>>>>   .../buildchroot/buildchroot-host.bb                |  1 +
>>>>   meta/recipes-devtools/buildchroot/buildchroot.inc  |  3 +--
>>>>   .../buildchroot/files/configscript.sh              |  4 ++--
>>>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb       |  3 +--
>>>>   13 files changed, 30 insertions(+), 41 deletions(-)
>>>>
>>>
>>>
>>
> 
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de