public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
* [PATCH v6 0/9] Cleanup rootfs creation
@ 2019-05-15 10:11 claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage claudius.heine.ext
                   ` (9 more replies)
  0 siblings, 10 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

Hi,

the issue apparently was that the PF variable of buildchroot-host.bb was
overwritten when buildchroot.inc was included. Moving the PF declaration to
buildchroot-target fixed this issue and now resulting paths are correct.

The fast build succeeded:
http://isar-build.org:8080/job/isar_claudius_ilbers_fast_ci/17/

regards,
Claudius

changes from v5:
 - moved PF from buildchroot.inc to buildchroot-target.bb

changes from v4:
 - Maxim rebased patchset to current next

changes from v3:
 - added patch for isar-bootstrap-host
 - fixed HOST_DISTRO for raspbian-stretch
 - fixed cross-compiling paths

changes from v2:
 - replaced 'ROOTFS_ARCH' with 'HOST_ARCH' in sdkchroot.bb to avoid
   populate_sdk issue

changes from v1:
 - fixed typo in commit message

Claudius Heine (9):
  isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  isar-bootstrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
    base.bbclass
  move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap
  buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
  isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
  bitbake.conf: remove unneeded and differently used variables
  image.bbclass: make IMAGE_ROOTFS overwritable
  bitbake.conf: set default QEMU_ARCH variables
  buildchroot/configscript: make creation of builder uid/gid idempotent

 meta-isar/conf/multiconfig/rpi-stretch.conf    |  1 +
 meta/classes/base.bbclass                      |  9 +++++++++
 meta/classes/buildchroot.bbclass               |  2 +-
 meta/classes/image-sdk-extension.bbclass       |  2 +-
 meta/classes/image.bbclass                     |  2 +-
 meta/classes/isar-bootstrap-helper.bbclass     | 18 ++----------------
 meta/conf/bitbake.conf                         | 13 ++++++++-----
 .../isar-bootstrap/isar-bootstrap-host.bb      | 13 +++++--------
 .../isar-bootstrap/isar-bootstrap-target.bb    |  6 +-----
 .../isar-bootstrap/isar-bootstrap.inc          |  1 +
 .../buildchroot/buildchroot-host.bb            |  3 ++-
 .../buildchroot/buildchroot-target.bb          |  1 +
 .../buildchroot/buildchroot.inc                |  2 --
 .../buildchroot/files/configscript.sh          |  4 ++--
 meta/recipes-devtools/sdkchroot/sdkchroot.bb   |  5 ++---
 15 files changed, 37 insertions(+), 45 deletions(-)

-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-21 16:56   ` Jan Kiszka
  2019-05-15 10:11 ` [PATCH v6 2/9] isar-bootstrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to base.bbclass claudius.heine.ext
                   ` (8 subsequent siblings)
  9 siblings, 1 reply; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine, Maxim Yu . Osipov

From: Claudius Heine <ch@denx.de>

isar-bootstrap-host only supports bootstrapping Debian root file
systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
distributions.

Signed-off-by: Claudius Heine <ch@denx.de>
[Maxim: rebased]
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
index 7ee4c61..5501e35 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
@@ -12,6 +12,8 @@ DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}"
 ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}.lock"
 
 require isar-bootstrap.inc
+# We only build debian host buildchroot environments
+DISTRO_BOOTSTRAP_KEYS = ""
 inherit isar-bootstrap-helper
 
 do_generate_keyrings[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 2/9] isar-bootstrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to base.bbclass
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 3/9] move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap claudius.heine.ext
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

This also sets the 'HOST_DISTRO' to 'DISTRO' per default. (jessie
support was dropped.)

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta-isar/conf/multiconfig/rpi-stretch.conf |  1 +
 meta/classes/base.bbclass                   |  9 +++++++++
 meta/classes/isar-bootstrap-helper.bbclass  | 12 ------------
 3 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/meta-isar/conf/multiconfig/rpi-stretch.conf b/meta-isar/conf/multiconfig/rpi-stretch.conf
index ec1f71d..f3fac38 100644
--- a/meta-isar/conf/multiconfig/rpi-stretch.conf
+++ b/meta-isar/conf/multiconfig/rpi-stretch.conf
@@ -12,6 +12,7 @@ MACHINE ?= "rpi"
 # This sets the default distribution configuration:
 DISTRO ?= "raspbian-stretch"
 DISTRO_ARCH ?= "armhf"
+HOST_DISTRO ?= "debian-stretch"
 
 KERNEL_NAME ?= "rpi-rpfv"
 
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 9a1837f..ca80bf8 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -20,6 +20,15 @@
 
 THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}"
 
+def get_deb_host_arch():
+    import subprocess
+    host_arch = subprocess.check_output(
+        ["dpkg", "--print-architecture"]
+    ).decode('utf-8').strip()
+    return host_arch
+HOST_ARCH ??= "${@get_deb_host_arch()}"
+HOST_DISTRO ??= "${DISTRO}"
+
 die() {
 	bbfatal "$*"
 }
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 38d90fb..59c447b 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -7,18 +7,6 @@
 
 IMAGE_TRANSIENT_PACKAGES ??= ""
 
-def get_deb_host_arch():
-    import subprocess
-    host_arch = subprocess.check_output(
-        ["dpkg", "--print-architecture"]
-    ).decode('utf-8').strip()
-    return host_arch
-
-#Debian Distribution for SDK host
-HOST_DISTRO ?= "debian-stretch"
-#Determine SDK host architecture if not explicitly set
-HOST_ARCH ?= "${@get_deb_host_arch()}"
-
 HOST_DISTRO_APT_SOURCES += "conf/distro/${HOST_DISTRO}.list"
 
 def reverse_bb_array(d, varname):
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 3/9] move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 2/9] isar-bootstrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to base.bbclass claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 4/9] buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH claudius.heine.ext
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine, Maxim Yu . Osipov

From: Claudius Heine <ch@denx.de>

This variable is only used in 'isar-bootstrap-host.bb' and mentioned in
'isar-bootstrap.inc'. So it does not make sense to have it available
anywhere else.

Signed-off-by: Claudius Heine <ch@denx.de>
[Maxim: rebased]
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/isar-bootstrap-helper.bbclass          | 2 --
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 1 +
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 59c447b..0d77929 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -7,8 +7,6 @@
 
 IMAGE_TRANSIENT_PACKAGES ??= ""
 
-HOST_DISTRO_APT_SOURCES += "conf/distro/${HOST_DISTRO}.list"
-
 def reverse_bb_array(d, varname):
     array = d.getVar(varname, True)
     if array is None:
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 09aed83..b740c57 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -28,6 +28,7 @@ DEPLOY_ISAR_BOOTSTRAP ?= ""
 DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_gnupg = ",gnupg2"
 DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = ",apt-transport-https,ca-certificates"
+HOST_DISTRO_APT_SOURCES += "conf/distro/${HOST_DISTRO}.list"
 
 DISTRO_APT_PREMIRRORS ?= "${@ "http://ftp\.(\S+\.)?debian.org  file:///${REPO_BASE_DIR} \n" if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else "" }"
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 4/9] buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (2 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 3/9] move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 5/9] isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR claudius.heine.ext
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/buildchroot.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass
index 1a87bf4..ea1538a 100644
--- a/meta/classes/buildchroot.bbclass
+++ b/meta/classes/buildchroot.bbclass
@@ -8,7 +8,7 @@ ISAR_CROSS_COMPILE ??= "0"
 # Add dependency from the correct buildchroot: host or target
 python __anonymous() {
     mode = d.getVar('ISAR_CROSS_COMPILE', True)
-    if mode == "0":
+    if mode == "0" or d.getVar('HOST_ARCH') == d.getVar('DISTRO_ARCH'):
         dep = "buildchroot-target:do_build"
         rootfs = d.getVar('BUILDCHROOT_TARGET_DIR', True)
     else:
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 5/9] isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (3 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 4/9] buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 6/9] bitbake.conf: remove unneeded and differently used variables claudius.heine.ext
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine, Maxim Yu . Osipov

From: Claudius Heine <ch@denx.de>

Each of those are dependent on both HOST_ARCH and DISTRO_ARCH, so the
stamps etc. should not be shared.

Signed-off-by: Claudius Heine <ch@denx.de>
[Maxim: rebased]
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/image-sdk-extension.bbclass              |  2 +-
 meta/classes/isar-bootstrap-helper.bbclass            |  4 ++--
 meta/conf/bitbake.conf                                |  4 ++--
 .../isar-bootstrap/isar-bootstrap-host.bb             | 11 +++--------
 .../isar-bootstrap/isar-bootstrap-target.bb           |  6 +-----
 meta/recipes-devtools/buildchroot/buildchroot-host.bb |  3 ++-
 .../buildchroot/buildchroot-target.bb                 |  1 +
 meta/recipes-devtools/buildchroot/buildchroot.inc     |  2 --
 meta/recipes-devtools/sdkchroot/sdkchroot.bb          |  5 ++---
 9 files changed, 14 insertions(+), 24 deletions(-)

diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 1838080..8df3ed1 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -5,7 +5,7 @@
 #
 # This class extends the image.bbclass to supply the creation of a sdk
 
-SDKCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/sdkchroot-${HOST_DISTRO}-${HOST_ARCH}"
+SDKCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/sdkchroot-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
 do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 do_populate_sdk[depends] = "sdkchroot:do_build"
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 0d77929..8612be2 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -52,8 +52,8 @@ setup_root_file_system() {
         --clean) CLEAN=1 ;;
         --copyisarapt) COPYISARAPT=1 ;;
         --fstab) FSTAB=$2; shift ;;
-        --host-arch) ROOTFS_ARCH=${HOST_ARCH} ;;
-        --host-distro) ROOTFS_DISTRO=${HOST_DISTRO} ;;
+        --host-arch) ROOTFS_ARCH="$2-$ROOTFS_ARCH"; shift ;;
+        --host-distro) ROOTFS_DISTRO="$2"; shift ;;
         --keep-apt-cache) KEEP_APT_CACHE=1 ;;
         -*) bbfatal "$0: invalid option specified: $1" ;;
         *) break ;;
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 3a79df3..6bd644b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -51,8 +51,8 @@ DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
 DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
 DL_DIR ?= "${TOPDIR}/downloads"
 SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
-BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
-BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
+BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}/rootfs"
+BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target-${DISTRO}-${DISTRO_ARCH}/rootfs"
 CACHE = "${TMPDIR}/cache"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
index 5501e35..ee8f362 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
@@ -6,19 +6,15 @@
 # SPDX-License-Identifier: MIT
 
 Description = "Minimal host Debian root file system"
+PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${HOST_DISTRO}-${HOST_ARCH}"
-DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}"
-ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}.lock"
+DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
+ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}.lock"
 
 require isar-bootstrap.inc
 # We only build debian host buildchroot environments
 DISTRO_BOOTSTRAP_KEYS = ""
-inherit isar-bootstrap-helper
 
-do_generate_keyrings[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-
-do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_apt_config_prepare[dirs] = "${WORKDIR}"
 do_apt_config_prepare[vardeps] += "\
                                    APTPREFS \
@@ -49,7 +45,6 @@ addtask apt_config_prepare before do_bootstrap after do_unpack
 
 OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}"
 
-do_bootstrap[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}"
 do_bootstrap[vardeps] += "HOST_DISTRO_APT_SOURCES"
 do_bootstrap() {
     isar_bootstrap --host
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
index 39f12b5..77276e9 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
@@ -6,16 +6,13 @@
 # SPDX-License-Identifier: MIT
 
 Description = "Minimal target Debian root file system"
+PF = "${PN}-${DISTRO}-${DISTRO_ARCH}"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}"
 ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}.lock"
 
 require isar-bootstrap.inc
 
-do_generate_keyrings[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-
-do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_apt_config_prepare[dirs] = "${WORKDIR}"
 do_apt_config_prepare[vardeps] += "\
                                    APTPREFS \
@@ -44,7 +41,6 @@ addtask apt_config_prepare before do_bootstrap after do_unpack
 
 OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}"
 
-do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_bootstrap[vardeps] += "DISTRO_APT_SOURCES"
 do_bootstrap() {
     isar_bootstrap
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
index c8c10c0..73cd548 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
@@ -4,6 +4,7 @@
 # Copyright (C) 2015-2018 ilbers GmbH
 
 DESCRIPTION = "Isar development filesystem for host"
+PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
 require buildchroot.inc
 
@@ -27,5 +28,5 @@ BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \
                                         crossbuild-essential-arm64"
 
 
-PARAMS = "--host-arch --host-distro"
+PARAMS = "--host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}'"
 do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb b/meta/recipes-devtools/buildchroot/buildchroot-target.bb
index ce6bd93..20f4d23 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot-target.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot-target.bb
@@ -4,6 +4,7 @@
 # Copyright (C) 2015-2018 ilbers GmbH
 
 DESCRIPTION = "Isar development filesystem for target"
+PF = "${PN}-${DISTRO}-${DISTRO_ARCH}"
 
 require buildchroot.inc
 
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
index 764bc5e..ca9ce51 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
@@ -15,10 +15,8 @@ PV = "1.0"
 
 inherit isar-bootstrap-helper
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 BUILDCHROOT_DIR = "${WORKDIR}/rootfs"
 
-do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
                             ${BUILDCHROOT_DIR}/isar-apt \
                             ${BUILDCHROOT_DIR}/downloads \
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index 895a454..49aeb47 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -14,6 +14,7 @@ SRC_URI = " \
 PV = "0.1"
 
 inherit isar-bootstrap-helper
+PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
 
 SDKCHROOT_PREINSTALL := "debhelper \
                            autotools-dev \
@@ -25,11 +26,9 @@ SDKCHROOT_PREINSTALL := "debhelper \
                            devscripts \
                            equivs"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${HOST_DISTRO}-${HOST_ARCH}"
 S = "${WORKDIR}/rootfs"
 
 do_build[dirs] = "${DEPLOY_DIR_IMAGE}"
-do_build[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}"
 do_build[root_cleandirs] = "${S} \
                             ${S}/isar-apt"
 
@@ -50,7 +49,7 @@ do_build() {
         packages="${SDKCHROOT_PREINSTALL} crossbuild-essential-${DISTRO_ARCH}"
     fi
 
-    setup_root_file_system --copyisarapt --host-arch --host-distro "${S}" $packages
+    setup_root_file_system --copyisarapt --host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}' "${S}" $packages
 
     # Configure root filesystem
     sudo install -m 644 ${WORKDIR}/README.sdk ${S}
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 6/9] bitbake.conf: remove unneeded and differently used variables
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (4 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 5/9] isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 7/9] image.bbclass: make IMAGE_ROOTFS overwritable claudius.heine.ext
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

IMAGE_CMD and TARGET_ARCH are not used in Isar or any bitbake code.

IMAGE_ROOTFS is used as the directory where image recipes store their
root file system. Pointing that to a global path like '${TMPDIR}/rootfs'
does not make sense for Isar.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/conf/bitbake.conf | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6bd644b..5edc5f1 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -26,8 +26,6 @@ DEPLOY_DIR = "${TMPDIR}/deploy"
 FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
 FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
 GITDIR = "${DL_DIR}/git"
-IMAGE_CMD = "_NO_DEFINED_IMAGE_TYPES_"
-IMAGE_ROOTFS = "${TMPDIR}/rootfs"
 P = "${PN}-${PV}"
 PF = "${PN}-${PV}-${PR}"
 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
@@ -40,7 +38,6 @@ SRC_URI = "file://${FILE}"
 STAMP = "${TMPDIR}/stamps/${PF}"
 SVNDIR = "${DL_DIR}/svn"
 T = "${WORKDIR}/temp"
-TARGET_ARCH = "${BUILD_ARCH}"
 TMPDIR = "${TOPDIR}/tmp"
 PERSISTENT_DIR = "${TMPDIR}/cache"
 GITPKGV = "${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 7/9] image.bbclass: make IMAGE_ROOTFS overwritable
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (5 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 6/9] bitbake.conf: remove unneeded and differently used variables claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 8/9] bitbake.conf: set default QEMU_ARCH variables claudius.heine.ext
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

This way recipes that inherit image.bbclass can set this path
themselves. This was previously not possible, because the bitbake.conf
set this variable with '='.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8e6f2c9..93d9bfc 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -5,7 +5,7 @@ PF = "${PN}-${DISTRO}-${MACHINE}"
 
 IMAGE_INSTALL ?= ""
 IMAGE_TYPE    ?= "ext4-img"
-IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
+IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
 
 IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 8/9] bitbake.conf: set default QEMU_ARCH variables
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (6 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 7/9] image.bbclass: make IMAGE_ROOTFS overwritable claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 10:11 ` [PATCH v6 9/9] buildchroot/configscript: make creation of builder uid/gid idempotent claudius.heine.ext
  2019-05-15 16:01 ` [PATCH v6 0/9] Cleanup rootfs creation Maxim Yu. Osipov
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/conf/bitbake.conf | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 5edc5f1..381c121 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -54,6 +54,12 @@ CACHE = "${TMPDIR}/cache"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
 
+# Setting default QEMU_ARCH variables for different DISTRO_ARCH:
+QEMU_ARCH_amd64 = "x86_64"
+QEMU_ARCH_i386 = "i386"
+QEMU_ARCH_armhf = "arm"
+QEMU_ARCH_arm64 = "aarch64"
+
 # Codename of the repository created by the caching class
 DEBDISTRONAME = "isar"
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v6 9/9] buildchroot/configscript: make creation of builder uid/gid idempotent
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (7 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 8/9] bitbake.conf: set default QEMU_ARCH variables claudius.heine.ext
@ 2019-05-15 10:11 ` claudius.heine.ext
  2019-05-15 16:01 ` [PATCH v6 0/9] Cleanup rootfs creation Maxim Yu. Osipov
  9 siblings, 0 replies; 23+ messages in thread
From: claudius.heine.ext @ 2019-05-15 10:11 UTC (permalink / raw)
  To: isar-users; +Cc: Claudius Heine

From: Claudius Heine <ch@denx.de>

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/recipes-devtools/buildchroot/files/configscript.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh
index 25a21ef..c4d8cf8 100644
--- a/meta/recipes-devtools/buildchroot/files/configscript.sh
+++ b/meta/recipes-devtools/buildchroot/files/configscript.sh
@@ -10,6 +10,6 @@ locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
 locales locales/default_environment_locale select en_US.UTF-8
 END
 
-groupadd --system builder -o --gid $2
-useradd --system -o --uid $1 --gid builder --no-create-home --home /home/builder --no-user-group --comment "Isar buildchroot build user" builder
+grep -q "builder:x:$2" /etc/group || groupadd -f --system builder -o --gid $2
+grep -q "builder:x:$1" /etc/passwd || useradd --system -o --uid $1 --gid builder --no-create-home --home /home/builder --no-user-group --comment "Isar buildchroot build user" builder
 chown -R builder:builder /home/builder
-- 
2.20.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 0/9] Cleanup rootfs creation
  2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
                   ` (8 preceding siblings ...)
  2019-05-15 10:11 ` [PATCH v6 9/9] buildchroot/configscript: make creation of builder uid/gid idempotent claudius.heine.ext
@ 2019-05-15 16:01 ` Maxim Yu. Osipov
  9 siblings, 0 replies; 23+ messages in thread
From: Maxim Yu. Osipov @ 2019-05-15 16:01 UTC (permalink / raw)
  To: claudius.heine.ext, isar-users; +Cc: Claudius Heine

On 5/15/19 12:11 PM, claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
> 
> Hi,
> 
> the issue apparently was that the PF variable of buildchroot-host.bb was
> overwritten when buildchroot.inc was included. Moving the PF declaration to
> buildchroot-target fixed this issue and now resulting paths are correct.
> 
> The fast build succeeded:
> http://isar-build.org:8080/job/isar_claudius_ilbers_fast_ci/17/


Applied to the 'next'.

Regards,
Maxim.

> regards,
> Claudius
> 
> changes from v5:
>   - moved PF from buildchroot.inc to buildchroot-target.bb
> 
> changes from v4:
>   - Maxim rebased patchset to current next
> 
> changes from v3:
>   - added patch for isar-bootstrap-host
>   - fixed HOST_DISTRO for raspbian-stretch
>   - fixed cross-compiling paths
> 
> changes from v2:
>   - replaced 'ROOTFS_ARCH' with 'HOST_ARCH' in sdkchroot.bb to avoid
>     populate_sdk issue
> 
> changes from v1:
>   - fixed typo in commit message
> 
> Claudius Heine (9):
>    isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
>    isar-bootstrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to
>      base.bbclass
>    move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap
>    buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH
>    isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR
>    bitbake.conf: remove unneeded and differently used variables
>    image.bbclass: make IMAGE_ROOTFS overwritable
>    bitbake.conf: set default QEMU_ARCH variables
>    buildchroot/configscript: make creation of builder uid/gid idempotent
> 
>   meta-isar/conf/multiconfig/rpi-stretch.conf    |  1 +
>   meta/classes/base.bbclass                      |  9 +++++++++
>   meta/classes/buildchroot.bbclass               |  2 +-
>   meta/classes/image-sdk-extension.bbclass       |  2 +-
>   meta/classes/image.bbclass                     |  2 +-
>   meta/classes/isar-bootstrap-helper.bbclass     | 18 ++----------------
>   meta/conf/bitbake.conf                         | 13 ++++++++-----
>   .../isar-bootstrap/isar-bootstrap-host.bb      | 13 +++++--------
>   .../isar-bootstrap/isar-bootstrap-target.bb    |  6 +-----
>   .../isar-bootstrap/isar-bootstrap.inc          |  1 +
>   .../buildchroot/buildchroot-host.bb            |  3 ++-
>   .../buildchroot/buildchroot-target.bb          |  1 +
>   .../buildchroot/buildchroot.inc                |  2 --
>   .../buildchroot/files/configscript.sh          |  4 ++--
>   meta/recipes-devtools/sdkchroot/sdkchroot.bb   |  5 ++---
>   15 files changed, 37 insertions(+), 45 deletions(-)
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-15 10:11 ` [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage claudius.heine.ext
@ 2019-05-21 16:56   ` Jan Kiszka
  2019-05-22  6:37     ` Claudius Heine
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Kiszka @ 2019-05-21 16:56 UTC (permalink / raw)
  To: [ext] claudius.heine.ext@siemens.com, isar-users
  Cc: Claudius Heine, Maxim Yu . Osipov

On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
> 
> isar-bootstrap-host only supports bootstrapping Debian root file
> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
> distributions.

Actually not totally true, as I just realized: What about bootstrapping the 
buildchroot from a custom debian repo that was differently signed (e.g. because 
it is a condensed version of upstream)? Seems we do need 
HOST_DISTRO_BOOTSTRAP_KEYS, right?

Jan

> 
> Signed-off-by: Claudius Heine <ch@denx.de>
> [Maxim: rebased]
> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
> ---
>   meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> index 7ee4c61..5501e35 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> @@ -12,6 +12,8 @@ DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}"
>   ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}.lock"
>   
>   require isar-bootstrap.inc
> +# We only build debian host buildchroot environments
> +DISTRO_BOOTSTRAP_KEYS = ""
>   inherit isar-bootstrap-helper
>   
>   do_generate_keyrings[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-21 16:56   ` Jan Kiszka
@ 2019-05-22  6:37     ` Claudius Heine
  2019-05-22  6:55       ` Maxim Yu. Osipov
  0 siblings, 1 reply; 23+ messages in thread
From: Claudius Heine @ 2019-05-22  6:37 UTC (permalink / raw)
  To: Jan Kiszka, isar-users; +Cc: Claudius Heine, Maxim Yu . Osipov

Hi Jan,

On 21/05/2019 18.56, Jan Kiszka wrote:
> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>> From: Claudius Heine <ch@denx.de>
>>
>> isar-bootstrap-host only supports bootstrapping Debian root file
>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>> distributions.
> 
> Actually not totally true, as I just realized: What about bootstrapping 
> the buildchroot from a custom debian repo that was differently signed 
> (e.g. because it is a condensed version of upstream)? Seems we do need 
> HOST_DISTRO_BOOTSTRAP_KEYS, right?

You are right, I haven't considered that case. Well, the next task on 
the todo list should probably be to refactor and streamline the 
isar-bootstrap, and especially the host bootstrap process and fix those 
kind of issues while doing that.

There are a lot of possible customization options gained if the current 
giant bootstrap function would be split up, similar to how the rootfs 
system works in the pre-processing patchset.

Maybe it makes sense to also start renaming "host" and "target" to be 
compatible with the gcc nomenclature [1] in that patchset. I am a bit 
reluctant to do so, because of the breakage involved. But the further we 
wait, to more stuff will break downstream.

The plan would be to rename all occurrences of "host" to "build" and 
"target" to "host". That would lead to the following recipe changes:

"buildchroot-host" -> "buildchroot-build"
"buildchroot-target" -> "buildchroot-host"
"isar-bootstrap-host" -> "isar-bootstrap-build"
"isar-bootstrap-target" -> "isar-bootstrap-host"

I am on the fence of that change. Correctness vs. no-breakage

Any comments about that?

Cheers,
Claudius

[1] https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html

> 
> Jan
> 
>>
>> Signed-off-by: Claudius Heine <ch@denx.de>
>> [Maxim: rebased]
>> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
>> ---
>>   meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb 
>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
>> index 7ee4c61..5501e35 100644
>> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
>> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
>> @@ -12,6 +12,8 @@ DEPLOY_ISAR_BOOTSTRAP = 
>> "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}"
>>   ISAR_BOOTSTRAP_LOCK = 
>> "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}.lock"
>>   require isar-bootstrap.inc
>> +# We only build debian host buildchroot environments
>> +DISTRO_BOOTSTRAP_KEYS = ""
>>   inherit isar-bootstrap-helper
>>   do_generate_keyrings[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22  6:37     ` Claudius Heine
@ 2019-05-22  6:55       ` Maxim Yu. Osipov
  2019-05-22  7:02         ` Jan Kiszka
  0 siblings, 1 reply; 23+ messages in thread
From: Maxim Yu. Osipov @ 2019-05-22  6:55 UTC (permalink / raw)
  To: Claudius Heine, Jan Kiszka, isar-users; +Cc: Claudius Heine

On 5/22/19 8:37 AM, Claudius Heine wrote:
> Hi Jan,
> 
> On 21/05/2019 18.56, Jan Kiszka wrote:
>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>> From: Claudius Heine <ch@denx.de>
>>>
>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>> distributions.
>>
>> Actually not totally true, as I just realized: What about 
>> bootstrapping the buildchroot from a custom debian repo that was 
>> differently signed (e.g. because it is a condensed version of 
>> upstream)? Seems we do need HOST_DISTRO_BOOTSTRAP_KEYS, right?
> 
> You are right, I haven't considered that case. Well, the next task on 
> the todo list should probably be to refactor and streamline the 
> isar-bootstrap, and especially the host bootstrap process and fix those 
> kind of issues while doing that.
> 
> There are a lot of possible customization options gained if the current 
> giant bootstrap function would be split up, similar to how the rootfs 
> system works in the pre-processing patchset.
> 
> Maybe it makes sense to also start renaming "host" and "target" to be 
> compatible with the gcc nomenclature [1] in that patchset. I am a bit 
> reluctant to do so, because of the breakage involved. But the further we 
> wait, to more stuff will break downstream.
> 
> The plan would be to rename all occurrences of "host" to "build" and 
> "target" to "host". That would lead to the following recipe changes:


> "buildchroot-host" -> "buildchroot-build"
> "buildchroot-target" -> "buildchroot-host"
> "isar-bootstrap-host" -> "isar-bootstrap-build"
> "isar-bootstrap-target" -> "isar-bootstrap-host"


> I am on the fence of that change. Correctness vs. no-breakage
> 
> Any comments about that?

I would prefer to avoid such a renaming taking into account needed 
efforts and possible confusion for current Isar users.

Regards,
Maixm.

> Cheers,
> Claudius
> 
> [1] https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html
> 
>>
>> Jan
>>
>>>
>>> Signed-off-by: Claudius Heine <ch@denx.de>
>>> [Maxim: rebased]
>>> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
>>> ---
>>>   meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb | 2 ++
>>>   1 file changed, 2 insertions(+)
>>>
>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb 
>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
>>> index 7ee4c61..5501e35 100644
>>> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
>>> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
>>> @@ -12,6 +12,8 @@ DEPLOY_ISAR_BOOTSTRAP = 
>>> "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}"
>>>   ISAR_BOOTSTRAP_LOCK = 
>>> "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}.lock"
>>>   require isar-bootstrap.inc
>>> +# We only build debian host buildchroot environments
>>> +DISTRO_BOOTSTRAP_KEYS = ""
>>>   inherit isar-bootstrap-helper
>>>   do_generate_keyrings[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>>
>>
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22  6:55       ` Maxim Yu. Osipov
@ 2019-05-22  7:02         ` Jan Kiszka
  2019-05-22  7:31           ` Claudius Heine
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Kiszka @ 2019-05-22  7:02 UTC (permalink / raw)
  To: Maxim Yu. Osipov, Claudius Heine, isar-users; +Cc: Claudius Heine

On 22.05.19 08:55, Maxim Yu. Osipov wrote:
> On 5/22/19 8:37 AM, Claudius Heine wrote:
>> Hi Jan,
>>
>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>> From: Claudius Heine <ch@denx.de>
>>>>
>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>>> distributions.
>>>
>>> Actually not totally true, as I just realized: What about bootstrapping the 
>>> buildchroot from a custom debian repo that was differently signed (e.g. 
>>> because it is a condensed version of upstream)? Seems we do need 
>>> HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>
>> You are right, I haven't considered that case. Well, the next task on the todo 
>> list should probably be to refactor and streamline the isar-bootstrap, and 
>> especially the host bootstrap process and fix those kind of issues while doing 
>> that.
>>
>> There are a lot of possible customization options gained if the current giant 
>> bootstrap function would be split up, similar to how the rootfs system works 
>> in the pre-processing patchset.
>>
>> Maybe it makes sense to also start renaming "host" and "target" to be 
>> compatible with the gcc nomenclature [1] in that patchset. I am a bit 
>> reluctant to do so, because of the breakage involved. But the further we wait, 
>> to more stuff will break downstream.
>>
>> The plan would be to rename all occurrences of "host" to "build" and "target" 
>> to "host". That would lead to the following recipe changes:
> 
> 
>> "buildchroot-host" -> "buildchroot-build"
>> "buildchroot-target" -> "buildchroot-host"
>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>> "isar-bootstrap-target" -> "isar-bootstrap-host"
> 
> 
>> I am on the fence of that change. Correctness vs. no-breakage
>>
>> Any comments about that?
> 
> I would prefer to avoid such a renaming taking into account needed efforts and 
> possible confusion for current Isar users.
> 

Is that build/host scheme then also in line with Debian naming? Then there will 
be eventually no way around it anyway. But we really need to do this thoroughly, 
specifically /wrt to user-visible interfaces, so that it will be one cut only.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22  7:02         ` Jan Kiszka
@ 2019-05-22  7:31           ` Claudius Heine
  2019-05-22 11:35             ` Jan Kiszka
  0 siblings, 1 reply; 23+ messages in thread
From: Claudius Heine @ 2019-05-22  7:31 UTC (permalink / raw)
  To: Jan Kiszka, Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine

Hi Jan,

On 22/05/2019 09.02, Jan Kiszka wrote:
> On 22.05.19 08:55, Maxim Yu. Osipov wrote:
>> On 5/22/19 8:37 AM, Claudius Heine wrote:
>>> Hi Jan,
>>>
>>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>>> From: Claudius Heine <ch@denx.de>
>>>>>
>>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>>>> distributions.
>>>>
>>>> Actually not totally true, as I just realized: What about 
>>>> bootstrapping the buildchroot from a custom debian repo that was 
>>>> differently signed (e.g. because it is a condensed version of 
>>>> upstream)? Seems we do need HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>>
>>> You are right, I haven't considered that case. Well, the next task on 
>>> the todo list should probably be to refactor and streamline the 
>>> isar-bootstrap, and especially the host bootstrap process and fix 
>>> those kind of issues while doing that.
>>>
>>> There are a lot of possible customization options gained if the 
>>> current giant bootstrap function would be split up, similar to how 
>>> the rootfs system works in the pre-processing patchset.
>>>
>>> Maybe it makes sense to also start renaming "host" and "target" to be 
>>> compatible with the gcc nomenclature [1] in that patchset. I am a bit 
>>> reluctant to do so, because of the breakage involved. But the further 
>>> we wait, to more stuff will break downstream.
>>>
>>> The plan would be to rename all occurrences of "host" to "build" and 
>>> "target" to "host". That would lead to the following recipe changes:
>>
>>
>>> "buildchroot-host" -> "buildchroot-build"
>>> "buildchroot-target" -> "buildchroot-host"
>>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>>> "isar-bootstrap-target" -> "isar-bootstrap-host"
>>
>>
>>> I am on the fence of that change. Correctness vs. no-breakage
>>>
>>> Any comments about that?
>>
>> I would prefer to avoid such a renaming taking into account needed 
>> efforts and possible confusion for current Isar users.
>>
> 
> Is that build/host scheme then also in line with Debian naming? Then 
> there will be eventually no way around it anyway. But we really need to 
> do this thoroughly, specifically /wrt to user-visible interfaces, so 
> that it will be one cut only.

 From dpkg-buildpackage(1):

[...]
        -a, --host-arch architecture
               Specify  the  Debian  architecture  we  build  for (long 
option since dpkg
               1.17.17).  The architecture of the  machine  we  build 
on  is  determined
               automatically, and is also the default for the host machine.
[...]
        --target-arch architecture
               Specify  the  Debian architecture the binaries built will 
build for (since
               dpkg 1.17.17).  The default value is the host machine.
[...]

Claudius

> 
> Jan
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22  7:31           ` Claudius Heine
@ 2019-05-22 11:35             ` Jan Kiszka
  2019-05-22 11:39               ` Claudius Heine
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Kiszka @ 2019-05-22 11:35 UTC (permalink / raw)
  To: Claudius Heine, Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine

On 22.05.19 09:31, Claudius Heine wrote:
> Hi Jan,
> 
> On 22/05/2019 09.02, Jan Kiszka wrote:
>> On 22.05.19 08:55, Maxim Yu. Osipov wrote:
>>> On 5/22/19 8:37 AM, Claudius Heine wrote:
>>>> Hi Jan,
>>>>
>>>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>
>>>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>>>>> distributions.
>>>>>
>>>>> Actually not totally true, as I just realized: What about bootstrapping the 
>>>>> buildchroot from a custom debian repo that was differently signed (e.g. 
>>>>> because it is a condensed version of upstream)? Seems we do need 
>>>>> HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>>>
>>>> You are right, I haven't considered that case. Well, the next task on the 
>>>> todo list should probably be to refactor and streamline the isar-bootstrap, 
>>>> and especially the host bootstrap process and fix those kind of issues while 
>>>> doing that.
>>>>
>>>> There are a lot of possible customization options gained if the current 
>>>> giant bootstrap function would be split up, similar to how the rootfs system 
>>>> works in the pre-processing patchset.
>>>>
>>>> Maybe it makes sense to also start renaming "host" and "target" to be 
>>>> compatible with the gcc nomenclature [1] in that patchset. I am a bit 
>>>> reluctant to do so, because of the breakage involved. But the further we 
>>>> wait, to more stuff will break downstream.
>>>>
>>>> The plan would be to rename all occurrences of "host" to "build" and 
>>>> "target" to "host". That would lead to the following recipe changes:
>>>
>>>
>>>> "buildchroot-host" -> "buildchroot-build"
>>>> "buildchroot-target" -> "buildchroot-host"
>>>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>>>> "isar-bootstrap-target" -> "isar-bootstrap-host"
>>>
>>>
>>>> I am on the fence of that change. Correctness vs. no-breakage
>>>>
>>>> Any comments about that?
>>>
>>> I would prefer to avoid such a renaming taking into account needed efforts 
>>> and possible confusion for current Isar users.
>>>
>>
>> Is that build/host scheme then also in line with Debian naming? Then there 
>> will be eventually no way around it anyway. But we really need to do this 
>> thoroughly, specifically /wrt to user-visible interfaces, so that it will be 
>> one cut only.
> 
>  From dpkg-buildpackage(1):
> 
> [...]
>         -a, --host-arch architecture
>                Specify  the  Debian  architecture  we  build  for (long option 
> since dpkg
>                1.17.17).  The architecture of the  machine  we  build on  is  
> determined
>                automatically, and is also the default for the host machine.
> [...]
>         --target-arch architecture
>                Specify  the  Debian architecture the binaries built will build 
> for (since
>                dpkg 1.17.17).  The default value is the host machine.
> [...]
> 

Then the current naming scheme is already accurate for Debian.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22 11:35             ` Jan Kiszka
@ 2019-05-22 11:39               ` Claudius Heine
  2019-05-22 12:15                 ` Jan Kiszka
  0 siblings, 1 reply; 23+ messages in thread
From: Claudius Heine @ 2019-05-22 11:39 UTC (permalink / raw)
  To: Jan Kiszka, Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine

On 22/05/2019 13.35, Jan Kiszka wrote:
> On 22.05.19 09:31, Claudius Heine wrote:
>> Hi Jan,
>>
>> On 22/05/2019 09.02, Jan Kiszka wrote:
>>> On 22.05.19 08:55, Maxim Yu. Osipov wrote:
>>>> On 5/22/19 8:37 AM, Claudius Heine wrote:
>>>>> Hi Jan,
>>>>>
>>>>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>>>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>>
>>>>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>>>>>> distributions.
>>>>>>
>>>>>> Actually not totally true, as I just realized: What about 
>>>>>> bootstrapping the buildchroot from a custom debian repo that was 
>>>>>> differently signed (e.g. because it is a condensed version of 
>>>>>> upstream)? Seems we do need HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>>>>
>>>>> You are right, I haven't considered that case. Well, the next task 
>>>>> on the todo list should probably be to refactor and streamline the 
>>>>> isar-bootstrap, and especially the host bootstrap process and fix 
>>>>> those kind of issues while doing that.
>>>>>
>>>>> There are a lot of possible customization options gained if the 
>>>>> current giant bootstrap function would be split up, similar to how 
>>>>> the rootfs system works in the pre-processing patchset.
>>>>>
>>>>> Maybe it makes sense to also start renaming "host" and "target" to 
>>>>> be compatible with the gcc nomenclature [1] in that patchset. I am 
>>>>> a bit reluctant to do so, because of the breakage involved. But the 
>>>>> further we wait, to more stuff will break downstream.
>>>>>
>>>>> The plan would be to rename all occurrences of "host" to "build" 
>>>>> and "target" to "host". That would lead to the following recipe 
>>>>> changes:
>>>>
>>>>
>>>>> "buildchroot-host" -> "buildchroot-build"
>>>>> "buildchroot-target" -> "buildchroot-host"
>>>>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>>>>> "isar-bootstrap-target" -> "isar-bootstrap-host"
>>>>
>>>>
>>>>> I am on the fence of that change. Correctness vs. no-breakage
>>>>>
>>>>> Any comments about that?
>>>>
>>>> I would prefer to avoid such a renaming taking into account needed 
>>>> efforts and possible confusion for current Isar users.
>>>>
>>>
>>> Is that build/host scheme then also in line with Debian naming? Then 
>>> there will be eventually no way around it anyway. But we really need 
>>> to do this thoroughly, specifically /wrt to user-visible interfaces, 
>>> so that it will be one cut only.
>>
>>  From dpkg-buildpackage(1):
>>
>> [...]
>>         -a, --host-arch architecture
>>                Specify  the  Debian  architecture  we  build  for 
>> (long option since dpkg
>>                1.17.17).  The architecture of the  machine  we  build 
>> on  is determined
>>                automatically, and is also the default for the host 
>> machine.
>> [...]
>>         --target-arch architecture
>>                Specify  the  Debian architecture the binaries built 
>> will build for (since
>>                dpkg 1.17.17).  The default value is the host machine.
>> [...]
>>
> 
> Then the current naming scheme is already accurate for Debian.

No? That is not how I read that documentation. "Host" is what we build 
for and 'Target' is what the binaries we build generate code for (which 
makes only sense for compilers)

Claudius

> 
> Jan
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22 11:39               ` Claudius Heine
@ 2019-05-22 12:15                 ` Jan Kiszka
  2019-05-22 12:50                   ` Jan Kiszka
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Kiszka @ 2019-05-22 12:15 UTC (permalink / raw)
  To: Claudius Heine, Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine

On 22.05.19 13:39, Claudius Heine wrote:
> On 22/05/2019 13.35, Jan Kiszka wrote:
>> On 22.05.19 09:31, Claudius Heine wrote:
>>> Hi Jan,
>>>
>>> On 22/05/2019 09.02, Jan Kiszka wrote:
>>>> On 22.05.19 08:55, Maxim Yu. Osipov wrote:
>>>>> On 5/22/19 8:37 AM, Claudius Heine wrote:
>>>>>> Hi Jan,
>>>>>>
>>>>>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>>>>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>>>
>>>>>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>>>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>>>>>>> distributions.
>>>>>>>
>>>>>>> Actually not totally true, as I just realized: What about bootstrapping 
>>>>>>> the buildchroot from a custom debian repo that was differently signed 
>>>>>>> (e.g. because it is a condensed version of upstream)? Seems we do need 
>>>>>>> HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>>>>>
>>>>>> You are right, I haven't considered that case. Well, the next task on the 
>>>>>> todo list should probably be to refactor and streamline the 
>>>>>> isar-bootstrap, and especially the host bootstrap process and fix those 
>>>>>> kind of issues while doing that.
>>>>>>
>>>>>> There are a lot of possible customization options gained if the current 
>>>>>> giant bootstrap function would be split up, similar to how the rootfs 
>>>>>> system works in the pre-processing patchset.
>>>>>>
>>>>>> Maybe it makes sense to also start renaming "host" and "target" to be 
>>>>>> compatible with the gcc nomenclature [1] in that patchset. I am a bit 
>>>>>> reluctant to do so, because of the breakage involved. But the further we 
>>>>>> wait, to more stuff will break downstream.
>>>>>>
>>>>>> The plan would be to rename all occurrences of "host" to "build" and 
>>>>>> "target" to "host". That would lead to the following recipe changes:
>>>>>
>>>>>
>>>>>> "buildchroot-host" -> "buildchroot-build"
>>>>>> "buildchroot-target" -> "buildchroot-host"
>>>>>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>>>>>> "isar-bootstrap-target" -> "isar-bootstrap-host"
>>>>>
>>>>>
>>>>>> I am on the fence of that change. Correctness vs. no-breakage
>>>>>>
>>>>>> Any comments about that?
>>>>>
>>>>> I would prefer to avoid such a renaming taking into account needed efforts 
>>>>> and possible confusion for current Isar users.
>>>>>
>>>>
>>>> Is that build/host scheme then also in line with Debian naming? Then there 
>>>> will be eventually no way around it anyway. But we really need to do this 
>>>> thoroughly, specifically /wrt to user-visible interfaces, so that it will be 
>>>> one cut only.
>>>
>>>  From dpkg-buildpackage(1):
>>>
>>> [...]
>>>         -a, --host-arch architecture
>>>                Specify  the  Debian  architecture  we  build  for (long 
>>> option since dpkg
>>>                1.17.17).  The architecture of the  machine  we  build on  is 
>>> determined
>>>                automatically, and is also the default for the host machine.
>>> [...]
>>>         --target-arch architecture
>>>                Specify  the  Debian architecture the binaries built will 
>>> build for (since
>>>                dpkg 1.17.17).  The default value is the host machine.
>>> [...]
>>>
>>
>> Then the current naming scheme is already accurate for Debian.
> 
> No? That is not how I read that documentation. "Host" is what we build for and 
> 'Target' is what the binaries we build generate code for (which makes only sense 
> for compilers)

https://wiki.debian.org/BuildingCrossCompilers

"Nomenclature

TARGET is the architecture the compiler builds code for. HOST [...] is the 
architecture the compiler runs on."

Mapping that on our buildchroots: buildchroot-host is where the compiler runs 
natively, buildchroot-target is where it runs on the target architecture. Thus, 
no need to flip anything.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22 12:15                 ` Jan Kiszka
@ 2019-05-22 12:50                   ` Jan Kiszka
  2019-05-22 13:37                     ` Claudius Heine
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Kiszka @ 2019-05-22 12:50 UTC (permalink / raw)
  To: Claudius Heine, Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine

On 22.05.19 14:15, [ext] Jan Kiszka wrote:
> On 22.05.19 13:39, Claudius Heine wrote:
>> On 22/05/2019 13.35, Jan Kiszka wrote:
>>> On 22.05.19 09:31, Claudius Heine wrote:
>>>> Hi Jan,
>>>>
>>>> On 22/05/2019 09.02, Jan Kiszka wrote:
>>>>> On 22.05.19 08:55, Maxim Yu. Osipov wrote:
>>>>>> On 5/22/19 8:37 AM, Claudius Heine wrote:
>>>>>>> Hi Jan,
>>>>>>>
>>>>>>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>>>>>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>>>>
>>>>>>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>>>>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from other
>>>>>>>>> distributions.
>>>>>>>>
>>>>>>>> Actually not totally true, as I just realized: What about bootstrapping 
>>>>>>>> the buildchroot from a custom debian repo that was differently signed 
>>>>>>>> (e.g. because it is a condensed version of upstream)? Seems we do need 
>>>>>>>> HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>>>>>>
>>>>>>> You are right, I haven't considered that case. Well, the next task on the 
>>>>>>> todo list should probably be to refactor and streamline the 
>>>>>>> isar-bootstrap, and especially the host bootstrap process and fix those 
>>>>>>> kind of issues while doing that.
>>>>>>>
>>>>>>> There are a lot of possible customization options gained if the current 
>>>>>>> giant bootstrap function would be split up, similar to how the rootfs 
>>>>>>> system works in the pre-processing patchset.
>>>>>>>
>>>>>>> Maybe it makes sense to also start renaming "host" and "target" to be 
>>>>>>> compatible with the gcc nomenclature [1] in that patchset. I am a bit 
>>>>>>> reluctant to do so, because of the breakage involved. But the further we 
>>>>>>> wait, to more stuff will break downstream.
>>>>>>>
>>>>>>> The plan would be to rename all occurrences of "host" to "build" and 
>>>>>>> "target" to "host". That would lead to the following recipe changes:
>>>>>>
>>>>>>
>>>>>>> "buildchroot-host" -> "buildchroot-build"
>>>>>>> "buildchroot-target" -> "buildchroot-host"
>>>>>>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>>>>>>> "isar-bootstrap-target" -> "isar-bootstrap-host"
>>>>>>
>>>>>>
>>>>>>> I am on the fence of that change. Correctness vs. no-breakage
>>>>>>>
>>>>>>> Any comments about that?
>>>>>>
>>>>>> I would prefer to avoid such a renaming taking into account needed efforts 
>>>>>> and possible confusion for current Isar users.
>>>>>>
>>>>>
>>>>> Is that build/host scheme then also in line with Debian naming? Then there 
>>>>> will be eventually no way around it anyway. But we really need to do this 
>>>>> thoroughly, specifically /wrt to user-visible interfaces, so that it will 
>>>>> be one cut only.
>>>>
>>>>  From dpkg-buildpackage(1):
>>>>
>>>> [...]
>>>>         -a, --host-arch architecture
>>>>                Specify  the  Debian  architecture  we  build  for (long 
>>>> option since dpkg
>>>>                1.17.17).  The architecture of the  machine  we  build on  is 
>>>> determined
>>>>                automatically, and is also the default for the host machine.
>>>> [...]
>>>>         --target-arch architecture
>>>>                Specify  the  Debian architecture the binaries built will 
>>>> build for (since
>>>>                dpkg 1.17.17).  The default value is the host machine.
>>>> [...]
>>>>
>>>
>>> Then the current naming scheme is already accurate for Debian.
>>
>> No? That is not how I read that documentation. "Host" is what we build for and 
>> 'Target' is what the binaries we build generate code for (which makes only 
>> sense for compilers)
> 
> https://wiki.debian.org/BuildingCrossCompilers
> 
> "Nomenclature
> 
> TARGET is the architecture the compiler builds code for. HOST [...] is the 
> architecture the compiler runs on."
> 
> Mapping that on our buildchroots: buildchroot-host is where the compiler runs 
> natively, buildchroot-target is where it runs on the target architecture. Thus, 
> no need to flip anything.
> 

Citing from the wrong wiki, I got confused about this unfortunate nomenclature 
(because of the fuzzy term "host"):

"host means the architecture of produced executable objects, i.e. the 
architecture of the host system where these guest objects will run on (called 
target or sometimes build elsewhere) "
https://wiki.debian.org/CrossCompiling

Given this, we should definitely avoid reusing "host" when doing any renaming. 
It will only make things worse, specifically if some user-facing variable that 
we used to call HOST_SOMETHING now gets different semantic without changing its 
name. One option would be only renaming host->build and dropping "target" from 
the rest.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22 12:50                   ` Jan Kiszka
@ 2019-05-22 13:37                     ` Claudius Heine
  2019-05-22 19:10                       ` Baurzhan Ismagulov
  0 siblings, 1 reply; 23+ messages in thread
From: Claudius Heine @ 2019-05-22 13:37 UTC (permalink / raw)
  To: Jan Kiszka, Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine

On 22/05/2019 14.50, Jan Kiszka wrote:
> On 22.05.19 14:15, [ext] Jan Kiszka wrote:
>> On 22.05.19 13:39, Claudius Heine wrote:
>>> On 22/05/2019 13.35, Jan Kiszka wrote:
>>>> On 22.05.19 09:31, Claudius Heine wrote:
>>>>> Hi Jan,
>>>>>
>>>>> On 22/05/2019 09.02, Jan Kiszka wrote:
>>>>>> On 22.05.19 08:55, Maxim Yu. Osipov wrote:
>>>>>>> On 5/22/19 8:37 AM, Claudius Heine wrote:
>>>>>>>> Hi Jan,
>>>>>>>>
>>>>>>>> On 21/05/2019 18.56, Jan Kiszka wrote:
>>>>>>>>> On 15.05.19 12:11, [ext] claudius.heine.ext@siemens.com wrote:
>>>>>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>>>>>
>>>>>>>>>> isar-bootstrap-host only supports bootstrapping Debian root file
>>>>>>>>>> systems. Therefore deactivate any DISTRO_BOOTSTRAP_KEYS from 
>>>>>>>>>> other
>>>>>>>>>> distributions.
>>>>>>>>>
>>>>>>>>> Actually not totally true, as I just realized: What about 
>>>>>>>>> bootstrapping the buildchroot from a custom debian repo that 
>>>>>>>>> was differently signed (e.g. because it is a condensed version 
>>>>>>>>> of upstream)? Seems we do need HOST_DISTRO_BOOTSTRAP_KEYS, right?
>>>>>>>>
>>>>>>>> You are right, I haven't considered that case. Well, the next 
>>>>>>>> task on the todo list should probably be to refactor and 
>>>>>>>> streamline the isar-bootstrap, and especially the host bootstrap 
>>>>>>>> process and fix those kind of issues while doing that.
>>>>>>>>
>>>>>>>> There are a lot of possible customization options gained if the 
>>>>>>>> current giant bootstrap function would be split up, similar to 
>>>>>>>> how the rootfs system works in the pre-processing patchset.
>>>>>>>>
>>>>>>>> Maybe it makes sense to also start renaming "host" and "target" 
>>>>>>>> to be compatible with the gcc nomenclature [1] in that patchset. 
>>>>>>>> I am a bit reluctant to do so, because of the breakage involved. 
>>>>>>>> But the further we wait, to more stuff will break downstream.
>>>>>>>>
>>>>>>>> The plan would be to rename all occurrences of "host" to "build" 
>>>>>>>> and "target" to "host". That would lead to the following recipe 
>>>>>>>> changes:
>>>>>>>
>>>>>>>
>>>>>>>> "buildchroot-host" -> "buildchroot-build"
>>>>>>>> "buildchroot-target" -> "buildchroot-host"
>>>>>>>> "isar-bootstrap-host" -> "isar-bootstrap-build"
>>>>>>>> "isar-bootstrap-target" -> "isar-bootstrap-host"
>>>>>>>
>>>>>>>
>>>>>>>> I am on the fence of that change. Correctness vs. no-breakage
>>>>>>>>
>>>>>>>> Any comments about that?
>>>>>>>
>>>>>>> I would prefer to avoid such a renaming taking into account 
>>>>>>> needed efforts and possible confusion for current Isar users.
>>>>>>>
>>>>>>
>>>>>> Is that build/host scheme then also in line with Debian naming? 
>>>>>> Then there will be eventually no way around it anyway. But we 
>>>>>> really need to do this thoroughly, specifically /wrt to 
>>>>>> user-visible interfaces, so that it will be one cut only.
>>>>>
>>>>>  From dpkg-buildpackage(1):
>>>>>
>>>>> [...]
>>>>>         -a, --host-arch architecture
>>>>>                Specify  the  Debian  architecture  we  build  for 
>>>>> (long option since dpkg
>>>>>                1.17.17).  The architecture of the  machine  we  
>>>>> build on  is determined
>>>>>                automatically, and is also the default for the host 
>>>>> machine.
>>>>> [...]
>>>>>         --target-arch architecture
>>>>>                Specify  the  Debian architecture the binaries built 
>>>>> will build for (since
>>>>>                dpkg 1.17.17).  The default value is the host machine.
>>>>> [...]
>>>>>
>>>>
>>>> Then the current naming scheme is already accurate for Debian.
>>>
>>> No? That is not how I read that documentation. "Host" is what we 
>>> build for and 'Target' is what the binaries we build generate code 
>>> for (which makes only sense for compilers)
>>
>> https://wiki.debian.org/BuildingCrossCompilers
>>
>> "Nomenclature
>>
>> TARGET is the architecture the compiler builds code for. HOST [...] is 
>> the architecture the compiler runs on."
>>
>> Mapping that on our buildchroots: buildchroot-host is where the 
>> compiler runs natively, buildchroot-target is where it runs on the 
>> target architecture. Thus, no need to flip anything.
>>
> 
> Citing from the wrong wiki, I got confused about this unfortunate 
> nomenclature (because of the fuzzy term "host"):
> 
> "host means the architecture of produced executable objects, i.e. the 
> architecture of the host system where these guest objects will run on 
> (called target or sometimes build elsewhere) "
> https://wiki.debian.org/CrossCompiling
> 
> Given this, we should definitely avoid reusing "host" when doing any 
> renaming. It will only make things worse, specifically if some 
> user-facing variable that we used to call HOST_SOMETHING now gets 
> different semantic without changing its name. One option would be only 
> renaming host->build and dropping "target" from the rest.

With my patchsets we could probably even drop 'build', 'host' and 
'target' from the recipes all together and just go for the build host 
tupel. For instance:

"buildchroot-host-amd64-arm" -> "buildchroot-amd64-arm"
"buildchroot-target-arm" -> "buildchroot-arm"
"isar-bootstrap-host-amd64-arm" -> "isar-bootstrap-amd64-arm"
"isar-bootstrap-target-arm" -> "isar-bootstrap-arm"

That would only affect the PN variable in those recipes, the file names 
would probably be 'isar-bootstrap-build.bb'/'isar-bootstrap-native' 
(which is what yocto is using to avoid 'build') and 
'isar-bootstrap-host.bb'/'isar-bootstrap.bb'.

For the variables just renaming 'HOST_*' stuff to 'BUILD_*' stuff would 
be ok, but maybe a bit confusing, because 'build' might be a bit vague 
there. How about 'NATIVE_*'?

regards,
Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22 13:37                     ` Claudius Heine
@ 2019-05-22 19:10                       ` Baurzhan Ismagulov
  2019-05-23  9:22                         ` Claudius Heine
  0 siblings, 1 reply; 23+ messages in thread
From: Baurzhan Ismagulov @ 2019-05-22 19:10 UTC (permalink / raw)
  To: isar-users

Hi,

what is the use case for such a change?

The system works. The names are intuitive for the users. If anything is not
clear, we can define the terms in the docs. Binutils and gcc are more complex
systems that require (a very non-intuitive) triplet; we don't. Having "host"
and "target" in the directory names is also convenient for the user. Therefore,
I'm not in favor of this change.

With kind regards,
Baurzhan.


On Wed, May 22, 2019 at 03:37:37PM +0200, Claudius Heine wrote:
> With my patchsets we could probably even drop 'build', 'host' and 'target'
> from the recipes all together and just go for the build host tupel. For
> instance:
> 
> "buildchroot-host-amd64-arm" -> "buildchroot-amd64-arm"
> "buildchroot-target-arm" -> "buildchroot-arm"
> "isar-bootstrap-host-amd64-arm" -> "isar-bootstrap-amd64-arm"
> "isar-bootstrap-target-arm" -> "isar-bootstrap-arm"
> 
> That would only affect the PN variable in those recipes, the file names
> would probably be 'isar-bootstrap-build.bb'/'isar-bootstrap-native' (which
> is what yocto is using to avoid 'build') and
> 'isar-bootstrap-host.bb'/'isar-bootstrap.bb'.
> 
> For the variables just renaming 'HOST_*' stuff to 'BUILD_*' stuff would be
> ok, but maybe a bit confusing, because 'build' might be a bit vague there.
> How about 'NATIVE_*'?

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage
  2019-05-22 19:10                       ` Baurzhan Ismagulov
@ 2019-05-23  9:22                         ` Claudius Heine
  0 siblings, 0 replies; 23+ messages in thread
From: Claudius Heine @ 2019-05-23  9:22 UTC (permalink / raw)
  To: isar-users

Hi Baurzhan,

On 22/05/2019 21.10, Baurzhan Ismagulov wrote:
> Hi,
> 
> what is the use case for such a change?

We are discussing here about possible ways to proceed, and AFAIK have 
not yet decided on it.

> The system works. The names are intuitive for the users. If anything is not
> clear, we can define the terms in the docs. Binutils and gcc are more complex
> systems that require (a very non-intuitive) triplet; we don't. Having "host"
> and "target" in the directory names is also convenient for the user. Therefore,
> I'm not in favor of this change.

The way isar currently uses the words 'host' and 'target' is in conflict 
which how gcc, Debian or the Yocto Project/OpenEmbedded uses them. This 
is a fact.

I think we have several ways to handle this issue:
1. Explain in the documentation front and center that we are using 
different definitions than most other prominent tools and projects.
2. Change our code to fit those other projects
3. Make up new words that are not used by those projects, but are 
comparatively descriptive.
4. Ignore that issue

Personally ignoring this issue would be the worst option. Since I 
guess/hope that there will be more people using isar in the future than 
currently are, so lowering the bar of entry by streamlining the use of 
technical terms should be desired.

regards,
Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2019-05-23  9:22 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-15 10:11 [PATCH v6 0/9] Cleanup rootfs creation claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 1/9] isar-bootstrap-host: disable DISTRO_BOOTSTRAP_KEYS usage claudius.heine.ext
2019-05-21 16:56   ` Jan Kiszka
2019-05-22  6:37     ` Claudius Heine
2019-05-22  6:55       ` Maxim Yu. Osipov
2019-05-22  7:02         ` Jan Kiszka
2019-05-22  7:31           ` Claudius Heine
2019-05-22 11:35             ` Jan Kiszka
2019-05-22 11:39               ` Claudius Heine
2019-05-22 12:15                 ` Jan Kiszka
2019-05-22 12:50                   ` Jan Kiszka
2019-05-22 13:37                     ` Claudius Heine
2019-05-22 19:10                       ` Baurzhan Ismagulov
2019-05-23  9:22                         ` Claudius Heine
2019-05-15 10:11 ` [PATCH v6 2/9] isar-bootstrap-helper: move 'HOST_ARCH' and 'HOST_DISTRO' to base.bbclass claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 3/9] move 'HOST_DISTRO_APT_SOURCES' from bootstrap-helper to isar-bootstrap claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 4/9] buildchroot.bbclass: only cross build if HOST_ARCH != DISTRO_ARCH claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 5/9] isar-bootstrap/buildchroot/sdkchroot: refactor PF and WORKDIR claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 6/9] bitbake.conf: remove unneeded and differently used variables claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 7/9] image.bbclass: make IMAGE_ROOTFS overwritable claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 8/9] bitbake.conf: set default QEMU_ARCH variables claudius.heine.ext
2019-05-15 10:11 ` [PATCH v6 9/9] buildchroot/configscript: make creation of builder uid/gid idempotent claudius.heine.ext
2019-05-15 16:01 ` [PATCH v6 0/9] Cleanup rootfs creation Maxim Yu. Osipov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox