* [PATCH v2 01/12] isar-bootstrap: Add routine to determine host arch
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
@ 2018-06-28 8:27 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 02/12] isar-bootstrap: Move common part to include Alexander Smirnov
` (12 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:27 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
This method will be used to distinguish host and target bootstraps. The
standard python platform module is not robust here, because it provides
the architecture label from kernel. For example running platfotm.machine()
on Fedora kernel will return 'x86_64', what is invalid specificator for
Debian systems.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/isar-bootstrap-helper.bbclass | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 3c10fc7..9d77ebc 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -7,6 +7,16 @@
IMAGE_TRANSIENT_PACKAGES ??= ""
+python() {
+ import subprocess
+ host_arch = subprocess.Popen("dpkg --print-architecture",
+ shell=True,
+ env=os.environ,
+ stdout=subprocess.PIPE
+ ).stdout.read().decode('utf-8').strip()
+ d.setVar("HOST_ARCH", host_arch);
+}
+
def reverse_bb_array(d, varname):
array = d.getVar(varname, True)
if array is None:
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 02/12] isar-bootstrap: Move common part to include
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
2018-06-28 8:27 ` [PATCH v2 01/12] isar-bootstrap: Add routine to determine host arch Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 03/12] isar-bootstrap: Add host architecture support Alexander Smirnov
` (11 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Eventually there should be two bootstraps: host and target. This
patch derives the acrchitecture independent part of original isar-bootstrap
recipe and put it in header file.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/image.bbclass | 2 +-
.../isar-bootstrap/isar-bootstrap-target.bb | 44 ++++
meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 259 ---------------------
.../recipes-core/isar-bootstrap/isar-bootstrap.inc | 223 ++++++++++++++++++
meta/recipes-devtools/buildchroot/buildchroot.bb | 2 +-
5 files changed, 269 insertions(+), 261 deletions(-)
create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
delete mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 3bdcb2f..4738cb8 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -47,7 +47,7 @@ INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')[1]}"
inherit ${IMAGE_TYPE}
do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}"
-do_rootfs[depends] = "isar-apt:do_cache_config isar-bootstrap:do_deploy"
+do_rootfs[depends] = "isar-apt:do_cache_config isar-bootstrap-target:do_deploy"
do_rootfs() {
die "No root filesystem function defined, please implement in your recipe"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
new file mode 100644
index 0000000..bb37a68
--- /dev/null
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
@@ -0,0 +1,44 @@
+# Minimal target Debian root file system
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+Description = "Minimal target Debian root file system"
+
+include isar-bootstrap.inc
+
+do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_bootstrap[vardeps] += "DISTRO_APT_SOURCES"
+do_bootstrap[vardeps] += "DISTRO_APT_PREMIRRORS"
+do_bootstrap() {
+ if [ -e "${ROOTFSDIR}" ]; then
+ sudo umount -l "${ROOTFSDIR}/dev" || true
+ sudo umount -l "${ROOTFSDIR}/proc" || true
+ sudo rm -rf "${ROOTFSDIR}"
+ fi
+ E="${@bb.utils.export_proxies(d)}"
+ sudo -E "${DEBOOTSTRAP}" --verbose \
+ --variant=minbase \
+ --arch="${DISTRO_ARCH}" \
+ --include=locales \
+ ${@get_distro_components_argument(d)} \
+ ${DEBOOTSTRAP_KEYRING} \
+ "${@get_distro_suite(d)}" \
+ "${ROOTFSDIR}" \
+ "${@get_distro_source(d)}"
+}
+addtask bootstrap before do_build after do_generate_keyring
+
+do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
+do_deploy() {
+ ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}"
+}
+addtask deploy before do_build after do_apt_update
+
+CLEANFUNCS = "clean_deploy"
+clean_deploy() {
+ rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}"
+}
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
deleted file mode 100644
index 497a4f4..0000000
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
+++ /dev/null
@@ -1,259 +0,0 @@
-# Minimal debian root file system
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-Description = "Minimal debian root file system"
-
-LICENSE = "gpl-2.0"
-LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
-FILESPATH_prepend := "${THISDIR}/files:"
-SRC_URI = " \
- file://isar-apt.conf \
- file://isar-apt-fallback.conf \
- file://locale \
- file://chroot-setup.sh"
-PV = "1.0"
-
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
-DEBOOTSTRAP ?= "qemu-debootstrap"
-ROOTFSDIR = "${WORKDIR}/rootfs"
-APTPREFS = "${WORKDIR}/apt-preferences"
-APTSRCS = "${WORKDIR}/apt-sources"
-APTKEYFILES = ""
-APTKEYRING = "${WORKDIR}/apt-keyring.gpg"
-DEBOOTSTRAP_KEYRING = ""
-
-python () {
- from urllib.parse import urlparse
- distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False)
- if distro_apt_keys:
- d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
- for key in distro_apt_keys.split():
- url = urlparse(key)
- filename = os.path.basename(url.path)
- d.appendVar("SRC_URI", " %s" % key)
- d.appendVar("APTKEYFILES", " %s" % filename)
-}
-
-def aggregate_files(d, file_list, file_out):
- import shutil
-
- with open(file_out, "wb") as out_fd:
- for entry in file_list:
- entry_real = bb.parse.resolve_file(entry, d)
- with open(entry_real, "rb") as in_fd:
- shutil.copyfileobj(in_fd, out_fd, 1024*1024*10)
- out_fd.write("\n".encode())
-
-def parse_aptsources_list_line(source_list_line):
- import re
-
- s = source_list_line.strip()
-
- if s.startswith("#"):
- return None
-
- type, s = re.split("\s+", s, maxsplit=1)
- if type not in ["deb", "deb-src"]:
- return None
-
- options = ""
- options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s)
- if options_match:
- options = options_match.group(0).strip()
- s = s[options_match.end():]
-
- source, s = re.split("\s+", s, maxsplit=1)
-
- suite, s = re.split("\s+", s, maxsplit=1)
-
- components = " ".join(s.split())
-
- return [type, options, source, suite, components]
-
-def get_apt_source_mirror(d, aptsources_entry_list):
- import re
-
- premirrors = d.getVar('DISTRO_APT_PREMIRRORS', True) or ""
- mirror_list = [entry.split()
- for entry in premirrors.split('\\n')
- if any(entry)]
-
- for regex, replace in mirror_list:
- match = re.search(regex, aptsources_entry_list[2])
-
- if match:
- new_aptsources_entry_list = aptsources_entry_list.copy()
- new_aptsources_entry_list[2] = re.sub(regex, replace,
- aptsources_entry_list[2],
- count = 1)
- return new_aptsources_entry_list
-
- return aptsources_entry_list
-
-def aggregate_aptsources_list(d, file_list, file_out):
- import shutil
-
- with open(file_out, "wb") as out_fd:
- for entry in file_list:
- entry_real = bb.parse.resolve_file(entry, d)
- with open(entry_real, "r") as in_fd:
- for line in in_fd:
- parsed = parse_aptsources_list_line(line)
- if parsed:
- parsed = get_apt_source_mirror(d, parsed)
- out_fd.write(" ".join(parsed).encode())
- else:
- out_fd.write(line.encode())
- out_fd.write("\n".encode())
- out_fd.write("\n".encode())
-
-def get_distro_primary_source_entry(d):
- apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split()
- for entry in apt_sources_list:
- entry_real = bb.parse.resolve_file(entry, d)
- with open(entry_real, "r") as in_fd:
- for line in in_fd:
- parsed = parse_aptsources_list_line(line)
- if parsed:
- parsed = get_apt_source_mirror(d, parsed)
- if parsed[0] == "deb":
- return parsed[2:]
- return ["", "", ""]
-
-def get_distro_source(d):
- return get_distro_primary_source_entry(d)[0]
-
-def get_distro_suite(d):
- return get_distro_primary_source_entry(d)[1]
-
-def get_distro_components_argument(d):
- components = get_distro_primary_source_entry(d)[2]
- if components and components.strip():
- return "--components=%s" % ",".join(components.split())
- else:
- return ""
-
-do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_generate_keyring[dirs] = "${WORKDIR}"
-do_generate_keyring[vardeps] += "DISTRO_APT_KEYS"
-do_generate_keyring() {
- if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then
- for keyfile in ${@d.getVar("APTKEYFILES", True)}; do
- gpg --no-default-keyring --keyring "${APTKEYRING}" \
- --homedir "${WORKDIR}" --import "$keyfile"
- done
- fi
-}
-addtask generate_keyring before do_build after do_unpack
-
-do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_apt_config_prepare[dirs] = "${WORKDIR}"
-do_apt_config_prepare[vardeps] += "\
- APTPREFS \
- DISTRO_APT_PREFERENCES \
- DEBDISTRONAME \
- APTSRCS \
- DISTRO_APT_SOURCES \
- "
-python do_apt_config_prepare() {
- apt_preferences_out = d.getVar("APTPREFS", True)
- apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", True) or ""
- ).split()
- aggregate_files(d, apt_preferences_list, apt_preferences_out)
-
- apt_sources_out = d.getVar("APTSRCS", True)
- apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split()
-
- aggregate_aptsources_list(d, apt_sources_list, apt_sources_out)
-}
-addtask apt_config_prepare before do_build after do_unpack
-
-do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_bootstrap[vardeps] += "DISTRO_APT_SOURCES"
-do_bootstrap[vardeps] += "DISTRO_APT_PREMIRRORS"
-do_bootstrap() {
- if [ -e "${ROOTFSDIR}" ]; then
- sudo umount -l "${ROOTFSDIR}/dev" || true
- sudo umount -l "${ROOTFSDIR}/proc" || true
- sudo rm -rf "${ROOTFSDIR}"
- fi
- E="${@bb.utils.export_proxies(d)}"
- sudo -E "${DEBOOTSTRAP}" --verbose \
- --variant=minbase \
- --arch="${DISTRO_ARCH}" \
- --include=locales \
- ${@get_distro_components_argument(d)} \
- ${DEBOOTSTRAP_KEYRING} \
- "${@get_distro_suite(d)}" \
- "${ROOTFSDIR}" \
- "${@get_distro_source(d)}"
-}
-addtask bootstrap before do_build after do_generate_keyring
-
-do_set_locale() {
- sudo install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
-
- sudo sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen"
- sudo -E chroot "${ROOTFSDIR}" /usr/sbin/locale-gen
-}
-addtask set_locale after do_bootstrap
-
-do_setup_chroot() {
- sudo install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh"
- sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
-}
-addtask setup_chroot before do_build after do_bootstrap
-
-def get_host_release():
- import platform
- rel = platform.release()
- return rel
-
-do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_apt_config_install() {
- sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d"
- sudo install -v -m644 "${APTPREFS}" \
- "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap"
- sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d"
- sudo install -v -m644 "${APTSRCS}" \
- "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
- sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list"
- sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d"
- sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \
- "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf"
-
- if [ "${@get_distro_suite(d)}" = "stretch" ] && [ "${@get_host_release().split('.')[0]}" -lt "4" ]; then
- sudo install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \
- "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf"
- fi
-}
-addtask apt_config_install before do_build after do_bootstrap do_apt_config_prepare
-
-do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_apt_update() {
- sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev
- sudo mount -t proc none ${ROOTFSDIR}/proc
-
- E="${@bb.utils.export_proxies(d)}"
- export DEBIAN_FRONTEND=noninteractive
- sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
- sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
- -o Debug::pkgProblemResolver=yes
-}
-addtask apt_update before do_build after do_apt_config_install do_set_locale do_setup_chroot
-
-do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
-do_deploy() {
- ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}"
-}
-addtask deploy before do_build after do_apt_update
-
-CLEANFUNCS = "clean_deploy"
-clean_deploy() {
- rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}"
-}
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
new file mode 100644
index 0000000..263f8af
--- /dev/null
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -0,0 +1,223 @@
+# Minimal debian root file system
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+LICENSE = "gpl-2.0"
+LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
+FILESPATH_prepend := "${THISDIR}/files:"
+SRC_URI = " \
+ file://isar-apt.conf \
+ file://isar-apt-fallback.conf \
+ file://locale \
+ file://chroot-setup.sh"
+PV = "1.0"
+
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
+DEBOOTSTRAP ?= "qemu-debootstrap"
+ROOTFSDIR = "${WORKDIR}/rootfs"
+APTPREFS = "${WORKDIR}/apt-preferences"
+APTSRCS = "${WORKDIR}/apt-sources"
+APTKEYFILES = ""
+APTKEYRING = "${WORKDIR}/apt-keyring.gpg"
+DEBOOTSTRAP_KEYRING = ""
+
+python () {
+ from urllib.parse import urlparse
+ distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False)
+ if distro_apt_keys:
+ d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}")
+ for key in distro_apt_keys.split():
+ url = urlparse(key)
+ filename = os.path.basename(url.path)
+ d.appendVar("SRC_URI", " %s" % key)
+ d.appendVar("APTKEYFILES", " %s" % filename)
+}
+
+def aggregate_files(d, file_list, file_out):
+ import shutil
+
+ with open(file_out, "wb") as out_fd:
+ for entry in file_list:
+ entry_real = bb.parse.resolve_file(entry, d)
+ with open(entry_real, "rb") as in_fd:
+ shutil.copyfileobj(in_fd, out_fd, 1024*1024*10)
+ out_fd.write("\n".encode())
+
+def parse_aptsources_list_line(source_list_line):
+ import re
+
+ s = source_list_line.strip()
+
+ if s.startswith("#"):
+ return None
+
+ type, s = re.split("\s+", s, maxsplit=1)
+ if type not in ["deb", "deb-src"]:
+ return None
+
+ options = ""
+ options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s)
+ if options_match:
+ options = options_match.group(0).strip()
+ s = s[options_match.end():]
+
+ source, s = re.split("\s+", s, maxsplit=1)
+
+ suite, s = re.split("\s+", s, maxsplit=1)
+
+ components = " ".join(s.split())
+
+ return [type, options, source, suite, components]
+
+def get_apt_source_mirror(d, aptsources_entry_list):
+ import re
+
+ premirrors = d.getVar('DISTRO_APT_PREMIRRORS', True) or ""
+ mirror_list = [entry.split()
+ for entry in premirrors.split('\\n')
+ if any(entry)]
+
+ for regex, replace in mirror_list:
+ match = re.search(regex, aptsources_entry_list[2])
+
+ if match:
+ new_aptsources_entry_list = aptsources_entry_list.copy()
+ new_aptsources_entry_list[2] = re.sub(regex, replace,
+ aptsources_entry_list[2],
+ count = 1)
+ return new_aptsources_entry_list
+
+ return aptsources_entry_list
+
+def aggregate_aptsources_list(d, file_list, file_out):
+ import shutil
+
+ with open(file_out, "wb") as out_fd:
+ for entry in file_list:
+ entry_real = bb.parse.resolve_file(entry, d)
+ with open(entry_real, "r") as in_fd:
+ for line in in_fd:
+ parsed = parse_aptsources_list_line(line)
+ if parsed:
+ parsed = get_apt_source_mirror(d, parsed)
+ out_fd.write(" ".join(parsed).encode())
+ else:
+ out_fd.write(line.encode())
+ out_fd.write("\n".encode())
+ out_fd.write("\n".encode())
+
+def get_distro_primary_source_entry(d):
+ apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split()
+ for entry in apt_sources_list:
+ entry_real = bb.parse.resolve_file(entry, d)
+ with open(entry_real, "r") as in_fd:
+ for line in in_fd:
+ parsed = parse_aptsources_list_line(line)
+ if parsed:
+ parsed = get_apt_source_mirror(d, parsed)
+ if parsed[0] == "deb":
+ return parsed[2:]
+ return ["", "", ""]
+
+def get_distro_source(d):
+ return get_distro_primary_source_entry(d)[0]
+
+def get_distro_suite(d):
+ return get_distro_primary_source_entry(d)[1]
+
+def get_distro_components_argument(d):
+ components = get_distro_primary_source_entry(d)[2]
+ if components and components.strip():
+ return "--components=%s" % ",".join(components.split())
+ else:
+ return ""
+
+do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_generate_keyring[dirs] = "${WORKDIR}"
+do_generate_keyring[vardeps] += "DISTRO_APT_KEYS"
+do_generate_keyring() {
+ if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then
+ for keyfile in ${@d.getVar("APTKEYFILES", True)}; do
+ gpg --no-default-keyring --keyring "${APTKEYRING}" \
+ --homedir "${WORKDIR}" --import "$keyfile"
+ done
+ fi
+}
+addtask generate_keyring before do_build after do_unpack
+
+do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_apt_config_prepare[dirs] = "${WORKDIR}"
+do_apt_config_prepare[vardeps] += "\
+ APTPREFS \
+ DISTRO_APT_PREFERENCES \
+ DEBDISTRONAME \
+ APTSRCS \
+ DISTRO_APT_SOURCES \
+ "
+python do_apt_config_prepare() {
+ apt_preferences_out = d.getVar("APTPREFS", True)
+ apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", True) or ""
+ ).split()
+ aggregate_files(d, apt_preferences_list, apt_preferences_out)
+
+ apt_sources_out = d.getVar("APTSRCS", True)
+ apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split()
+
+ aggregate_aptsources_list(d, apt_sources_list, apt_sources_out)
+}
+addtask apt_config_prepare before do_build after do_unpack
+
+do_set_locale() {
+ sudo install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale"
+
+ sudo sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen"
+ sudo -E chroot "${ROOTFSDIR}" /usr/sbin/locale-gen
+}
+addtask set_locale after do_bootstrap
+
+do_setup_chroot() {
+ sudo install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh"
+ sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
+}
+addtask setup_chroot before do_build after do_bootstrap
+
+def get_host_release():
+ import platform
+ rel = platform.release()
+ return rel
+
+do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_apt_config_install() {
+ sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d"
+ sudo install -v -m644 "${APTPREFS}" \
+ "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap"
+ sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d"
+ sudo install -v -m644 "${APTSRCS}" \
+ "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
+ sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list"
+ sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d"
+ sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \
+ "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf"
+
+ if [ "${@get_distro_suite(d)}" = "stretch" ] && [ "${@get_host_release().split('.')[0]}" -lt "4" ]; then
+ sudo install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \
+ "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf"
+ fi
+}
+addtask apt_config_install before do_build after do_bootstrap do_apt_config_prepare
+
+do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_apt_update() {
+ sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev
+ sudo mount -t proc none ${ROOTFSDIR}/proc
+
+ E="${@bb.utils.export_proxies(d)}"
+ export DEBIAN_FRONTEND=noninteractive
+ sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
+ sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
+ -o Debug::pkgProblemResolver=yes
+}
+addtask apt_update before do_build after do_apt_config_install do_set_locale do_setup_chroot
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index 538c577..7db8206 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -66,7 +66,7 @@ do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
${BUILDCHROOT_DIR}/isar-apt \
${BUILDCHROOT_DIR}/downloads \
${BUILDCHROOT_DIR}/home/builder"
-do_build[depends] = "isar-apt:do_cache_config isar-bootstrap:do_deploy"
+do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-target:do_deploy"
do_build() {
setup_root_file_system "${BUILDCHROOT_DIR}" ${BUILDCHROOT_PREINSTALL}
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 03/12] isar-bootstrap: Add host architecture support
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
2018-06-28 8:27 ` [PATCH v2 01/12] isar-bootstrap: Add routine to determine host arch Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 02/12] isar-bootstrap: Move common part to include Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 04/12] isar-bootstrap-helper: Add parameter to set arch Alexander Smirnov
` (10 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Add recipe to generate bootstrap with the host architecture.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
.../isar-bootstrap/isar-bootstrap-host.bb | 44 ++++++++++++++++++++++
1 file changed, 44 insertions(+)
create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
new file mode 100644
index 0000000..c61c665
--- /dev/null
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
@@ -0,0 +1,44 @@
+# Minimal host Debian root file system
+#
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2018
+#
+# SPDX-License-Identifier: MIT
+
+Description = "Minimal host Debian root file system"
+
+include isar-bootstrap.inc
+inherit isar-bootstrap-helper
+
+do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_bootstrap[vardeps] += "DISTRO_APT_SOURCES"
+do_bootstrap[vardeps] += "DISTRO_APT_PREMIRRORS"
+do_bootstrap() {
+ if [ -e "${ROOTFSDIR}" ]; then
+ sudo umount -l "${ROOTFSDIR}/dev" || true
+ sudo umount -l "${ROOTFSDIR}/proc" || true
+ sudo rm -rf "${ROOTFSDIR}"
+ fi
+ E="${@bb.utils.export_proxies(d)}"
+ sudo -E "${DEBOOTSTRAP}" --verbose \
+ --variant=minbase \
+ --include=locales \
+ ${@get_distro_components_argument(d)} \
+ ${DEBOOTSTRAP_KEYRING} \
+ "${@get_distro_suite(d)}" \
+ "${ROOTFSDIR}" \
+ "${@get_distro_source(d)}"
+}
+addtask bootstrap before do_build after do_generate_keyring
+
+do_deploy[stamp-extra-info] = "${DISTRO}-${HOST_ARCH}"
+do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
+do_deploy() {
+ ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${HOST_ARCH}"
+}
+addtask deploy before do_build after do_apt_update
+
+CLEANFUNCS = "clean_deploy"
+clean_deploy() {
+ rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${HOST_ARCH}"
+}
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 04/12] isar-bootstrap-helper: Add parameter to set arch
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (2 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 03/12] isar-bootstrap: Add host architecture support Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 05/12] buildchroot-cross: Initial implementation Alexander Smirnov
` (9 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Initially this helper could produce only rootfs with hardcoded target
architecture. This patch adds parameter, so now it's possible to also
produce rootfs with the host architecture.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/isar-bootstrap-helper.bbclass | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 9d77ebc..d4f8d9a 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -27,10 +27,12 @@ def reverse_bb_array(d, varname):
setup_root_file_system() {
CLEAN=""
FSTAB=""
+ ROOTFS_ARCH="${DISTRO_ARCH}"
while true; do
case "$1" in
--clean) CLEAN=1 ;;
--fstab) FSTAB=$2; shift ;;
+ --host-arch) ROOTFS_ARCH=${HOST_ARCH} ;;
-*) bbfatal "$0: invalid option specified: $1" ;;
*) break ;;
esac
@@ -43,7 +45,7 @@ setup_root_file_system() {
CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
sudo cp -Trpfx \
- "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}/" \
+ "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-$ROOTFS_ARCH/" \
"$ROOTFSDIR"
[ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee "$ROOTFSDIR/etc/fstab"
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 05/12] buildchroot-cross: Initial implementation
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (3 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 04/12] isar-bootstrap-helper: Add parameter to set arch Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-29 17:36 ` Jan Kiszka
2018-06-28 8:28 ` [PATCH v2 06/12] classes/dpkg*: Relocate dependency from buildchroot Alexander Smirnov
` (8 subsequent siblings)
13 siblings, 1 reply; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Add initial generation of buildchroot for cross-compilation.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/isar-bootstrap-helper.bbclass | 5 ++
meta/conf/isar-bitbake.conf | 1 +
.../buildchroot/buildchroot-cross.bb | 57 ++++++++++++++++++++++
3 files changed, 63 insertions(+)
create mode 100644 meta/recipes-devtools/buildchroot/buildchroot-cross.bb
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index d4f8d9a..a3ef19d 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -70,6 +70,11 @@ setup_root_file_system() {
-o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
-o Dir::Etc::sourceparts="-" \
-o APT::Get::List-Cleanup="0"
+ # Add multiarch for cross-target
+ if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
+ sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH}
+ sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
+ fi
sudo -E chroot "$ROOTFSDIR" \
/usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
${IMAGE_TRANSIENT_PACKAGES}
diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index ead7798..1412508 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -22,6 +22,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
DL_DIR = "${TOPDIR}/downloads"
SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
+BUILDCHROOT_CROSS_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-cross/rootfs"
CACHE = "${TMPDIR}/cache"
OVERRIDES_append = ":${DISTRO}:${DISTRO_ARCH}"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-cross.bb b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
new file mode 100644
index 0000000..bc0bc66
--- /dev/null
+++ b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
@@ -0,0 +1,57 @@
+# Root filesystem for packages cross-building
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2016 ilbers GmbH
+
+DESCRIPTION = "Isar development cross-filesystem"
+
+LICENSE = "gpl-2.0"
+LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
+
+FILESPATH_prepend := "${THISDIR}/files:"
+SRC_URI = "file://configscript.sh \
+ file://build.sh"
+PV = "1.0"
+
+inherit isar-bootstrap-helper
+
+BUILDCHROOT_CROSS_PREINSTALL ?= "make \
+ debhelper \
+ autotools-dev \
+ dpkg \
+ locales \
+ docbook-to-man \
+ apt \
+ automake \
+ devscripts \
+ equivs \
+ libc6:${DISTRO_ARCH}"
+
+# Please note: this works for Stretch distro only. According to the wiki page:
+# https://wiki.debian.org/CrossToolchains
+# Jessie doesn't contain toolchain. It should be fetched from the external
+# repository:
+# http://emdebian.org/tools/debian/
+BUILDCHROOT_CROSS_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf \
+ crossbuild-essential-armhf"
+
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
+
+do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+do_build[root_cleandirs] = "${BUILDCHROOT_CROSS_DIR} \
+ ${BUILDCHROOT_CROSS_DIR}/isar-apt \
+ ${BUILDCHROOT_CROSS_DIR}/downloads \
+ ${BUILDCHROOT_CROSS_DIR}/home/builder"
+do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_deploy"
+
+do_build() {
+ setup_root_file_system --host-arch "${BUILDCHROOT_CROSS_DIR}" ${BUILDCHROOT_CROSS_PREINSTALL}
+
+ # Install package builder script
+ sudo chmod -R a+rw "${BUILDCHROOT_CROSS_DIR}/home/builder"
+ sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_CROSS_DIR}/build.sh
+
+ # Configure root filesystem
+ sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_CROSS_DIR}
+ sudo chroot ${BUILDCHROOT_CROSS_DIR} /configscript.sh
+}
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 05/12] buildchroot-cross: Initial implementation
2018-06-28 8:28 ` [PATCH v2 05/12] buildchroot-cross: Initial implementation Alexander Smirnov
@ 2018-06-29 17:36 ` Jan Kiszka
2018-07-04 19:56 ` Alexander Smirnov
0 siblings, 1 reply; 21+ messages in thread
From: Jan Kiszka @ 2018-06-29 17:36 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
[-- Attachment #1.1: Type: text/plain, Size: 4916 bytes --]
On 2018-06-28 10:28, Alexander Smirnov wrote:
> Add initial generation of buildchroot for cross-compilation.
>
> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> ---
> meta/classes/isar-bootstrap-helper.bbclass | 5 ++
> meta/conf/isar-bitbake.conf | 1 +
> .../buildchroot/buildchroot-cross.bb | 57 ++++++++++++++++++++++
> 3 files changed, 63 insertions(+)
> create mode 100644 meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>
> diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
> index d4f8d9a..a3ef19d 100644
> --- a/meta/classes/isar-bootstrap-helper.bbclass
> +++ b/meta/classes/isar-bootstrap-helper.bbclass
> @@ -70,6 +70,11 @@ setup_root_file_system() {
> -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
> -o Dir::Etc::sourceparts="-" \
> -o APT::Get::List-Cleanup="0"
> + # Add multiarch for cross-target
> + if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
> + sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH}
> + sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
> + fi
> sudo -E chroot "$ROOTFSDIR" \
> /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
> ${IMAGE_TRANSIENT_PACKAGES}
> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
> index ead7798..1412508 100644
> --- a/meta/conf/isar-bitbake.conf
> +++ b/meta/conf/isar-bitbake.conf
> @@ -22,6 +22,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
> DL_DIR = "${TOPDIR}/downloads"
> SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
> BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
> +BUILDCHROOT_CROSS_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-cross/rootfs"
> CACHE = "${TMPDIR}/cache"
>
> OVERRIDES_append = ":${DISTRO}:${DISTRO_ARCH}"
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-cross.bb b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
> new file mode 100644
> index 0000000..bc0bc66
> --- /dev/null
> +++ b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
> @@ -0,0 +1,57 @@
> +# Root filesystem for packages cross-building
> +#
> +# This software is a part of ISAR.
> +# Copyright (C) 2015-2016 ilbers GmbH
> +
> +DESCRIPTION = "Isar development cross-filesystem"
> +
> +LICENSE = "gpl-2.0"
> +LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
> +
> +FILESPATH_prepend := "${THISDIR}/files:"
> +SRC_URI = "file://configscript.sh \
> + file://build.sh"
> +PV = "1.0"
> +
> +inherit isar-bootstrap-helper
> +
> +BUILDCHROOT_CROSS_PREINSTALL ?= "make \
> + debhelper \
> + autotools-dev \
> + dpkg \
> + locales \
> + docbook-to-man \
> + apt \
> + automake \
> + devscripts \
> + equivs \
> + libc6:${DISTRO_ARCH}"
> +
> +# Please note: this works for Stretch distro only. According to the wiki page:
> +# https://wiki.debian.org/CrossToolchains
> +# Jessie doesn't contain toolchain. It should be fetched from the external
> +# repository:
> +# http://emdebian.org/tools/debian/
> +BUILDCHROOT_CROSS_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf \
> + crossbuild-essential-armhf"
> +
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
> +
> +do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +do_build[root_cleandirs] = "${BUILDCHROOT_CROSS_DIR} \
> + ${BUILDCHROOT_CROSS_DIR}/isar-apt \
> + ${BUILDCHROOT_CROSS_DIR}/downloads \
> + ${BUILDCHROOT_CROSS_DIR}/home/builder"
> +do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_deploy"
> +
> +do_build() {
> + setup_root_file_system --host-arch "${BUILDCHROOT_CROSS_DIR}" ${BUILDCHROOT_CROSS_PREINSTALL}
> +
> + # Install package builder script
> + sudo chmod -R a+rw "${BUILDCHROOT_CROSS_DIR}/home/builder"
> + sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_CROSS_DIR}/build.sh
> +
> + # Configure root filesystem
> + sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_CROSS_DIR}
> + sudo chroot ${BUILDCHROOT_CROSS_DIR} /configscript.sh
> +}
>
This is almost identical to buildchroot:do_build - except for one
differently named variable, the first line, and missing fix
39e0c488d8b2. Please avoid those duplications.
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 05/12] buildchroot-cross: Initial implementation
2018-06-29 17:36 ` Jan Kiszka
@ 2018-07-04 19:56 ` Alexander Smirnov
2018-07-04 20:05 ` Jan Kiszka
0 siblings, 1 reply; 21+ messages in thread
From: Alexander Smirnov @ 2018-07-04 19:56 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 06/29/2018 08:36 PM, Jan Kiszka wrote:
> On 2018-06-28 10:28, Alexander Smirnov wrote:
>> Add initial generation of buildchroot for cross-compilation.
>>
>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>> ---
>> meta/classes/isar-bootstrap-helper.bbclass | 5 ++
>> meta/conf/isar-bitbake.conf | 1 +
>> .../buildchroot/buildchroot-cross.bb | 57 ++++++++++++++++++++++
>> 3 files changed, 63 insertions(+)
>> create mode 100644 meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>>
>> diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
>> index d4f8d9a..a3ef19d 100644
>> --- a/meta/classes/isar-bootstrap-helper.bbclass
>> +++ b/meta/classes/isar-bootstrap-helper.bbclass
>> @@ -70,6 +70,11 @@ setup_root_file_system() {
>> -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
>> -o Dir::Etc::sourceparts="-" \
>> -o APT::Get::List-Cleanup="0"
>> + # Add multiarch for cross-target
>> + if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
>> + sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH}
>> + sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
>> + fi
>> sudo -E chroot "$ROOTFSDIR" \
>> /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
>> ${IMAGE_TRANSIENT_PACKAGES}
>> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
>> index ead7798..1412508 100644
>> --- a/meta/conf/isar-bitbake.conf
>> +++ b/meta/conf/isar-bitbake.conf
>> @@ -22,6 +22,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
>> DL_DIR = "${TOPDIR}/downloads"
>> SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
>> BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
>> +BUILDCHROOT_CROSS_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-cross/rootfs"
>> CACHE = "${TMPDIR}/cache"
>>
>> OVERRIDES_append = ":${DISTRO}:${DISTRO_ARCH}"
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-cross.bb b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>> new file mode 100644
>> index 0000000..bc0bc66
>> --- /dev/null
>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>> @@ -0,0 +1,57 @@
>> +# Root filesystem for packages cross-building
>> +#
>> +# This software is a part of ISAR.
>> +# Copyright (C) 2015-2016 ilbers GmbH
>> +
>> +DESCRIPTION = "Isar development cross-filesystem"
>> +
>> +LICENSE = "gpl-2.0"
>> +LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
>> +
>> +FILESPATH_prepend := "${THISDIR}/files:"
>> +SRC_URI = "file://configscript.sh \
>> + file://build.sh"
>> +PV = "1.0"
>> +
>> +inherit isar-bootstrap-helper
>> +
>> +BUILDCHROOT_CROSS_PREINSTALL ?= "make \
>> + debhelper \
>> + autotools-dev \
>> + dpkg \
>> + locales \
>> + docbook-to-man \
>> + apt \
>> + automake \
>> + devscripts \
>> + equivs \
>> + libc6:${DISTRO_ARCH}"
>> +
>> +# Please note: this works for Stretch distro only. According to the wiki page:
>> +# https://wiki.debian.org/CrossToolchains
>> +# Jessie doesn't contain toolchain. It should be fetched from the external
>> +# repository:
>> +# http://emdebian.org/tools/debian/
>> +BUILDCHROOT_CROSS_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf \
>> + crossbuild-essential-armhf"
>> +
>> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>> +
>> +do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>> +do_build[root_cleandirs] = "${BUILDCHROOT_CROSS_DIR} \
>> + ${BUILDCHROOT_CROSS_DIR}/isar-apt \
>> + ${BUILDCHROOT_CROSS_DIR}/downloads \
>> + ${BUILDCHROOT_CROSS_DIR}/home/builder"
>> +do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_deploy"
>> +
>> +do_build() {
>> + setup_root_file_system --host-arch "${BUILDCHROOT_CROSS_DIR}" ${BUILDCHROOT_CROSS_PREINSTALL}
>> +
>> + # Install package builder script
>> + sudo chmod -R a+rw "${BUILDCHROOT_CROSS_DIR}/home/builder"
>> + sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_CROSS_DIR}/build.sh
>> +
>> + # Configure root filesystem
>> + sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_CROSS_DIR}
>> + sudo chroot ${BUILDCHROOT_CROSS_DIR} /configscript.sh
>> +}
>>
>
> This is almost identical to buildchroot:do_build - except for one
> differently named variable, the first line, and missing fix
> 39e0c488d8b2. Please avoid those duplications.
>
Sorry, didn't get this. It's absolutely different logical units:
- crossbuildchroot has its own lifecycle including stamps, so it could
not be shared with the original buildchroot (only via common include)
- crossbuildchroot and buildchroot have different default packages to
install and they are incompatible due to possibly different
architectures (cross has always host one, bchroot has target one).
Could you please clarify what you mean here?
Alex
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 05/12] buildchroot-cross: Initial implementation
2018-07-04 19:56 ` Alexander Smirnov
@ 2018-07-04 20:05 ` Jan Kiszka
0 siblings, 0 replies; 21+ messages in thread
From: Jan Kiszka @ 2018-07-04 20:05 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
[-- Attachment #1.1: Type: text/plain, Size: 6703 bytes --]
On 2018-07-04 21:56, Alexander Smirnov wrote:
>
> On 06/29/2018 08:36 PM, Jan Kiszka wrote:
>> On 2018-06-28 10:28, Alexander Smirnov wrote:
>>> Add initial generation of buildchroot for cross-compilation.
>>>
>>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>>> ---
>>> meta/classes/isar-bootstrap-helper.bbclass | 5 ++
>>> meta/conf/isar-bitbake.conf | 1 +
>>> .../buildchroot/buildchroot-cross.bb | 57
>>> ++++++++++++++++++++++
>>> 3 files changed, 63 insertions(+)
>>> create mode 100644
>>> meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>>>
>>> diff --git a/meta/classes/isar-bootstrap-helper.bbclass
>>> b/meta/classes/isar-bootstrap-helper.bbclass
>>> index d4f8d9a..a3ef19d 100644
>>> --- a/meta/classes/isar-bootstrap-helper.bbclass
>>> +++ b/meta/classes/isar-bootstrap-helper.bbclass
>>> @@ -70,6 +70,11 @@ setup_root_file_system() {
>>> -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
>>> -o Dir::Etc::sourceparts="-" \
>>> -o APT::Get::List-Cleanup="0"
>>> + # Add multiarch for cross-target
>>> + if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
>>> + sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture
>>> ${DISTRO_ARCH}
>>> + sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
>>> + fi
>>> sudo -E chroot "$ROOTFSDIR" \
>>> /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
>>> ${IMAGE_TRANSIENT_PACKAGES}
>>> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
>>> index ead7798..1412508 100644
>>> --- a/meta/conf/isar-bitbake.conf
>>> +++ b/meta/conf/isar-bitbake.conf
>>> @@ -22,6 +22,7 @@ WORKDIR =
>>> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
>>> DL_DIR = "${TOPDIR}/downloads"
>>> SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
>>> BUILDCHROOT_DIR =
>>> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
>>> +BUILDCHROOT_CROSS_DIR =
>>> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-cross/rootfs"
>>> CACHE = "${TMPDIR}/cache"
>>> OVERRIDES_append = ":${DISTRO}:${DISTRO_ARCH}"
>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>>> b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>>> new file mode 100644
>>> index 0000000..bc0bc66
>>> --- /dev/null
>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>>> @@ -0,0 +1,57 @@
>>> +# Root filesystem for packages cross-building
>>> +#
>>> +# This software is a part of ISAR.
>>> +# Copyright (C) 2015-2016 ilbers GmbH
>>> +
>>> +DESCRIPTION = "Isar development cross-filesystem"
>>> +
>>> +LICENSE = "gpl-2.0"
>>> +LIC_FILES_CHKSUM =
>>> "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
>>>
>>> +
>>> +FILESPATH_prepend := "${THISDIR}/files:"
>>> +SRC_URI = "file://configscript.sh \
>>> + file://build.sh"
>>> +PV = "1.0"
>>> +
>>> +inherit isar-bootstrap-helper
>>> +
>>> +BUILDCHROOT_CROSS_PREINSTALL ?= "make \
>>> + debhelper \
>>> + autotools-dev \
>>> + dpkg \
>>> + locales \
>>> + docbook-to-man \
>>> + apt \
>>> + automake \
>>> + devscripts \
>>> + equivs \
>>> + libc6:${DISTRO_ARCH}"
>>> +
>>> +# Please note: this works for Stretch distro only. According to the
>>> wiki page:
>>> +# https://wiki.debian.org/CrossToolchains
>>> +# Jessie doesn't contain toolchain. It should be fetched from the
>>> external
>>> +# repository:
>>> +# http://emdebian.org/tools/debian/
>>> +BUILDCHROOT_CROSS_PREINSTALL_append_armhf +=
>>> "binutils-arm-linux-gnueabihf \
>>> +
>>> crossbuild-essential-armhf"
>>> +
>>> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>> +
>>> +do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>> +do_build[root_cleandirs] = "${BUILDCHROOT_CROSS_DIR} \
>>> + ${BUILDCHROOT_CROSS_DIR}/isar-apt \
>>> + ${BUILDCHROOT_CROSS_DIR}/downloads \
>>> + ${BUILDCHROOT_CROSS_DIR}/home/builder"
>>> +do_build[depends] = "isar-apt:do_cache_config
>>> isar-bootstrap-host:do_deploy"
>>> +
>>> +do_build() {
>>> + setup_root_file_system --host-arch "${BUILDCHROOT_CROSS_DIR}"
>>> ${BUILDCHROOT_CROSS_PREINSTALL}
>>> +
>>> + # Install package builder script
>>> + sudo chmod -R a+rw "${BUILDCHROOT_CROSS_DIR}/home/builder"
>>> + sudo install -m 755 ${WORKDIR}/build.sh
>>> ${BUILDCHROOT_CROSS_DIR}/build.sh
>>> +
>>> + # Configure root filesystem
>>> + sudo install -m 755 ${WORKDIR}/configscript.sh
>>> ${BUILDCHROOT_CROSS_DIR}
>>> + sudo chroot ${BUILDCHROOT_CROSS_DIR} /configscript.sh
>>> +}
>>>
>>
>> This is almost identical to buildchroot:do_build - except for one
>> differently named variable, the first line, and missing fix
>> 39e0c488d8b2. Please avoid those duplications.
>>
>
> Sorry, didn't get this. It's absolutely different logical units:
No, the code is identical. Step back, look at it again.
> - crossbuildchroot has its own lifecycle including stamps, so it could
> not be shared with the original buildchroot (only via common include)
> - crossbuildchroot and buildchroot have different default packages to
> install and they are incompatible due to possibly different
> architectures (cross has always host one, bchroot has target one).
>
> Could you please clarify what you mean here?
I'm not suggesting to have only a single recipe. I'm asking to pull out
identical code into a shared function, class, whatever so that we can
maintain identical logic in one place (and fix bugs only once).
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 06/12] classes/dpkg*: Relocate dependency from buildchroot
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (4 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 05/12] buildchroot-cross: Initial implementation Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 07/12] classes/dpkg*: Drop hardcoded buildchroot blobs Alexander Smirnov
` (7 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Move dependency from buildchroot to the classes that actually require
buildchroot. This will make possible to re-use dpkg-base class for
buildchroot-cross.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/dpkg-base.bbclass | 3 ---
meta/classes/dpkg-raw.bbclass | 3 +++
meta/classes/dpkg.bbclass | 3 +++
meta/recipes-kernel/linux/linux-custom.inc | 2 ++
4 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 75b45a7..24fe2ca 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -14,9 +14,6 @@ do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
inherit patch
addtask patch after do_adjust_git before do_build
-# Add dependency from buildchroot creation
-do_build[depends] = "buildchroot:do_build"
-
# Add dependency between Isar recipes
DEPENDS ?= ""
do_build[deptask] = "do_deploy_deb"
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index 28233ac..5c7be6b 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -6,6 +6,9 @@ inherit dpkg-base
DEBIAN_DEPENDS ?= ""
MAINTAINER ?= "FIXME Unknown maintainer"
+# Add dependency from buildchroot creation
+do_build[depends] = "buildchroot:do_build"
+
D = "${WORKDIR}/image/"
# Populate folder that will be picked up as package
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index c8d4ac5..c483fe2 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -3,6 +3,9 @@
inherit dpkg-base
+# Add dependency from buildchroot creation
+do_build[depends] = "buildchroot:do_build"
+
# Build package from sources using build script
dpkg_runbuild() {
E="${@ bb.utils.export_proxies(d)}"
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 1176b25..8eccba8 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -9,6 +9,8 @@ FILESPATH =. "${LAYERDIR_core}/recipes-kernel/linux/files:"
DESCRIPTION ?= "Custom kernel"
+do_build[depends] = "buildchroot:do_build"
+
KERNEL_FLAVOR ?= "${@ d.getVar('PN', True).partition('linux-')[2]}"
KERNEL_DEFCONFIG ?= "defconfig"
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 07/12] classes/dpkg*: Drop hardcoded buildchroot blobs
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (5 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 06/12] classes/dpkg*: Relocate dependency from buildchroot Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 08/12] build.sh: Add target parameter Alexander Smirnov
` (6 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
This patch makes possible to parametrize the chroot will be actually
used by dpkg-base class.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/dpkg-base.bbclass | 14 +++++++-------
meta/classes/dpkg-raw.bbclass | 4 +++-
meta/classes/dpkg.bbclass | 4 +++-
3 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 24fe2ca..0668e25 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -31,7 +31,7 @@ def get_package_srcdir(d):
PP = "/home/builder/${PN}"
PPS ?= "${@get_package_srcdir(d)}"
-BUILDROOT = "${BUILDCHROOT_DIR}/${PP}"
+BUILDROOT = "${ROOTFS_DIR}/${PP}"
do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
# default to "emtpy" implementation
@@ -39,7 +39,7 @@ dpkg_runbuild() {
die "This should never be called, overwrite it in your derived class"
}
-MOUNT_LOCKFILE = "${BUILDCHROOT_DIR}/mount.lock"
+MOUNT_LOCKFILE = "${ROOTFS_DIR}/mount.lock"
# Wrap the function dpkg_runbuild with the bind mount for buildroot
do_build() {
@@ -47,11 +47,11 @@ do_build() {
sudo mount --bind ${WORKDIR} ${BUILDROOT}
sudo flock ${MOUNT_LOCKFILE} -c ' \
- if ! grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts; then \
- mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt; \
- mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads; \
- mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev; \
- mount -t proc none ${BUILDCHROOT_DIR}/proc; \
+ if ! grep -q ${ROOTFS_DIR}/isar-apt /proc/mounts; then \
+ mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${ROOTFS_DIR}/isar-apt; \
+ mount --bind ${DL_DIR} ${ROOTFS_DIR}/downloads; \
+ mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFS_DIR}/dev; \
+ mount -t proc none ${ROOTFS_DIR}/proc; \
fi'
dpkg_runbuild
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index 5c7be6b..1b661a7 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -9,6 +9,8 @@ MAINTAINER ?= "FIXME Unknown maintainer"
# Add dependency from buildchroot creation
do_build[depends] = "buildchroot:do_build"
+ROOTFS_DIR = "${BUILDCHROOT_DIR}"
+
D = "${WORKDIR}/image/"
# Populate folder that will be picked up as package
@@ -59,5 +61,5 @@ addtask deb_package_conffiles after do_deb_package_prepare before do_build
dpkg_runbuild() {
sudo chown -R root:root ${D}/DEBIAN/
- sudo chroot ${BUILDCHROOT_DIR} dpkg-deb --build ${PP}/image ${PP}
+ sudo chroot ${ROOTFS_DIR} dpkg-deb --build ${PP}/image ${PP}
}
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index c483fe2..195174a 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -6,8 +6,10 @@ inherit dpkg-base
# Add dependency from buildchroot creation
do_build[depends] = "buildchroot:do_build"
+ROOTFS_DIR = "${BUILDCHROOT_DIR}"
+
# Build package from sources using build script
dpkg_runbuild() {
E="${@ bb.utils.export_proxies(d)}"
- sudo -E chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${PPS}
+ sudo -E chroot ${ROOTFS_DIR} /build.sh ${PP}/${PPS}
}
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 08/12] build.sh: Add target parameter
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (6 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 07/12] classes/dpkg*: Drop hardcoded buildchroot blobs Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 09/12] classes/dpkg-cross: Initial implementation Alexander Smirnov
` (5 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Pass target architecture to build as a parameter for the script.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/dpkg.bbclass | 2 +-
meta/recipes-devtools/buildchroot/files/build.sh | 17 +++++++++++++++--
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 195174a..87ffbce 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -11,5 +11,5 @@ ROOTFS_DIR = "${BUILDCHROOT_DIR}"
# Build package from sources using build script
dpkg_runbuild() {
E="${@ bb.utils.export_proxies(d)}"
- sudo -E chroot ${ROOTFS_DIR} /build.sh ${PP}/${PPS}
+ sudo -E chroot ${ROOTFS_DIR} /build.sh ${PP}/${PPS} ${DISTRO_ARCH}
}
diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh
index 73c0889..d127dae 100644
--- a/meta/recipes-devtools/buildchroot/files/build.sh
+++ b/meta/recipes-devtools/buildchroot/files/build.sh
@@ -6,6 +6,19 @@
set -e
+# Create human-readable names
+target_arch=$2
+
+# Notes:
+# mk-build-deps for jessie and jtretch has different parameter name to specify
+# host architecture.
+debian_version=$(cat /etc/debian_version | cut -c 1)
+if [ $(($debian_version)) -ge 9 ]; then
+ set_arch="--host-arch $target_arch"
+else
+ set_arch="-a target_arch"
+fi
+
# Go to build directory
cd $1
@@ -36,7 +49,7 @@ install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y
-o APT::Get::List-Cleanup="0"
# Install all build deps
- mk-build-deps -t "${install_cmd}" -i -r debian/control
+ mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control
) 42>/dpkg.lock
# If autotools files have been created, update their timestamp to
@@ -48,4 +61,4 @@ for i in configure aclocal.m4 Makefile.am Makefile.in; do
done
# Build the package
-dpkg-buildpackage
+dpkg-buildpackage -a$target_arch
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 09/12] classes/dpkg-cross: Initial implementation
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (7 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 08/12] build.sh: Add target parameter Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 10/12] dpkg: Resolve dep from buildchroot Alexander Smirnov
` (4 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Add basic class for cross-compilation.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/dpkg-cross.bbclass | 15 +++++++++++++++
1 file changed, 15 insertions(+)
create mode 100644 meta/classes/dpkg-cross.bbclass
diff --git a/meta/classes/dpkg-cross.bbclass b/meta/classes/dpkg-cross.bbclass
new file mode 100644
index 0000000..7e5a1fd
--- /dev/null
+++ b/meta/classes/dpkg-cross.bbclass
@@ -0,0 +1,15 @@
+# This software is a part of ISAR.
+# Copyright (C) 2015-2018 ilbers GmbH
+
+inherit dpkg-base
+
+# Add dependency from buildchroot creation
+do_build[depends] = "buildchroot-cross:do_build"
+
+ROOTFS_DIR="${BUILDCHROOT_CROSS_DIR}"
+
+# Build package from sources using build script
+dpkg_runbuild() {
+ E="${@ bb.utils.export_proxies(d)}"
+ sudo -E chroot ${ROOTFS_DIR} /build.sh ${PP}/${PPS} ${DISTRO_ARCH}
+}
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 10/12] dpkg: Resolve dep from buildchroot
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (8 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 09/12] classes/dpkg-cross: Initial implementation Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 11/12] linux: Add cross-compilation support Alexander Smirnov
` (3 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Automatically resolve the dependency from native or cross buildchroot.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/classes/dpkg-base.bbclass | 16 ++++++++++++++++
meta/classes/dpkg-cross.bbclass | 7 ++-----
meta/classes/dpkg-raw.bbclass | 5 -----
meta/classes/dpkg.bbclass | 7 ++-----
4 files changed, 20 insertions(+), 15 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 0668e25..ad440ad 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -1,6 +1,22 @@
# This software is a part of ISAR.
# Copyright (C) 2017 Siemens AG
+ISAR_CROSS_COMPILE ?= "0"
+
+# Add dependency from the correct buildchroot: cross or native
+python __anonymous() {
+ mode = d.getVar('ISAR_CROSS_COMPILE', True)
+ if mode == "0":
+ dep = "buildchroot:do_build"
+ rootfs = d.getVar('BUILDCHROOT_DIR', True)
+ else:
+ dep = "buildchroot-cross:do_build"
+ rootfs = d.getVar('BUILDCHROOT_CROSS_DIR', True)
+
+ d.setVarFlag('do_build', 'depends', dep)
+ d.setVar('ROOTFS_DIR', rootfs)
+}
+
do_adjust_git() {
if [ -f ${S}/.git/objects/info/alternates ]; then
sed -i ${S}/.git/objects/info/alternates \
diff --git a/meta/classes/dpkg-cross.bbclass b/meta/classes/dpkg-cross.bbclass
index 7e5a1fd..f097dec 100644
--- a/meta/classes/dpkg-cross.bbclass
+++ b/meta/classes/dpkg-cross.bbclass
@@ -1,12 +1,9 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2018 ilbers GmbH
-inherit dpkg-base
-
-# Add dependency from buildchroot creation
-do_build[depends] = "buildchroot-cross:do_build"
+ISAR_CROSS_COMPILE := "1"
-ROOTFS_DIR="${BUILDCHROOT_CROSS_DIR}"
+inherit dpkg-base
# Build package from sources using build script
dpkg_runbuild() {
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index 1b661a7..a6865e7 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -6,11 +6,6 @@ inherit dpkg-base
DEBIAN_DEPENDS ?= ""
MAINTAINER ?= "FIXME Unknown maintainer"
-# Add dependency from buildchroot creation
-do_build[depends] = "buildchroot:do_build"
-
-ROOTFS_DIR = "${BUILDCHROOT_DIR}"
-
D = "${WORKDIR}/image/"
# Populate folder that will be picked up as package
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 87ffbce..5420289 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -1,12 +1,9 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2016 ilbers GmbH
-inherit dpkg-base
-
-# Add dependency from buildchroot creation
-do_build[depends] = "buildchroot:do_build"
+ISAR_CROSS_COMPILE := "0"
-ROOTFS_DIR = "${BUILDCHROOT_DIR}"
+inherit dpkg-base
# Build package from sources using build script
dpkg_runbuild() {
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 11/12] linux: Add cross-compilation support
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (9 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 10/12] dpkg: Resolve dep from buildchroot Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-28 8:28 ` [PATCH v2 12/12] local.conf: Add cross-compilation option Alexander Smirnov
` (2 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Add possibility to cross-compile ARM target.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta/recipes-kernel/linux/files/build-kernel.sh | 16 ++++++++++++++++
meta/recipes-kernel/linux/linux-custom.inc | 4 ++--
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 1ec094e..ecfbf83 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -9,6 +9,22 @@
set -e
+host_arch=$(dpkg --print-architecture)
+target_arch=$2
+
+if [ "$host_arch" != "$target_arch" ]; then
+ case $target_arch in
+ armhf)
+ export ARCH=arm
+ export CROSS_COMPILE="arm-linux-gnueabihf-"
+ ;;
+ *)
+ echo "error: unsupported architecture ($target_arch)"
+ exit 1
+ ;;
+ esac
+fi
+
REPACK_DIR="$1/../repack"
REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image"
REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers"
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 8eccba8..8f33880 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -32,7 +32,7 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1, gcc"
dpkg_runbuild() {
# Install package builder script
- sudo install -m 755 ${WORKDIR}/build-kernel.sh ${BUILDCHROOT_DIR}
+ sudo install -m 755 ${WORKDIR}/build-kernel.sh ${ROOTFS_DIR}
sudo cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config
@@ -45,5 +45,5 @@ dpkg_runbuild() {
export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}"
export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}"
- sudo -E chroot ${BUILDCHROOT_DIR} /build-kernel.sh ${PP}/${PPS}
+ sudo -E chroot ${ROOTFS_DIR} /build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH}
}
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH v2 12/12] local.conf: Add cross-compilation option
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (10 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 11/12] linux: Add cross-compilation support Alexander Smirnov
@ 2018-06-28 8:28 ` Alexander Smirnov
2018-06-29 13:31 ` [PATCH v2 00/12] Cross-compilation Henning Schild
2018-06-29 18:28 ` Jan Kiszka
13 siblings, 0 replies; 21+ messages in thread
From: Alexander Smirnov @ 2018-06-28 8:28 UTC (permalink / raw)
To: isar-users; +Cc: Alexander Smirnov
Add option to enable cross-compilation by default.
Please also note, that the only stretch-armhf target supported for
cross-compilation only.
Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
meta-isar/conf/local.conf.sample | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 16ef488..75b6f01 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -156,3 +156,8 @@ CONF_VERSION = "1"
#
# The default list of extra packages to be installed.
IMAGE_INSTALL = "example-hello example-raw example-module"
+
+#
+# Enable cross-compilation support
+# NOTE: this works only for stretch-armhf target for now
+ISAR_CROSS_COMPILE ?= "0"
--
2.1.4
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] Cross-compilation
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (11 preceding siblings ...)
2018-06-28 8:28 ` [PATCH v2 12/12] local.conf: Add cross-compilation option Alexander Smirnov
@ 2018-06-29 13:31 ` Henning Schild
2018-06-29 18:28 ` Jan Kiszka
13 siblings, 0 replies; 21+ messages in thread
From: Henning Schild @ 2018-06-29 13:31 UTC (permalink / raw)
To: Alexander Smirnov; +Cc: isar-users
Am Thu, 28 Jun 2018 10:27:58 +0200
schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> Hi all,
>
> [NOTE]
> Cover letter is too long, so I don't expect that you read it all.
> So I'd like to start with this message to avoid possible
> question. :-) This series has intersection with SDK one, so it should
> be rebased after SDK is applied to next. Thanks!
> [/NOTE]
[NOTE]
Stopped review here, waiting for v3 since SDK has been applied.
[/NOTE]
Henning
>
> here is the second version of cross-compilation support.
> This version adds support for cross-compiling stretch-armhf target.
>
> What's new here:
> - Build dependencies are resolved automatically, so isar-image-base
> could be built using cross-compilation only (libhello, example-hello,
> example-raw)
> - Linux kernel could be also cross-compiled now for arm
>
> Test scenarios:
> 1. Build isar-image-base:
> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
> - Edit recipes: libhello and example-hello and replace
> "inherit dpkg" by "inherit dpkg-cross"
> - Build using "bitbake multiconfig:qemuarm-stretch:isar-image-base"
> - With cross-compilation the image is generated twice faster
>
> 2. Build arm kernel:
> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
> - There is no default arm config, so I derived the one from linux-cip
> tree: "zx_config" and add it to recipe
> - Build it "bitbake multiconfig:qemuarm-stretch:linux-cip"
> - Aftewards I have the following packages:
> * linux-image-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
> * linux-image-4.4.112-cip18-dbg_4.4.112-cip18-1_armhf.deb
> * linux-headers-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
> - The compilation tooks me about 2-3 minutes. Much faster than in
> QEMU
>
> Issue:
> 1. No support for jessie. The main issue here, that jessie doesn't
> have armhf compiler in default apt. Official wiki asks you to use
> embian apt to get it
> 2. mk-build-deps has two params: --host-arch, --cross-arch. But they
> seem to work incorrectly. If set both of them in deps list I got
> unexisting packages. Now I use --host-arch only and it works good. I
> want to check this in buster and then probably ask Debian community
> reagrding this 3. i386 architecture for now is not considered as a
> target for cross compilation
> 4. Currently libhello and example-hello applications have hardcoded
> native compilation (inherit dpkg). This should be reworked to depend
> on ISAR_CROSS_COMPILE variable
>
> Huh, no more ideas, let's start with this.
>
> Alex
>
> Alexander Smirnov (12):
> isar-bootstrap: Add routine to determine host arch
> isar-bootstrap: Move common part to include
> isar-bootstrap: Add host architecture support
> isar-bootstrap-helper: Add parameter to set arch
> buildchroot-cross: Initial implementation
> classes/dpkg*: Relocate dependency from buildchroot
> classes/dpkg*: Drop hardcoded buildchroot blobs
> build.sh: Add target parameter
> classes/dpkg-cross: Initial implementation
> dpkg: Resolve dep from buildchroot
> linux: Add cross-compilation support
> local.conf: Add cross-compilation option
>
> meta-isar/conf/local.conf.sample | 5 +
> meta/classes/dpkg-base.bbclass | 33 ++-
> meta/classes/dpkg-cross.bbclass | 12 +
> meta/classes/dpkg-raw.bbclass | 2 +-
> meta/classes/dpkg.bbclass | 4 +-
> meta/classes/image.bbclass | 2 +-
> meta/classes/isar-bootstrap-helper.bbclass | 19 +-
> meta/conf/isar-bitbake.conf | 1 +
> .../isar-bootstrap/isar-bootstrap-host.bb | 44 ++++
> .../isar-bootstrap/isar-bootstrap-target.bb | 44 ++++
> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 259
> --------------------- .../recipes-core/isar-bootstrap/isar-bootstrap.inc
> | 223
> ++++++++++++++++++ .../buildchroot/buildchroot-cross.bb
> | 57 +++++ meta/recipes-devtools/buildchroot/buildchroot.bb | 2
> +- meta/recipes-devtools/buildchroot/files/build.sh | 17 +-
> meta/recipes-kernel/linux/files/build-kernel.sh | 16 ++
> meta/recipes-kernel/linux/linux-custom.inc | 6 +- 17 files
> changed, 468 insertions(+), 278 deletions(-) create mode 100644
> meta/classes/dpkg-cross.bbclass create mode 100644
> meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb create mode
> 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
> delete mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> create mode 100644
> meta/recipes-core/isar-bootstrap/isar-bootstrap.inc create mode
> 100644 meta/recipes-devtools/buildchroot/buildchroot-cross.bb
>
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] Cross-compilation
2018-06-28 8:27 [PATCH v2 00/12] Cross-compilation Alexander Smirnov
` (12 preceding siblings ...)
2018-06-29 13:31 ` [PATCH v2 00/12] Cross-compilation Henning Schild
@ 2018-06-29 18:28 ` Jan Kiszka
2018-06-30 9:05 ` Jan Kiszka
2018-07-04 19:40 ` Alexander Smirnov
13 siblings, 2 replies; 21+ messages in thread
From: Jan Kiszka @ 2018-06-29 18:28 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
[-- Attachment #1.1: Type: text/plain, Size: 5432 bytes --]
On 2018-06-28 10:27, Alexander Smirnov wrote:
> Hi all,
>
> [NOTE]
> Cover letter is too long, so I don't expect that you read it all. So I'd
> like to start with this message to avoid possible question. :-) This series
> has intersection with SDK one, so it should be rebased after SDK is applied
> to next. Thanks!
> [/NOTE]
>
> here is the second version of cross-compilation support.
> This version adds support for cross-compiling stretch-armhf target.
>
> What's new here:
> - Build dependencies are resolved automatically, so isar-image-base could be
> built using cross-compilation only (libhello, example-hello, example-raw)
> - Linux kernel could be also cross-compiled now for arm
>
> Test scenarios:
> 1. Build isar-image-base:
> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
> - Edit recipes: libhello and example-hello and replace
> "inherit dpkg" by "inherit dpkg-cross"
> - Build using "bitbake multiconfig:qemuarm-stretch:isar-image-base"
> - With cross-compilation the image is generated twice faster
>
> 2. Build arm kernel:
> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
> - There is no default arm config, so I derived the one from linux-cip
> tree: "zx_config" and add it to recipe
> - Build it "bitbake multiconfig:qemuarm-stretch:linux-cip"
> - Aftewards I have the following packages:
> * linux-image-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
> * linux-image-4.4.112-cip18-dbg_4.4.112-cip18-1_armhf.deb
> * linux-headers-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
> - The compilation tooks me about 2-3 minutes. Much faster than in QEMU
>
> Issue:
> 1. No support for jessie. The main issue here, that jessie doesn't have
> armhf compiler in default apt. Official wiki asks you to use embian
> apt to get it
> 2. mk-build-deps has two params: --host-arch, --cross-arch. But they seem
> to work incorrectly. If set both of them in deps list I got unexisting
> packages. Now I use --host-arch only and it works good. I want to check
> this in buster and then probably ask Debian community reagrding this
> 3. i386 architecture for now is not considered as a target for cross
> compilation
> 4. Currently libhello and example-hello applications have hardcoded native
> compilation (inherit dpkg). This should be reworked to depend on
> ISAR_CROSS_COMPILE variable
>
> Huh, no more ideas, let's start with this.
I've started to play with this series (on top of next before the SDK
series got merged).
Setup:
- https://github.com/siemens/jailhouse-images
- KAS_TARGET=multiconfig:orangepi-zero-jailhouse:xradio ./build-images.sh
Findings:
- Kernel build works
- Module build does not yet. So I've hacked this in:
---8<---
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
index 1ee634c..e4af37e 100644
--- a/meta/recipes-kernel/linux-module/files/debian/control
+++ b/meta/recipes-kernel/linux-module/files/debian/control
@@ -2,7 +2,7 @@ Source: @PN@
Section: kernel
Priority: optional
Standards-Version: 3.9.6
-Build-Depends: linux-headers-@KERNEL_NAME@
+Build-Depends: linux-headers-@KERNEL_NAME@:amd64
Maintainer: ISAR project <isar-users@googlegroups.com>
Package: @PN@
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
index 3075f44..86776c3 100644
--- a/meta/recipes-kernel/linux-module/module.inc
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -15,7 +15,7 @@ SRC_URI += "file://debian/"
AUTOLOAD ?= "0"
-inherit dpkg
+inherit dpkg-cross
dpkg_runbuild_prepend() {
cp -r ${WORKDIR}/debian ${S}/
---8<---
An interesting pattern is that I had to add the host architecture to
the cross-built kernel headers package, or it would not have been
found. This will be needed again below.
Still, modules don't built, we also need to feed in ARCH=<kernel-
target-arch> to the build. Didn't hack that up so far.
- This, well, interesting package I was able to "cross"-compile as
well:
https://github.com/siemens/jailhouse-images/blob/master/recipes-core/non-root-initramfs/non-root-initramfs_2018.05.bb
It required the following change (beside "inherit dpkg-cross"):
---8<---
diff --git a/recipes-core/non-root-initramfs/files/debian/control b/recipes-core/non-root-initramfs/files/debian/control
index b57ee83..f5eeef1 100644
--- a/recipes-core/non-root-initramfs/files/debian/control
+++ b/recipes-core/non-root-initramfs/files/debian/control
@@ -2,7 +2,7 @@ Source: non-root-initramfs
Section: misc
Priority: optional
Standards-Version: 3.9.6
-Build-Depends: build-essential, wget, cpio, unzip, rsync, python, bc
+Build-Depends: build-essential:amd64, wget, cpio, unzip, rsync, python:amd64, bc
Maintainer: Jan Kiszka <jan.kiszka@siemens.com>
Package: non-root-initramfs
---8<---
Again, dependency installation failed without explicit host arch
specification for the two packages, but this time they come from
upstream. Any idea why? Obviously, this hack breaks native
compilation now.
That said, I'm excited to see this nice progress. I suspect we will
quickly sort the issues above out as well, and then more recipes - and
the whole jailhouse-images - will be cross-buildable.
Thanks,
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] Cross-compilation
2018-06-29 18:28 ` Jan Kiszka
@ 2018-06-30 9:05 ` Jan Kiszka
2018-07-04 19:40 ` Alexander Smirnov
1 sibling, 0 replies; 21+ messages in thread
From: Jan Kiszka @ 2018-06-30 9:05 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
[-- Attachment #1.1: Type: text/plain, Size: 6060 bytes --]
On 2018-06-29 20:28, Jan Kiszka wrote:
> On 2018-06-28 10:27, Alexander Smirnov wrote:
>> Hi all,
>>
>> [NOTE]
>> Cover letter is too long, so I don't expect that you read it all. So I'd
>> like to start with this message to avoid possible question. :-) This series
>> has intersection with SDK one, so it should be rebased after SDK is applied
>> to next. Thanks!
>> [/NOTE]
>>
>> here is the second version of cross-compilation support.
>> This version adds support for cross-compiling stretch-armhf target.
>>
>> What's new here:
>> - Build dependencies are resolved automatically, so isar-image-base could be
>> built using cross-compilation only (libhello, example-hello, example-raw)
>> - Linux kernel could be also cross-compiled now for arm
>>
>> Test scenarios:
>> 1. Build isar-image-base:
>> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
>> - Edit recipes: libhello and example-hello and replace
>> "inherit dpkg" by "inherit dpkg-cross"
>> - Build using "bitbake multiconfig:qemuarm-stretch:isar-image-base"
>> - With cross-compilation the image is generated twice faster
>>
>> 2. Build arm kernel:
>> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
>> - There is no default arm config, so I derived the one from linux-cip
>> tree: "zx_config" and add it to recipe
>> - Build it "bitbake multiconfig:qemuarm-stretch:linux-cip"
>> - Aftewards I have the following packages:
>> * linux-image-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
>> * linux-image-4.4.112-cip18-dbg_4.4.112-cip18-1_armhf.deb
>> * linux-headers-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
>> - The compilation tooks me about 2-3 minutes. Much faster than in QEMU
>>
>> Issue:
>> 1. No support for jessie. The main issue here, that jessie doesn't have
>> armhf compiler in default apt. Official wiki asks you to use embian
>> apt to get it
>> 2. mk-build-deps has two params: --host-arch, --cross-arch. But they seem
>> to work incorrectly. If set both of them in deps list I got unexisting
>> packages. Now I use --host-arch only and it works good. I want to check
>> this in buster and then probably ask Debian community reagrding this
>> 3. i386 architecture for now is not considered as a target for cross
>> compilation
>> 4. Currently libhello and example-hello applications have hardcoded native
>> compilation (inherit dpkg). This should be reworked to depend on
>> ISAR_CROSS_COMPILE variable
>>
>> Huh, no more ideas, let's start with this.
>
> I've started to play with this series (on top of next before the SDK
> series got merged).
>
> Setup:
> - https://github.com/siemens/jailhouse-images
> - KAS_TARGET=multiconfig:orangepi-zero-jailhouse:xradio ./build-images.sh
>
> Findings:
> - Kernel build works
> - Module build does not yet. So I've hacked this in:
>
> ---8<---
> diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
> index 1ee634c..e4af37e 100644
> --- a/meta/recipes-kernel/linux-module/files/debian/control
> +++ b/meta/recipes-kernel/linux-module/files/debian/control
> @@ -2,7 +2,7 @@ Source: @PN@
> Section: kernel
> Priority: optional
> Standards-Version: 3.9.6
> -Build-Depends: linux-headers-@KERNEL_NAME@
> +Build-Depends: linux-headers-@KERNEL_NAME@:amd64
> Maintainer: ISAR project <isar-users@googlegroups.com>
>
> Package: @PN@
> diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
> index 3075f44..86776c3 100644
> --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -15,7 +15,7 @@ SRC_URI += "file://debian/"
>
> AUTOLOAD ?= "0"
>
> -inherit dpkg
> +inherit dpkg-cross
>
> dpkg_runbuild_prepend() {
> cp -r ${WORKDIR}/debian ${S}/
> ---8<---
>
> An interesting pattern is that I had to add the host architecture to
> the cross-built kernel headers package, or it would not have been
> found. This will be needed again below.
>
> Still, modules don't built, we also need to feed in ARCH=<kernel-
> target-arch> to the build. Didn't hack that up so far.
>
> - This, well, interesting package I was able to "cross"-compile as
> well:
>
> https://github.com/siemens/jailhouse-images/blob/master/recipes-core/non-root-initramfs/non-root-initramfs_2018.05.bb
>
> It required the following change (beside "inherit dpkg-cross"):
>
> ---8<---
> diff --git a/recipes-core/non-root-initramfs/files/debian/control b/recipes-core/non-root-initramfs/files/debian/control
> index b57ee83..f5eeef1 100644
> --- a/recipes-core/non-root-initramfs/files/debian/control
> +++ b/recipes-core/non-root-initramfs/files/debian/control
> @@ -2,7 +2,7 @@ Source: non-root-initramfs
> Section: misc
> Priority: optional
> Standards-Version: 3.9.6
> -Build-Depends: build-essential, wget, cpio, unzip, rsync, python, bc
> +Build-Depends: build-essential:amd64, wget, cpio, unzip, rsync, python:amd64, bc
> Maintainer: Jan Kiszka <jan.kiszka@siemens.com>
>
> Package: non-root-initramfs
> ---8<---
>
> Again, dependency installation failed without explicit host arch
> specification for the two packages, but this time they come from
> upstream. Any idea why? Obviously, this hack breaks native
> compilation now.
>
> That said, I'm excited to see this nice progress. I suspect we will
> quickly sort the issues above out as well, and then more recipes - and
> the whole jailhouse-images - will be cross-buildable.
>
And another finding:
ISAR_CROSS_COMPILE has no impact on recipes when the inherit dpkg-cross.
This seems to be because of the hard assignment in dpgk-cross.bbclass.
But the logic should be:
inherit dpkg -> not cross-compatible, only build natively
inherit dpkg-cross -> can cross-compile, ISAR_CROSS_COMPILE decides
Should be fixed for v3.
Thanks
Jan
> Thanks,
> Jan
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] Cross-compilation
2018-06-29 18:28 ` Jan Kiszka
2018-06-30 9:05 ` Jan Kiszka
@ 2018-07-04 19:40 ` Alexander Smirnov
2018-07-04 20:03 ` Jan Kiszka
1 sibling, 1 reply; 21+ messages in thread
From: Alexander Smirnov @ 2018-07-04 19:40 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 06/29/2018 09:28 PM, Jan Kiszka wrote:
> On 2018-06-28 10:27, Alexander Smirnov wrote:
>> Hi all,
>>
>> [NOTE]
>> Cover letter is too long, so I don't expect that you read it all. So I'd
>> like to start with this message to avoid possible question. :-) This series
>> has intersection with SDK one, so it should be rebased after SDK is applied
>> to next. Thanks!
>> [/NOTE]
>>
>> here is the second version of cross-compilation support.
>> This version adds support for cross-compiling stretch-armhf target.
>>
>> What's new here:
>> - Build dependencies are resolved automatically, so isar-image-base could be
>> built using cross-compilation only (libhello, example-hello, example-raw)
>> - Linux kernel could be also cross-compiled now for arm
>>
>> Test scenarios:
>> 1. Build isar-image-base:
>> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
>> - Edit recipes: libhello and example-hello and replace
>> "inherit dpkg" by "inherit dpkg-cross"
>> - Build using "bitbake multiconfig:qemuarm-stretch:isar-image-base"
>> - With cross-compilation the image is generated twice faster
>>
>> 2. Build arm kernel:
>> - Set variable in local.conf ISAR_CROSS_COMPILE to "1"
>> - There is no default arm config, so I derived the one from linux-cip
>> tree: "zx_config" and add it to recipe
>> - Build it "bitbake multiconfig:qemuarm-stretch:linux-cip"
>> - Aftewards I have the following packages:
>> * linux-image-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
>> * linux-image-4.4.112-cip18-dbg_4.4.112-cip18-1_armhf.deb
>> * linux-headers-4.4.112-cip18_4.4.112-cip18-1_armhf.deb
>> - The compilation tooks me about 2-3 minutes. Much faster than in QEMU
>>
>> Issue:
>> 1. No support for jessie. The main issue here, that jessie doesn't have
>> armhf compiler in default apt. Official wiki asks you to use embian
>> apt to get it
>> 2. mk-build-deps has two params: --host-arch, --cross-arch. But they seem
>> to work incorrectly. If set both of them in deps list I got unexisting
>> packages. Now I use --host-arch only and it works good. I want to check
>> this in buster and then probably ask Debian community reagrding this
>> 3. i386 architecture for now is not considered as a target for cross
>> compilation
>> 4. Currently libhello and example-hello applications have hardcoded native
>> compilation (inherit dpkg). This should be reworked to depend on
>> ISAR_CROSS_COMPILE variable
>>
>> Huh, no more ideas, let's start with this.
>
> I've started to play with this series (on top of next before the SDK
> series got merged).
>
> Setup:
> - https://github.com/siemens/jailhouse-images
> - KAS_TARGET=multiconfig:orangepi-zero-jailhouse:xradio ./build-images.sh
>
> Findings:
> - Kernel build works
> - Module build does not yet. So I've hacked this in:
Yeah, unfortunately I didn't have time to take a look on module
cross-compilation support. And forgot to mention this in cover letter,
sorry :-(
>
> ---8<---
> diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
> index 1ee634c..e4af37e 100644
> --- a/meta/recipes-kernel/linux-module/files/debian/control
> +++ b/meta/recipes-kernel/linux-module/files/debian/control
> @@ -2,7 +2,7 @@ Source: @PN@
> Section: kernel
> Priority: optional
> Standards-Version: 3.9.6
> -Build-Depends: linux-headers-@KERNEL_NAME@
> +Build-Depends: linux-headers-@KERNEL_NAME@:amd64
> Maintainer: ISAR project <isar-users@googlegroups.com>
>
> Package: @PN@
> diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
> index 3075f44..86776c3 100644
> --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -15,7 +15,7 @@ SRC_URI += "file://debian/"
>
> AUTOLOAD ?= "0"
>
> -inherit dpkg
> +inherit dpkg-cross
>
> dpkg_runbuild_prepend() {
> cp -r ${WORKDIR}/debian ${S}/
> ---8<---
>
> An interesting pattern is that I had to add the host architecture to
> the cross-built kernel headers package, or it would not have been
> found. This will be needed again below.
>
> Still, modules don't built, we also need to feed in ARCH=<kernel-
> target-arch> to the build. Didn't hack that up so far.
>
> - This, well, interesting package I was able to "cross"-compile as
> well:
>
> https://github.com/siemens/jailhouse-images/blob/master/recipes-core/non-root-initramfs/non-root-initramfs_2018.05.bb
>
> It required the following change (beside "inherit dpkg-cross"):
>
> ---8<---
> diff --git a/recipes-core/non-root-initramfs/files/debian/control b/recipes-core/non-root-initramfs/files/debian/control
> index b57ee83..f5eeef1 100644
> --- a/recipes-core/non-root-initramfs/files/debian/control
> +++ b/recipes-core/non-root-initramfs/files/debian/control
> @@ -2,7 +2,7 @@ Source: non-root-initramfs
> Section: misc
> Priority: optional
> Standards-Version: 3.9.6
> -Build-Depends: build-essential, wget, cpio, unzip, rsync, python, bc
> +Build-Depends: build-essential:amd64, wget, cpio, unzip, rsync, python:amd64, bc
Could please clarify why you added build-essential package to deps? This
could be problem for cross-compilation, because we always need
build-essential of *host* architecture, but for example for ARM the
package with different name should be installed:
'crossbuild-essential-armhf', i.e. build-essential:armhf is invalid (it
will pull lots of other host utils with armhf arch, what will totally
break cross-buildchroot :-(.
To be honest, I can't remember if I have ever seen build-essentials in
debian/control. I've quickly checked Debian policy:
https://www.debian.org/doc/debian-policy/#relationships-between-source-and-binary-packages-build-depends-build-depends-indep-build-depends-arch-build-conflicts-build-conflicts-indep-build-conflicts-arch
so build-essential is optional, so I'd suggest to drop it.
> Maintainer: Jan Kiszka <jan.kiszka@siemens.com>
>
> Package: non-root-initramfs
> ---8<---
>
> Again, dependency installation failed without explicit host arch
> specification for the two packages, but this time they come from
> upstream. Any idea why? Obviously, this hack breaks native
> compilation now.
>
> That said, I'm excited to see this nice progress. I suspect we will
> quickly sort the issues above out as well, and then more recipes - and
> the whole jailhouse-images - will be cross-buildable.
Thank you for the detailed feedback, this is very helpful for me!
Alex
>
> Thanks,
> Jan
>
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] Cross-compilation
2018-07-04 19:40 ` Alexander Smirnov
@ 2018-07-04 20:03 ` Jan Kiszka
0 siblings, 0 replies; 21+ messages in thread
From: Jan Kiszka @ 2018-07-04 20:03 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
[-- Attachment #1.1: Type: text/plain, Size: 1274 bytes --]
On 2018-07-04 21:40, Alexander Smirnov wrote:
> On 06/29/2018 09:28 PM, Jan Kiszka wrote:
>> -Build-Depends: build-essential, wget, cpio, unzip, rsync, python, bc
>> +Build-Depends: build-essential:amd64, wget, cpio, unzip, rsync,
>> python:amd64, bc
>
> Could please clarify why you added build-essential package to deps? This
> could be problem for cross-compilation, because we always need
> build-essential of *host* architecture, but for example for ARM the
> package with different name should be installed:
> 'crossbuild-essential-armhf', i.e. build-essential:armhf is invalid (it
> will pull lots of other host utils with armhf arch, what will totally
> break cross-buildchroot :-(.
>
> To be honest, I can't remember if I have ever seen build-essentials in
> debian/control. I've quickly checked Debian policy:
>
> https://www.debian.org/doc/debian-policy/#relationships-between-source-and-binary-packages-build-depends-build-depends-indep-build-depends-arch-build-conflicts-build-conflicts-indep-build-conflicts-arch
>
>
> so build-essential is optional, so I'd suggest to drop it.
I can try, no problem.
Maybe - hopefully - it will resolve the problem for the python
dependency as well because that one can't be removed.
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread