* [PATCH v4 0/5] Debootstrap integration
@ 2018-03-07 16:44 claudius.heine.ext
2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext
` (5 more replies)
0 siblings, 6 replies; 27+ messages in thread
From: claudius.heine.ext @ 2018-03-07 16:44 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Hi,
sorry, just saw a small error, some changes to isar-bootstrap got into
the buildchroot patch.
Cheers,
Claudius
Changes from v3:
- A small modification to isar-bootstrap.bb snuck into the buildchroot
patch.
Changes from v2:
- rebased to current next
- added missing newlines at the end of file
- use cleandirs with root permissions
this could be improved in some later patch series, so that this
feature can be shared.
- moved main source url of packages to apt source list files of the
distributions:
- Implemented some basic parsing of source.lists files in
isar-bootstrap.
- Made 'DISTRO_SUITE', 'DISTRO_APT_SOURCE' and 'DISTRO_COMPONENTS'
variables obsolete.
- implemented repo key fetching to support rasbian repositories
- now the clean task removes the symlink of isar-bootstrap deploy task
Changes from v1:
- rebased to current next
- added unmounting of /dev and /proc at the end of do_rootfs in isar-image-base
This was necessary for image file creation, otherwise it tried to
package stuff from the host.
- added 'isar-bootstrap:do_deploy' dependency to do_rootfs task in
image.bbclass.
- Changed 'RFSDIR' variable name to 'ROOTFSDIR' in
isar-bootstrap-helper.bbclass to be consistent with the variable
name in isar-bootstrap.bb
- Moved 'isar-apt' apt-preference settings from isar-bootstrap.bb to
isar-bootstrap-helper.bbclass
- Removed '--no-install-recommends' parameters in favor of adding the
apt configuration in 'isar-bootstrap.bb'
- unmount and remove possible existing bootstraped directories before
bootstrapping in do_bootstrap task of 'isar-bootstrap.bb'
- changed from /dev ro bind mounts to mounting devtmpfs to /dev to be
consistent with the other places where /dev is mounted.
Claudius Heine (5):
implement isar-bootstrap using debootstrap
meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally
meta/buildchroot: switch to using isar-bootstrap
meta-isar/isar-image-base: switch to using isar-bootstrap
meta-isar/multiconfig: remove multistrap references
meta-isar/conf/distro/debian-jessie.conf | 9 +-
meta-isar/conf/distro/debian-jessie.list | 3 +
meta-isar/conf/distro/debian-stretch.conf | 7 +-
meta-isar/conf/distro/debian-stretch.list | 3 +
meta-isar/conf/distro/debian-wheezy.conf | 9 +-
meta-isar/conf/distro/debian-wheezy.list | 3 +
meta-isar/conf/distro/raspbian-jessie.conf | 12 +-
meta-isar/conf/distro/raspbian-jessie.list | 1 +
meta-isar/conf/local.conf.sample | 4 +-
meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +-
meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +-
meta-isar/conf/multiconfig/rpi-jessie.conf | 2 +-
.../recipes-core/images/files/download_dev-random | 24 ---
.../recipes-core/images/files/multistrap.conf.in | 38 ----
meta-isar/recipes-core/images/files/setup.sh | 77 --------
meta-isar/recipes-core/images/isar-image-base.bb | 64 +++----
meta/classes/image.bbclass | 2 +-
meta/classes/isar-bootstrap-helper.bbclass | 39 ++++
.../isar-bootstrap/files/isar-apt.conf | 2 +
meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 +++++++++++++++++++++
meta/recipes-devtools/buildchroot/buildchroot.bb | 59 ++----
meta/recipes-devtools/buildchroot/files/build.sh | 2 +-
.../buildchroot/files/configscript.sh | 10 -
.../buildchroot/files/multistrap.conf.in | 37 ----
meta/recipes-devtools/buildchroot/files/setup.sh | 82 ---------
25 files changed, 310 insertions(+), 388 deletions(-)
create mode 100644 meta-isar/conf/distro/debian-jessie.list
create mode 100644 meta-isar/conf/distro/debian-stretch.list
create mode 100644 meta-isar/conf/distro/debian-wheezy.list
create mode 100644 meta-isar/conf/distro/raspbian-jessie.list
delete mode 100644 meta-isar/recipes-core/images/files/download_dev-random
delete mode 100644 meta-isar/recipes-core/images/files/multistrap.conf.in
delete mode 100755 meta-isar/recipes-core/images/files/setup.sh
create mode 100644 meta/classes/isar-bootstrap-helper.bbclass
create mode 100644 meta/recipes-core/isar-bootstrap/files/isar-apt.conf
create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in
delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh
--
2.16.1
^ permalink raw reply [flat|nested] 27+ messages in thread* [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext @ 2018-03-07 16:44 ` claudius.heine.ext 2018-03-08 7:53 ` Claudius Heine ` (2 more replies) 2018-03-07 16:44 ` [PATCH v4 2/5] meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally claudius.heine.ext ` (4 subsequent siblings) 5 siblings, 3 replies; 27+ messages in thread From: claudius.heine.ext @ 2018-03-07 16:44 UTC (permalink / raw) To: isar-users; +Cc: Claudius Heine From: Claudius Heine <ch@denx.de> Since multistrap is deprecated for some years, it is required to change to debootstrap. This patch introduces the 'isar-bootstrap' recipe that implement the creation of a minimal base root file system using debootstrap. Signed-off-by: Claudius Heine <ch@denx.de> --- meta-isar/conf/distro/debian-jessie.conf | 15 +- meta-isar/conf/distro/debian-jessie.list | 3 + meta-isar/conf/distro/debian-stretch.conf | 11 +- meta-isar/conf/distro/debian-stretch.list | 3 + meta-isar/conf/distro/debian-wheezy.conf | 11 +- meta-isar/conf/distro/debian-wheezy.list | 3 + meta-isar/conf/distro/raspbian-jessie.conf | 10 +- meta-isar/conf/distro/raspbian-jessie.list | 1 + .../isar-bootstrap/files/isar-apt.conf | 2 + meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 +++++++++++++++++++++ 10 files changed, 247 insertions(+), 17 deletions(-) create mode 100644 meta-isar/conf/distro/debian-jessie.list create mode 100644 meta-isar/conf/distro/debian-stretch.list create mode 100644 meta-isar/conf/distro/debian-wheezy.list create mode 100644 meta-isar/conf/distro/raspbian-jessie.list create mode 100644 meta/recipes-core/isar-bootstrap/files/isar-apt.conf create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta-isar/conf/distro/debian-jessie.conf index 3e62b14..df27fd7 100644 --- a/meta-isar/conf/distro/debian-jessie.conf +++ b/meta-isar/conf/distro/debian-jessie.conf @@ -1,16 +1,19 @@ -# Debian jessie multistrap configuration +# Debian jessie distribution configuration # # This software is a part of ISAR. # Copyright (C) 2015-2016 ilbers GmbH +DISTRO_APT_SOURCES += "conf/distro/debian-jessie.list" +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" +DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 armmp \ + armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 octeon \ + orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k-ip32 \ + s390x sb1-bcm91250a versatile" + +# No longer needed: DISTRO_SUITE ?= "jessie" DISTRO_COMPONENTS ?= "main contrib non-free" DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" -DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 armmp \ - armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 octeon \ - orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k-ip32 \ - s390x sb1-bcm91250a versatile" diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta-isar/conf/distro/debian-jessie.list new file mode 100644 index 0000000..be46a57 --- /dev/null +++ b/meta-isar/conf/distro/debian-jessie.list @@ -0,0 +1,3 @@ +deb http://ftp.de.debian.org/debian jessie main contrib non-free +deb http://ftp.de.debian.org/debian jessie-updates main contrib non-free +deb http://security.debian.org jessie/updates main contrib non-free diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta-isar/conf/distro/debian-stretch.conf index 5a85d52..13c4f94 100644 --- a/meta-isar/conf/distro/debian-stretch.conf +++ b/meta-isar/conf/distro/debian-stretch.conf @@ -1,13 +1,16 @@ # This software is a part of ISAR. # Copyright (C) 2017 ilbers GmbH +DISTRO_APT_SOURCES += "conf/distro/debian-stretch.list" +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" +DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 armmp \ + armmp-lpae kirkwood loongson-3 marvell octeon orion5x powerpc64le \ + rt-686-pae rt-amd64 s390x" + +# No longer needed: DISTRO_SUITE ?= "stretch" DISTRO_COMPONENTS ?= "main contrib non-free" DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" -DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 armmp \ - armmp-lpae kirkwood loongson-3 marvell octeon orion5x powerpc64le \ - rt-686-pae rt-amd64 s390x" diff --git a/meta-isar/conf/distro/debian-stretch.list b/meta-isar/conf/distro/debian-stretch.list new file mode 100644 index 0000000..bfa5cf9 --- /dev/null +++ b/meta-isar/conf/distro/debian-stretch.list @@ -0,0 +1,3 @@ +deb http://ftp.de.debian.org/debian stretch main contrib non-free +deb http://ftp.de.debian.org/debian stretch-updates main contrib non-free +deb http://security.debian.org stretch/updates main contrib non-free diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta-isar/conf/distro/debian-wheezy.conf index f5ab6eb..2108f9b 100644 --- a/meta-isar/conf/distro/debian-wheezy.conf +++ b/meta-isar/conf/distro/debian-wheezy.conf @@ -1,14 +1,17 @@ -# Debian wheezy multistrap configuration +# Debian wheezy distribution configuration # # This software is a part of ISAR. # Copyright (C) 2015-2016 ilbers GmbH +DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list" +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" +DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile vexpress \ + iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" + +# No longer needed: DISTRO_SUITE ?= "wheezy" DISTRO_COMPONENTS ?= "main contrib non-free" DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" -DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile vexpress \ - iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" diff --git a/meta-isar/conf/distro/debian-wheezy.list b/meta-isar/conf/distro/debian-wheezy.list new file mode 100644 index 0000000..0bfa98e --- /dev/null +++ b/meta-isar/conf/distro/debian-wheezy.list @@ -0,0 +1,3 @@ +deb http://ftp.de.debian.org/debian wheezy main contrib non-free +deb http://ftp.de.debian.org/debian wheezy-updates main contrib non-free +deb http://security.debian.org wheezy/updates main contrib non-free diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-isar/conf/distro/raspbian-jessie.conf index c2a867d..57d0e7d 100644 --- a/meta-isar/conf/distro/raspbian-jessie.conf +++ b/meta-isar/conf/distro/raspbian-jessie.conf @@ -1,13 +1,17 @@ -# Raspbian stable multistrap configuration +# Raspbian stable distribution configuration # # This software is a part of ISAR. # Copyright (C) 2015-2016 ilbers GmbH +DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list" +DISTRO_APT_KEYS += "https://archive.raspbian.org/raspbian.public.key;sha256sum=ca59cd4f2bcbc3a1d41ba6815a02a8dc5c175467a59bd87edeac458f4a5345de" +DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh" +DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" + +# No longer needed: DISTRO_SUITE ?= "jessie" DISTRO_COMPONENTS ?= "main contrib non-free firmware" DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian" DISTRO_APT_SOURCE_SEC ?= "" -DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh" DISTRO_MULTICONF_BOOTSTRAP ?= "base" DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" -DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta-isar/conf/distro/raspbian-jessie.list new file mode 100644 index 0000000..792f9c6 --- /dev/null +++ b/meta-isar/conf/distro/raspbian-jessie.list @@ -0,0 +1 @@ +deb http://archive.raspbian.org/raspbian jessie main contrib non-free firmware diff --git a/meta/recipes-core/isar-bootstrap/files/isar-apt.conf b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf new file mode 100644 index 0000000..a62feb1 --- /dev/null +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf @@ -0,0 +1,2 @@ +APT::Install-Recommends "0"; +APT::Install-Suggests "0"; diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb new file mode 100644 index 0000000..2ddefa4 --- /dev/null +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb @@ -0,0 +1,205 @@ +# Minimal debian root file system +# +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2018 +# +# SPDX-License-Identifier: MIT + +Description = "Minimal debian root file system" + +LICENSE = "gpl-2.0" +LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" +FILESPATH_prepend := "${THISDIR}/files:" +SRC_URI = "file://isar-apt.conf" +PV = "1.0" + +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" +DEBOOTSTRAP ?= "" +ROOTFSDIR = "${WORKDIR}/rootfs" +APTPREFS = "${WORKDIR}/apt-preferences" +APTSRCS = "${WORKDIR}/apt-sources" +APTKEYFILES = "" +APTKEYRING = "${WORKDIR}/apt-keyring.gpg" +DEBOOTSTRAP_KEYRING = "" + +python () { + from urllib.parse import urlparse + + debootstrap = d.getVar("DEBOOTSTRAP", True) + if not debootstrap: + target = d.getVar("DISTRO_ARCH", True) + machine = os.uname()[4] + m = { + "x86_64": ["i386", "amd64"], + "x86": ["i386"], + } + if machine not in m or target not in m[machine]: + debootstrap = "qemu-debootstrap" + else: + debootstrap = "debootstrap" + d.setVar("DEBOOTSTRAP", debootstrap) + + distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False) + if distro_apt_keys: + d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}") + for key in distro_apt_keys.split(): + url = urlparse(key) + filename = os.path.basename(url.path) + d.appendVar("SRC_URI", " %s" % key) + d.appendVar("APTKEYFILES", " %s" % filename) +} + +def aggregate_files(d, file_list, file_out): + import shutil + + with open(file_out, "wb") as out_fd: + for entry in file_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "rb") as in_fd: + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) + out_fd.write("\n".encode()) + +def parse_aptsources_list_line(source_list_line): + import re + + s = source_list_line.strip() + + if s.startswith("#"): + return None + + type, s = re.split("\s+", s, maxsplit=1) + if type not in ["deb", "deb-src"]: + return None + + options = "" + options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) + if options_match: + options = options_match.group(0).strip() + s = s[options_match.end():] + + source, s = re.split("\s+", s, maxsplit=1) + + suite, s = re.split("\s+", s, maxsplit=1) + + components = " ".join(s.split()) + + return type, options, source, suite, components + +def get_distro_primary_source_entry(d): + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() + for entry in apt_sources_list: + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + type, _, source, suite, components = parsed + if type == "deb": + return source, suite, components + return "", "", "" + +def get_distro_source(d): + return get_distro_primary_source_entry(d)[0] + +def get_distro_suite(d): + return get_distro_primary_source_entry(d)[1] + +def get_distro_components_argument(d): + components = get_distro_primary_source_entry(d)[2] + if components and components.strip(): + return "--components=%s" % ",".join(components.split()) + else: + return "" + +do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +do_generate_keyring[dirs] = "${WORKDIR}" +do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" +do_generate_keyring() { + if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then + for keyfile in ${@d.getVar("APTKEYFILES", True)}; do + gpg --no-default-keyring --keyring "${APTKEYRING}" --import "$keyfile" + done + fi +} +addtask generate_keyring before do_build after do_unpack + +do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +do_apt_config_prepare[vardeps] += "\ + APTPREFS \ + DISTRO_APT_PREFERENCES \ + DEBDISTRONAME \ + APTSRCS \ + DISTRO_APT_SOURCES \ + " +python do_apt_config_prepare() { + apt_preferences_out = d.getVar("APTPREFS", True) + apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", True) or "" + ).split() + aggregate_files(d, apt_preferences_list, apt_preferences_out) + + apt_sources_out = d.getVar("APTSRCS", True) + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() + + aggregate_files(d, apt_sources_list, apt_sources_out) +} +addtask apt_config_prepare before do_build after do_generate_keyring + +do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" +do_bootstrap() { + if [ -e "${ROOTFSDIR}" ]; then + sudo umount -l "${ROOTFSDIR}/dev" || true + sudo umount -l "${ROOTFSDIR}/proc" || true + sudo rm -rf "${ROOTFSDIR}" + fi + E="${@bb.utils.export_proxies(d)}" + sudo -E "${DEBOOTSTRAP}" --verbose \ + --variant minbase \ + --arch "${DISTRO_ARCH}" \ + ${@get_distro_components_argument(d)} \ + ${DEBOOTSTRAP_KEYRING} \ + "${@get_distro_suite(d)}" \ + "${ROOTFSDIR}" \ + "${@get_distro_source(d)}" +} +addtask bootstrap before do_build after do_apt_config_prepare + +do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +do_apt_config_install() { + sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" + sudo install -v -m644 "${APTPREFS}" \ + "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" + sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" + sudo install -v -m644 "${APTSRCS}" \ + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" + sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list" + sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" + sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ + "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" +} +addtask apt_config_install before do_build after do_bootstrap + +do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +do_apt_update() { + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev + sudo mount -t proc none ${ROOTFSDIR}/proc + + E="${@bb.utils.export_proxies(d)}" + export DEBIAN_FRONTEND=noninteractive + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ + -o Debug::pkgProblemResolver=yes +} +addtask apt_update before do_build after do_apt_config_install + +do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}" +do_deploy() { + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" +} +addtask deploy before do_build after do_apt_update + +CLEANFUNCS = "clean_deploy" +clean_deploy() { + rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" +} -- 2.16.1 ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext @ 2018-03-08 7:53 ` Claudius Heine 2018-03-14 8:56 ` Alexander Smirnov 2018-03-14 14:04 ` Alexander Smirnov 2 siblings, 0 replies; 27+ messages in thread From: Claudius Heine @ 2018-03-08 7:53 UTC (permalink / raw) To: claudius.heine.ext, isar-users [-- Attachment #1: Type: text/plain, Size: 19269 bytes --] Hi, just reviewed it with a fresh mind. There are still some places where stuff can be optimized. But IMO that could also be done later. I might not have time to do it this month myself. Here are some of my points: On Wed, 2018-03-07 at 17:44 +0100, claudius.heine.ext@siemens.com wrote: > From: Claudius Heine <ch@denx.de> > > Since multistrap is deprecated for some years, it is required to > change > to debootstrap. > > This patch introduces the 'isar-bootstrap' recipe that implement the > creation of a minimal base root file system using debootstrap. > > Signed-off-by: Claudius Heine <ch@denx.de> > --- > meta-isar/conf/distro/debian-jessie.conf | 15 +- > meta-isar/conf/distro/debian-jessie.list | 3 + > meta-isar/conf/distro/debian-stretch.conf | 11 +- > meta-isar/conf/distro/debian-stretch.list | 3 + > meta-isar/conf/distro/debian-wheezy.conf | 11 +- > meta-isar/conf/distro/debian-wheezy.list | 3 + > meta-isar/conf/distro/raspbian-jessie.conf | 10 +- > meta-isar/conf/distro/raspbian-jessie.list | 1 + > .../isar-bootstrap/files/isar-apt.conf | 2 + > meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 > +++++++++++++++++++++ > 10 files changed, 247 insertions(+), 17 deletions(-) > create mode 100644 meta-isar/conf/distro/debian-jessie.list > create mode 100644 meta-isar/conf/distro/debian-stretch.list > create mode 100644 meta-isar/conf/distro/debian-wheezy.list > create mode 100644 meta-isar/conf/distro/raspbian-jessie.list > create mode 100644 meta/recipes-core/isar-bootstrap/files/isar- > apt.conf > create mode 100644 meta/recipes-core/isar-bootstrap/isar- > bootstrap.bb > > diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta- > isar/conf/distro/debian-jessie.conf > index 3e62b14..df27fd7 100644 > --- a/meta-isar/conf/distro/debian-jessie.conf > +++ b/meta-isar/conf/distro/debian-jessie.conf > @@ -1,16 +1,19 @@ > -# Debian jessie multistrap configuration > +# Debian jessie distribution configuration > # > # This software is a part of ISAR. > # Copyright (C) 2015-2016 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/debian-jessie.list" > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > +DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 > armmp \ > + armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 > octeon \ > + orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k- > ip32 \ > + s390x sb1-bcm91250a versatile" > + > +# No longer needed: > DISTRO_SUITE ?= "jessie" > DISTRO_COMPONENTS ?= "main contrib non-free" > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security > " > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 > armmp \ > - armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 > octeon \ > - orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k- > ip32 \ > - s390x sb1-bcm91250a versatile" > diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta- > isar/conf/distro/debian-jessie.list > new file mode 100644 > index 0000000..be46a57 > --- /dev/null > +++ b/meta-isar/conf/distro/debian-jessie.list > @@ -0,0 +1,3 @@ > +deb http://ftp.de.debian.org/debian class="Apple-tab-span" > style="white-space:pre"> jessie main contrib non-free > +deb http://ftp.de.debian.org/debian jessie-updates mai > n contrib non-free > +deb http://security.debian.org class="Apple-tab-span" > style="white-space:pre"> jessie/updates main contrib non-free > diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta- > isar/conf/distro/debian-stretch.conf > index 5a85d52..13c4f94 100644 > --- a/meta-isar/conf/distro/debian-stretch.conf > +++ b/meta-isar/conf/distro/debian-stretch.conf > @@ -1,13 +1,16 @@ > # This software is a part of ISAR. > # Copyright (C) 2017 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/debian-stretch.list" > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > +DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 > armmp \ > + armmp-lpae kirkwood loongson-3 marvell octeon orion5x > powerpc64le \ > + rt-686-pae rt-amd64 s390x" > + > +# No longer needed: > DISTRO_SUITE ?= "stretch" > DISTRO_COMPONENTS ?= "main contrib non-free" > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security > " > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 > armmp \ > - armmp-lpae kirkwood loongson-3 marvell octeon orion5x > powerpc64le \ > - rt-686-pae rt-amd64 s390x" > diff --git a/meta-isar/conf/distro/debian-stretch.list b/meta- > isar/conf/distro/debian-stretch.list > new file mode 100644 > index 0000000..bfa5cf9 > --- /dev/null > +++ b/meta-isar/conf/distro/debian-stretch.list > @@ -0,0 +1,3 @@ > +deb http://ftp.de.debian.org/debian stretch main > contrib non-free > +deb http://ftp.de.debian.org/debian stretch-updates ma > in contrib non-free > +deb http://security.debian.org class="Apple-tab-span" > style="white-space:pre"> stretch/updates main contrib non-free > diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta- > isar/conf/distro/debian-wheezy.conf > index f5ab6eb..2108f9b 100644 > --- a/meta-isar/conf/distro/debian-wheezy.conf > +++ b/meta-isar/conf/distro/debian-wheezy.conf > @@ -1,14 +1,17 @@ > -# Debian wheezy multistrap configuration > +# Debian wheezy distribution configuration > # > # This software is a part of ISAR. > # Copyright (C) 2015-2016 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list" > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > +DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile > vexpress \ > + iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" > + > +# No longer needed: > DISTRO_SUITE ?= "wheezy" > DISTRO_COMPONENTS ?= "main contrib non-free" > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security > " > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile > vexpress \ > - iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" > diff --git a/meta-isar/conf/distro/debian-wheezy.list b/meta- > isar/conf/distro/debian-wheezy.list > new file mode 100644 > index 0000000..0bfa98e > --- /dev/null > +++ b/meta-isar/conf/distro/debian-wheezy.list > @@ -0,0 +1,3 @@ > +deb http://ftp.de.debian.org/debian wheezy main > contrib non-free > +deb http://ftp.de.debian.org/debian wheezy-updates mai > n contrib non-free > +deb http://security.debian.org class="Apple-tab-span" > style="white-space:pre"> wheezy/updates main contrib non-free > diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta- > isar/conf/distro/raspbian-jessie.conf > index c2a867d..57d0e7d 100644 > --- a/meta-isar/conf/distro/raspbian-jessie.conf > +++ b/meta-isar/conf/distro/raspbian-jessie.conf > @@ -1,13 +1,17 @@ > -# Raspbian stable multistrap configuration > +# Raspbian stable distribution configuration > # > # This software is a part of ISAR. > # Copyright (C) 2015-2016 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list" > +DISTRO_APT_KEYS += "https://archive.raspbian.org/raspbian.public.key > ;sha256sum=ca59cd4f2bcbc3a1d41ba6815a02a8dc5c175467a59bd87edeac458f4a > 5345de" > +DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh" > +DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" > + > +# No longer needed: > DISTRO_SUITE ?= "jessie" > DISTRO_COMPONENTS ?= "main contrib non-free firmware" > DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian" > DISTRO_APT_SOURCE_SEC ?= "" > -DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" > diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta- > isar/conf/distro/raspbian-jessie.list > new file mode 100644 > index 0000000..792f9c6 > --- /dev/null > +++ b/meta-isar/conf/distro/raspbian-jessie.list > @@ -0,0 +1 @@ > +deb http://archive.raspbian.org/raspbian class="Apple-tab- > span" style="white-space:pre"> jessie main contrib non-free > firmware > diff --git a/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > new file mode 100644 > index 0000000..a62feb1 > --- /dev/null > +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > @@ -0,0 +1,2 @@ > +APT::Install-Recommends "0"; > +APT::Install-Suggests "0"; > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > new file mode 100644 > index 0000000..2ddefa4 > --- /dev/null > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > @@ -0,0 +1,205 @@ > +# Minimal debian root file system > +# > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2018 > +# > +# SPDX-License-Identifier: MIT > + > +Description = "Minimal debian root file system" > + > +LICENSE = "gpl-2.0" > +LIC_FILES_CHKSUM = > "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499 > f7abaabaa882bbe" > +FILESPATH_prepend := "${THISDIR}/files:" > +SRC_URI = "file://isar-apt.conf" > +PV = "1.0" > + > +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > +DEBOOTSTRAP ?= "" > +ROOTFSDIR = "${WORKDIR}/rootfs" > +APTPREFS = "${WORKDIR}/apt-preferences" > +APTSRCS = "${WORKDIR}/apt-sources" > +APTKEYFILES = "" > +APTKEYRING = "${WORKDIR}/apt-keyring.gpg" > +DEBOOTSTRAP_KEYRING = "" > + > +python () { > + from urllib.parse import urlparse > + > + debootstrap = d.getVar("DEBOOTSTRAP", True) > + if not debootstrap: > + target = d.getVar("DISTRO_ARCH", True) > + machine = os.uname()[4] > + m = { > + "x86_64": ["i386", "amd64"], > + "x86": ["i386"], > + } > + if machine not in m or target not in m[machine]: > + debootstrap = "qemu-debootstrap" > + else: > + debootstrap = "debootstrap" > + d.setVar("DEBOOTSTRAP", debootstrap) > + > + distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False) > + if distro_apt_keys: > + d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}") > + for key in distro_apt_keys.split(): > + url = urlparse(key) > + filename = os.path.basename(url.path) > + d.appendVar("SRC_URI", " %s" % key) > + d.appendVar("APTKEYFILES", " %s" % filename) > +} > + > +def aggregate_files(d, file_list, file_out): > + import shutil > + > + with open(file_out, "wb") as out_fd: > + for entry in file_list: > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "rb") as in_fd: > + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) > + out_fd.write("\n".encode()) > + > +def parse_aptsources_list_line(source_list_line): > + import re > + > + s = source_list_line.strip() > + > + if s.startswith("#"): if not s or s.startswith("#"): That way empty lines will be ignored as well. > + return None > + > + type, s = re.split("\s+", s, maxsplit=1) s.split(maxsplit=1) No need for regex. > + if type not in ["deb", "deb-src"]: > + return None > + > + options = "" > + options_match = > re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) Overly complicated regex. Something like this might be good enough: "\[[^\]]*]\s+" I written the complicated one first, because I wanted to extract all the individual options, but that is not really necessary now and maybe ever. > + if options_match: > + options = options_match.group(0).strip() > + s = s[options_match.end():] > + > + source, s = re.split("\s+", s, maxsplit=1) > + > + suite, s = re.split("\s+", s, maxsplit=1) Here re.split can be replaced with just split as well. > + > + components = " ".join(s.split()) > + > + return type, options, source, suite, components This whole function could also be simplified to just one regex. I originally tried that, but I wanted to extract the information completely as seen in the one real regex left. A regex like this could work: r"(?P<type>deb|deb- src)\s+(?P<options>\[[^\]]*]\s+)?(?P<source>\S+)\s+(?P<suite>\S+)\s+(?P <components>.*)" > + > +def get_distro_primary_source_entry(d): > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or > "").split() > + for entry in apt_sources_list: > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "r") as in_fd: > + for line in in_fd: > + parsed = parse_aptsources_list_line(line) > + if parsed: > + type, _, source, suite, components = parsed > + if type == "deb": > + return source, suite, components > + return "", "", "" Maybe extract an iterator out of this function, and use this when generating the bootstrap.list file in do_apt_config_prepare as well. This way parse_aptsources_list_line or the interator itself could later be used to implement PREMIRRORS, similar to the PREMIRRORS in OE. Cheers, Claudius > + > +def get_distro_source(d): > + return get_distro_primary_source_entry(d)[0] > + > +def get_distro_suite(d): > + return get_distro_primary_source_entry(d)[1] > + > +def get_distro_components_argument(d): > + components = get_distro_primary_source_entry(d)[2] > + if components and components.strip(): > + return "--components=%s" % ",".join(components.split()) > + else: > + return "" > + > +do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_generate_keyring[dirs] = "${WORKDIR}" > +do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > +do_generate_keyring() { > + if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then > + for keyfile in ${@d.getVar("APTKEYFILES", True)}; do > + gpg --no-default-keyring --keyring "${APTKEYRING}" -- > import "$keyfile" > + done > + fi > +} > +addtask generate_keyring before do_build after do_unpack > + > +do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_apt_config_prepare[vardeps] += "\ > + APTPREFS \ > + DISTRO_APT_PREFERENCES \ > + DEBDISTRONAME \ > + APTSRCS \ > + DISTRO_APT_SOURCES \ > + " > +python do_apt_config_prepare() { > + apt_preferences_out = d.getVar("APTPREFS", True) > + apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", True) > or "" > + ).split() > + aggregate_files(d, apt_preferences_list, apt_preferences_out) > + > + apt_sources_out = d.getVar("APTSRCS", True) > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or > "").split() > + > + aggregate_files(d, apt_sources_list, apt_sources_out) > +} > +addtask apt_config_prepare before do_build after do_generate_keyring > + > +do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" > +do_bootstrap() { > + if [ -e "${ROOTFSDIR}" ]; then > + sudo umount -l "${ROOTFSDIR}/dev" || true > + sudo umount -l "${ROOTFSDIR}/proc" || true > + sudo rm -rf "${ROOTFSDIR}" > + fi > + E="${@bb.utils.export_proxies(d)}" > + sudo -E "${DEBOOTSTRAP}" --verbose \ > + --variant minbase \ > + --arch "${DISTRO_ARCH}" \ > + ${@get_distro_components_argument(d)} \ > + ${DEBOOTSTRAP_KEYRING} \ > + "${@get_distro_suite(d)}" \ > + "${ROOTFSDIR}" \ > + "${@get_distro_source(d)}" > +} > +addtask bootstrap before do_build after do_apt_config_prepare > + > +do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_apt_config_install() { > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" > + sudo install -v -m644 "${APTPREFS}" \ > + "${ROOTFSDIR}/etc/apt/preferences.d/bootst > rap" > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" > + sudo install -v -m644 "${APTSRCS}" \ > + "${ROOTFSDIR}/etc/apt/sources.list.d/boots > trap.list" > + sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list" > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" > + sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ > + "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.co > nf" > +} > +addtask apt_config_install before do_build after do_bootstrap > + > +do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_apt_update() { > + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs > ${ROOTFSDIR}/dev > + sudo mount -t proc none ${ROOTFSDIR}/proc > + > + E="${@bb.utils.export_proxies(d)}" > + export DEBIAN_FRONTEND=noninteractive > + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ > + -o > Debug::pkgProblemResolver=yes > +} > +addtask apt_update before do_build after do_apt_config_install > + > +do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}" > +do_deploy() { > + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}- > ${DISTRO_ARCH}" > +} > +addtask deploy before do_build after do_apt_update > + > +CLEANFUNCS = "clean_deploy" > +clean_deploy() { > + rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" > +} -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153 Keyserver: hkp://pool.sks-keyservers.net [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext 2018-03-08 7:53 ` Claudius Heine @ 2018-03-14 8:56 ` Alexander Smirnov 2018-03-14 9:52 ` Claudius Heine 2018-03-14 14:04 ` Alexander Smirnov 2 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-14 8:56 UTC (permalink / raw) To: claudius.heine.ext, isar-users; +Cc: Claudius Heine On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: > From: Claudius Heine <ch@denx.de> > > Since multistrap is deprecated for some years, it is required to change > to debootstrap. > > This patch introduces the 'isar-bootstrap' recipe that implement the > creation of a minimal base root file system using debootstrap. > Why do you use term 'bootstrap'? It's completely different topic than debootstrap. According to the wiki: https://wiki.debian.org/DebianBootstrap#Bootstrapping Bootstraping is a process of creation Debian from zero to a full archive. While debootstrap: https://wiki.debian.org/Debootstrap is a tool which just installs already existing Debian distro. So, to avoid misunderstanding with Debian community, the names of recipes/tasks should not contain buzzwords that belong to different topics. I propose to use the following: isar-debootstrap.bb do_debootstrap etc... Alex ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 8:56 ` Alexander Smirnov @ 2018-03-14 9:52 ` Claudius Heine 2018-03-14 10:38 ` Alexander Smirnov 0 siblings, 1 reply; 27+ messages in thread From: Claudius Heine @ 2018-03-14 9:52 UTC (permalink / raw) To: Alexander Smirnov, claudius.heine.ext, isar-users [-- Attachment #1: Type: text/plain, Size: 1891 bytes --] Hi Alex, On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: > On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: > > From: Claudius Heine <ch@denx.de> > > > > Since multistrap is deprecated for some years, it is required to > > change > > to debootstrap. > > > > This patch introduces the 'isar-bootstrap' recipe that implement > > the > > creation of a minimal base root file system using debootstrap. > > > > Why do you use term 'bootstrap'? It's completely different topic > than > debootstrap. Because I want to be implementation independent. If for some reason someone wants to change the name of the tool again (I know its not to be expected), this way only the implementation of the isar-bootstrap needs to be changed. In theory with an alternative implementation of some interfaces other distribution could be supported. I don't know in which direction isar will go, but binding your interface names to tightly to the name of some third-party products is not a good idea. What I tried is called loose coupling and that is generally seen as a good software design pattern. > According to the wiki: > > https://wiki.debian.org/DebianBootstrap#Bootstrapping > > Bootstraping is a process of creation Debian from zero to a full > archive. The word "bootstrapping" is used in many projects and generally describes starting something from almost nothing. Also this patchset doesn't use "DebianBootstrap" is uses "isar-bootstrap". Cheers, Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153 Keyserver: hkp://pool.sks-keyservers.net [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 9:52 ` Claudius Heine @ 2018-03-14 10:38 ` Alexander Smirnov 2018-03-14 14:14 ` Claudius Heine 0 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-14 10:38 UTC (permalink / raw) To: Claudius Heine, claudius.heine.ext, isar-users On 03/14/2018 12:52 PM, Claudius Heine wrote: > Hi Alex, > > On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: >> On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: >>> From: Claudius Heine <ch@denx.de> >>> >>> Since multistrap is deprecated for some years, it is required to >>> change >>> to debootstrap. >>> >>> This patch introduces the 'isar-bootstrap' recipe that implement >>> the >>> creation of a minimal base root file system using debootstrap. >>> >> >> Why do you use term 'bootstrap'? It's completely different topic >> than >> debootstrap. > > Because I want to be implementation independent. If for some reason > someone wants to change the name of the tool again (I know its not to > be expected), this way only the implementation of the isar-bootstrap > needs to be changed. > In theory with an alternative implementation of some interfaces other > distribution could be supported. I don't know in which direction isar > will go, but binding your interface names to tightly to the name of > some third-party products is not a good idea. > > What I tried is called loose coupling and that is generally seen as a > good software design pattern. > >> According to the wiki: >> >> https://wiki.debian.org/DebianBootstrap#Bootstrapping >> >> Bootstraping is a process of creation Debian from zero to a full >> archive. > > The word "bootstrapping" is used in many projects and generally > describes starting something from almost nothing. Also this patchset > doesn't use "DebianBootstrap" is uses "isar-bootstrap". In this case just do not use terms from Debian world. It would be enough to name it like 'base-rootfs'. That's for example, how 'buildchroot' term was invented, to avoid intersections with 'sysroot' and 'buildroot'. AFAIK there are 2 kinds opinions in Debian community about bootstraping: - Build from sources (DebianBootstrap, rebootstrap) - Install from debs (debootstrap) My question, why it's so important to use initially ambiguous term, which may lead to potential confusing for Debian users? Alex ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 10:38 ` Alexander Smirnov @ 2018-03-14 14:14 ` Claudius Heine 2018-03-14 14:25 ` Alexander Smirnov 0 siblings, 1 reply; 27+ messages in thread From: Claudius Heine @ 2018-03-14 14:14 UTC (permalink / raw) To: Alexander Smirnov, claudius.heine.ext, isar-users [-- Attachment #1: Type: text/plain, Size: 4296 bytes --] Hi Alex, On Wed, 2018-03-14 at 13:38 +0300, Alexander Smirnov wrote: > On 03/14/2018 12:52 PM, Claudius Heine wrote: > > Hi Alex, > > > > On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: > > > On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: > > > > From: Claudius Heine <ch@denx.de> > > > > > > > > Since multistrap is deprecated for some years, it is required > > > > to > > > > change > > > > to debootstrap. > > > > > > > > This patch introduces the 'isar-bootstrap' recipe that > > > > implement > > > > the > > > > creation of a minimal base root file system using debootstrap. > > > > > > > > > > Why do you use term 'bootstrap'? It's completely different topic > > > than > > > debootstrap. > > > > Because I want to be implementation independent. If for some reason > > someone wants to change the name of the tool again (I know its not > > to > > be expected), this way only the implementation of the isar- > > bootstrap > > needs to be changed. > > In theory with an alternative implementation of some interfaces > > other > > distribution could be supported. I don't know in which direction > > isar > > will go, but binding your interface names to tightly to the name of > > some third-party products is not a good idea. > > > > What I tried is called loose coupling and that is generally seen as > > a > > good software design pattern. > > > > > According to the wiki: > > > > > > https://wiki.debian.org/DebianBootstrap#Bootstrapping > > > > > > Bootstraping is a process of creation Debian from zero to a full > > > archive. > > > > The word "bootstrapping" is used in many projects and generally > > describes starting something from almost nothing. Also this > > patchset > > doesn't use "DebianBootstrap" is uses "isar-bootstrap". > > In this case just do not use terms from Debian world. I didn't. I use 'isar-bootstrap' and neither it nor its parts 'isar' nor 'bootstrap' is a term that is exclusively used with Debian. 'bootstrap' is a common term in computer science with many different meaning. If someone heard 'bootstrap' only used by the Debian project before, that that is their problem. I think I heard it first in some projects as the 'bootstrap.sh' shell script that takes care about initializing the build environment correctly. I might have been a bit confused when I heard that they also describe the first compilation of compiler by an other one written in a different language as bootstrapping, but that might just be because English is my second language. > It would be enough > to name it like 'base-rootfs'. That's for example, how 'buildchroot' > term was invented, to avoid intersections with 'sysroot' and > 'buildroot'. 'sysroot' and 'buildroot' are names that where invented by someone, 'bootstrap' is a word, like 'face' or 'book'. I don't automatically think of 'facebook' just because I read the word 'book' or 'face' in some other context. Personally I don't like to add another '*base*' thing to isar. We already have 'isar-image-base' and 'dpkg-base'. And this recipe is should not be used as a 'base' to be included or inherited upon like the others. So 'base-roofs' as a recipe name does not fit IMO. (Also when I hear 'base' I think about numbers and get very confused ;) I still prefer 'isar-bootstrap' since its output should be what of some kind of distro-specific system bootstrap machanism, like debootstrap, febootstrap (now supermin), pacstrap, ..., generates. If the distro-specific bootstrap mechanism builds its 'bootstraped root file system' from a bunch of binary packages or directly from source is implementation and distro specific. So the distinction only makes sense in the upstream project but not here, since we are just using what the upstream distributions provides for general consumption. regards, Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153 Keyserver: hkp://pool.sks-keyservers.net [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 14:14 ` Claudius Heine @ 2018-03-14 14:25 ` Alexander Smirnov 2018-03-14 18:53 ` Jan Kiszka 0 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-14 14:25 UTC (permalink / raw) To: Claudius Heine, claudius.heine.ext, isar-users On 03/14/2018 05:14 PM, Claudius Heine wrote: > Hi Alex, > > On Wed, 2018-03-14 at 13:38 +0300, Alexander Smirnov wrote: >> On 03/14/2018 12:52 PM, Claudius Heine wrote: >>> Hi Alex, >>> >>> On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: >>>> On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: >>>>> From: Claudius Heine <ch@denx.de> >>>>> >>>>> Since multistrap is deprecated for some years, it is required >>>>> to >>>>> change >>>>> to debootstrap. >>>>> >>>>> This patch introduces the 'isar-bootstrap' recipe that >>>>> implement >>>>> the >>>>> creation of a minimal base root file system using debootstrap. >>>>> >>>> >>>> Why do you use term 'bootstrap'? It's completely different topic >>>> than >>>> debootstrap. >>> >>> Because I want to be implementation independent. If for some reason >>> someone wants to change the name of the tool again (I know its not >>> to >>> be expected), this way only the implementation of the isar- >>> bootstrap >>> needs to be changed. >>> In theory with an alternative implementation of some interfaces >>> other >>> distribution could be supported. I don't know in which direction >>> isar >>> will go, but binding your interface names to tightly to the name of >>> some third-party products is not a good idea. >>> >>> What I tried is called loose coupling and that is generally seen as >>> a >>> good software design pattern. >>> >>>> According to the wiki: >>>> >>>> https://wiki.debian.org/DebianBootstrap#Bootstrapping >>>> >>>> Bootstraping is a process of creation Debian from zero to a full >>>> archive. >>> >>> The word "bootstrapping" is used in many projects and generally >>> describes starting something from almost nothing. Also this >>> patchset >>> doesn't use "DebianBootstrap" is uses "isar-bootstrap". >> >> In this case just do not use terms from Debian world. > > I didn't. I use 'isar-bootstrap' and neither it nor its parts 'isar' > nor 'bootstrap' is a term that is exclusively used with Debian. > > 'bootstrap' is a common term in computer science with many different > meaning. If someone heard 'bootstrap' only used by the Debian project > before, that that is their problem. I think I heard it first in some > projects as the 'bootstrap.sh' shell script that takes care about > initializing the build environment correctly. I might have been a bit > confused when I heard that they also describe the first compilation of > compiler by an other one written in a different language as > bootstrapping, but that might just be because English is my second > language. > >> It would be enough >> to name it like 'base-rootfs'. That's for example, how 'buildchroot' >> term was invented, to avoid intersections with 'sysroot' and >> 'buildroot'. > > 'sysroot' and 'buildroot' are names that where invented by someone, > 'bootstrap' is a word, like 'face' or 'book'. I don't automatically > think of 'facebook' just because I read the word 'book' or 'face' in > some other context. > > Personally I don't like to add another '*base*' thing to isar. We I didn't ask to use it, I only tried to describe the idea. Whatever else could be used, but without intersections with Debian. > already have 'isar-image-base' and 'dpkg-base'. And this recipe is > should not be used as a 'base' to be included or inherited upon like > the others. So 'base-roofs' as a recipe name does not fit IMO. > > (Also when I hear 'base' I think about numbers and get very confused ;) > > I still prefer 'isar-bootstrap' since its output should be what of some > kind of distro-specific system bootstrap machanism, like debootstrap, > febootstrap (now supermin), pacstrap, ..., generates. > If the distro-specific bootstrap mechanism builds its 'bootstraped root > file system' from a bunch of binary packages or directly from source is > implementation and distro specific. So the distinction only makes sense > in the upstream project but not here, since we are just using what the > upstream distributions provides for general consumption. I asked the question, but you cut it, so I'm going to repost the whole quote here: 8<-- AFAIK there are 2 kinds opinions in Debian community about bootstraping: - Build from sources (DebianBootstrap, rebootstrap) - Install from debs (debootstrap) My question, why it's so important to use initially ambiguous term, which may lead to potential confusing for Debian users? 8<-- Alex ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 14:25 ` Alexander Smirnov @ 2018-03-14 18:53 ` Jan Kiszka 2018-03-15 8:05 ` Claudius Heine 0 siblings, 1 reply; 27+ messages in thread From: Jan Kiszka @ 2018-03-14 18:53 UTC (permalink / raw) To: Alexander Smirnov, Claudius Heine, claudius.heine.ext, isar-users On 2018-03-14 07:25, Alexander Smirnov wrote: > On 03/14/2018 05:14 PM, Claudius Heine wrote: >> Hi Alex, >> >> On Wed, 2018-03-14 at 13:38 +0300, Alexander Smirnov wrote: >>> On 03/14/2018 12:52 PM, Claudius Heine wrote: >>>> Hi Alex, >>>> >>>> On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: >>>>> On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: >>>>>> From: Claudius Heine <ch@denx.de> >>>>>> >>>>>> Since multistrap is deprecated for some years, it is required >>>>>> to >>>>>> change >>>>>> to debootstrap. >>>>>> >>>>>> This patch introduces the 'isar-bootstrap' recipe that >>>>>> implement >>>>>> the >>>>>> creation of a minimal base root file system using debootstrap. >>>>>> >>>>> >>>>> Why do you use term 'bootstrap'? It's completely different topic >>>>> than >>>>> debootstrap. >>>> >>>> Because I want to be implementation independent. If for some reason >>>> someone wants to change the name of the tool again (I know its not >>>> to >>>> be expected), this way only the implementation of the isar- >>>> bootstrap >>>> needs to be changed. >>>> In theory with an alternative implementation of some interfaces >>>> other >>>> distribution could be supported. I don't know in which direction >>>> isar >>>> will go, but binding your interface names to tightly to the name of >>>> some third-party products is not a good idea. >>>> >>>> What I tried is called loose coupling and that is generally seen as >>>> a >>>> good software design pattern. >>>> >>>>> According to the wiki: >>>>> >>>>> https://wiki.debian.org/DebianBootstrap#Bootstrapping >>>>> >>>>> Bootstraping is a process of creation Debian from zero to a full >>>>> archive. >>>> >>>> The word "bootstrapping" is used in many projects and generally >>>> describes starting something from almost nothing. Also this >>>> patchset >>>> doesn't use "DebianBootstrap" is uses "isar-bootstrap". >>> >>> In this case just do not use terms from Debian world. >> >> I didn't. I use 'isar-bootstrap' and neither it nor its parts 'isar' >> nor 'bootstrap' is a term that is exclusively used with Debian. >> >> 'bootstrap' is a common term in computer science with many different >> meaning. If someone heard 'bootstrap' only used by the Debian project >> before, that that is their problem. I think I heard it first in some >> projects as the 'bootstrap.sh' shell script that takes care about >> initializing the build environment correctly. I might have been a bit >> confused when I heard that they also describe the first compilation of >> compiler by an other one written in a different language as >> bootstrapping, but that might just be because English is my second >> language. >> >>> It would be enough >>> to name it like 'base-rootfs'. That's for example, how 'buildchroot' >>> term was invented, to avoid intersections with 'sysroot' and >>> 'buildroot'. >> >> 'sysroot' and 'buildroot' are names that where invented by someone, >> 'bootstrap' is a word, like 'face' or 'book'. I don't automatically >> think of 'facebook' just because I read the word 'book' or 'face' in >> some other context. >> >> Personally I don't like to add another '*base*' thing to isar. We > > I didn't ask to use it, I only tried to describe the idea. Whatever else > could be used, but without intersections with Debian. > >> already have 'isar-image-base' and 'dpkg-base'. And this recipe is >> should not be used as a 'base' to be included or inherited upon like >> the others. So 'base-roofs' as a recipe name does not fit IMO. >> >> (Also when I hear 'base' I think about numbers and get very confused ;) >> >> I still prefer 'isar-bootstrap' since its output should be what of some >> kind of distro-specific system bootstrap machanism, like debootstrap, >> febootstrap (now supermin), pacstrap, ..., generates. >> If the distro-specific bootstrap mechanism builds its 'bootstraped root >> file system' from a bunch of binary packages or directly from source is >> implementation and distro specific. So the distinction only makes sense >> in the upstream project but not here, since we are just using what the >> upstream distributions provides for general consumption. > > I asked the question, but you cut it, so I'm going to repost the whole > quote here: > > 8<-- > > AFAIK there are 2 kinds opinions in Debian community about bootstraping: > - Build from sources (DebianBootstrap, rebootstrap) > - Install from debs (debootstrap) > > My question, why it's so important to use initially ambiguous term, > which may lead to potential confusing for Debian users? > > 8<-- I hope we can settle on this topic quickly, specifically as this internal naming of classes are not really user-facing. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 18:53 ` Jan Kiszka @ 2018-03-15 8:05 ` Claudius Heine 2018-03-15 8:58 ` Alexander Smirnov 0 siblings, 1 reply; 27+ messages in thread From: Claudius Heine @ 2018-03-15 8:05 UTC (permalink / raw) To: Jan Kiszka, Alexander Smirnov, Claudius Heine, isar-users Hi Alex, hi Jan, On 03/14/2018 07:53 PM, Jan Kiszka wrote: > On 2018-03-14 07:25, Alexander Smirnov wrote: >> On 03/14/2018 05:14 PM, Claudius Heine wrote: >>> Hi Alex, >>> >>> On Wed, 2018-03-14 at 13:38 +0300, Alexander Smirnov wrote: >>>> On 03/14/2018 12:52 PM, Claudius Heine wrote: >>>>> Hi Alex, >>>>> >>>>> On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: >>>>>> On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: >>>>>>> From: Claudius Heine <ch@denx.de> >>>>>>> >>>>>>> Since multistrap is deprecated for some years, it is required >>>>>>> to >>>>>>> change >>>>>>> to debootstrap. >>>>>>> >>>>>>> This patch introduces the 'isar-bootstrap' recipe that >>>>>>> implement >>>>>>> the >>>>>>> creation of a minimal base root file system using debootstrap. >>>>>>> >>>>>> >>>>>> Why do you use term 'bootstrap'? It's completely different topic >>>>>> than >>>>>> debootstrap. >>>>> >>>>> Because I want to be implementation independent. If for some reason >>>>> someone wants to change the name of the tool again (I know its not >>>>> to >>>>> be expected), this way only the implementation of the isar- >>>>> bootstrap >>>>> needs to be changed. >>>>> In theory with an alternative implementation of some interfaces >>>>> other >>>>> distribution could be supported. I don't know in which direction >>>>> isar >>>>> will go, but binding your interface names to tightly to the name of >>>>> some third-party products is not a good idea. >>>>> >>>>> What I tried is called loose coupling and that is generally seen as >>>>> a >>>>> good software design pattern. >>>>> >>>>>> According to the wiki: >>>>>> >>>>>> https://wiki.debian.org/DebianBootstrap#Bootstrapping >>>>>> >>>>>> Bootstraping is a process of creation Debian from zero to a full >>>>>> archive. >>>>> >>>>> The word "bootstrapping" is used in many projects and generally >>>>> describes starting something from almost nothing. Also this >>>>> patchset >>>>> doesn't use "DebianBootstrap" is uses "isar-bootstrap". >>>> >>>> In this case just do not use terms from Debian world. >>> >>> I didn't. I use 'isar-bootstrap' and neither it nor its parts 'isar' >>> nor 'bootstrap' is a term that is exclusively used with Debian. >>> >>> 'bootstrap' is a common term in computer science with many different >>> meaning. If someone heard 'bootstrap' only used by the Debian project >>> before, that that is their problem. I think I heard it first in some >>> projects as the 'bootstrap.sh' shell script that takes care about >>> initializing the build environment correctly. I might have been a bit >>> confused when I heard that they also describe the first compilation of >>> compiler by an other one written in a different language as >>> bootstrapping, but that might just be because English is my second >>> language. >>> >>>> It would be enough >>>> to name it like 'base-rootfs'. That's for example, how 'buildchroot' >>>> term was invented, to avoid intersections with 'sysroot' and >>>> 'buildroot'. >>> >>> 'sysroot' and 'buildroot' are names that where invented by someone, >>> 'bootstrap' is a word, like 'face' or 'book'. I don't automatically >>> think of 'facebook' just because I read the word 'book' or 'face' in >>> some other context. >>> >>> Personally I don't like to add another '*base*' thing to isar. We >> >> I didn't ask to use it, I only tried to describe the idea. Whatever else >> could be used, but without intersections with Debian. So then why did you bring this up if you don't have an alternative suggestion I should use instead? That just sounds like destructive criticism. Since I don't think that there is an intersection in the nomenclatures of Debian with the name 'isar-bootstrap' I choose to use this name. Or should I now start guessing what other name you might prefer? What is this? >> >>> already have 'isar-image-base' and 'dpkg-base'. And this recipe is >>> should not be used as a 'base' to be included or inherited upon like >>> the others. So 'base-roofs' as a recipe name does not fit IMO. >>> >>> (Also when I hear 'base' I think about numbers and get very confused ;) >>> >>> I still prefer 'isar-bootstrap' since its output should be what of some >>> kind of distro-specific system bootstrap machanism, like debootstrap, >>> febootstrap (now supermin), pacstrap, ..., generates. >>> If the distro-specific bootstrap mechanism builds its 'bootstraped root >>> file system' from a bunch of binary packages or directly from source is >>> implementation and distro specific. So the distinction only makes sense >>> in the upstream project but not here, since we are just using what the >>> upstream distributions provides for general consumption. >> >> I asked the question, but you cut it, so I'm going to repost the whole >> quote here: >> >> 8<-- >> >> AFAIK there are 2 kinds opinions in Debian community about bootstraping: >> - Build from sources (DebianBootstrap, rebootstrap) >> - Install from debs (debootstrap) >> >> My question, why it's so important to use initially ambiguous term, >> which may lead to potential confusing for Debian users? >> >> 8<-- > > I hope we can settle on this topic quickly, specifically as this > internal naming of classes are not really user-facing. I really hope so as well. Arguing about internal names is very annoying. Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-15 8:05 ` Claudius Heine @ 2018-03-15 8:58 ` Alexander Smirnov 2018-03-15 22:07 ` Claudius Heine 0 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-15 8:58 UTC (permalink / raw) To: Claudius Heine, Jan Kiszka, Claudius Heine, isar-users On 03/15/2018 11:05 AM, Claudius Heine wrote: > Hi Alex, hi Jan, > > On 03/14/2018 07:53 PM, Jan Kiszka wrote: >> On 2018-03-14 07:25, Alexander Smirnov wrote: >>> On 03/14/2018 05:14 PM, Claudius Heine wrote: >>>> Hi Alex, >>>> >>>> On Wed, 2018-03-14 at 13:38 +0300, Alexander Smirnov wrote: >>>>> On 03/14/2018 12:52 PM, Claudius Heine wrote: >>>>>> Hi Alex, >>>>>> >>>>>> On Wed, 2018-03-14 at 11:56 +0300, Alexander Smirnov wrote: >>>>>>> On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: >>>>>>>> From: Claudius Heine <ch@denx.de> >>>>>>>> >>>>>>>> Since multistrap is deprecated for some years, it is required >>>>>>>> to >>>>>>>> change >>>>>>>> to debootstrap. >>>>>>>> >>>>>>>> This patch introduces the 'isar-bootstrap' recipe that >>>>>>>> implement >>>>>>>> the >>>>>>>> creation of a minimal base root file system using debootstrap. >>>>>>>> >>>>>>> >>>>>>> Why do you use term 'bootstrap'? It's completely different topic >>>>>>> than >>>>>>> debootstrap. >>>>>> >>>>>> Because I want to be implementation independent. If for some reason >>>>>> someone wants to change the name of the tool again (I know its not >>>>>> to >>>>>> be expected), this way only the implementation of the isar- >>>>>> bootstrap >>>>>> needs to be changed. >>>>>> In theory with an alternative implementation of some interfaces >>>>>> other >>>>>> distribution could be supported. I don't know in which direction >>>>>> isar >>>>>> will go, but binding your interface names to tightly to the name of >>>>>> some third-party products is not a good idea. >>>>>> >>>>>> What I tried is called loose coupling and that is generally seen as >>>>>> a >>>>>> good software design pattern. >>>>>> >>>>>>> According to the wiki: >>>>>>> >>>>>>> https://wiki.debian.org/DebianBootstrap#Bootstrapping >>>>>>> >>>>>>> Bootstraping is a process of creation Debian from zero to a full >>>>>>> archive. >>>>>> >>>>>> The word "bootstrapping" is used in many projects and generally >>>>>> describes starting something from almost nothing. Also this >>>>>> patchset >>>>>> doesn't use "DebianBootstrap" is uses "isar-bootstrap". >>>>> >>>>> In this case just do not use terms from Debian world. >>>> >>>> I didn't. I use 'isar-bootstrap' and neither it nor its parts 'isar' >>>> nor 'bootstrap' is a term that is exclusively used with Debian. >>>> >>>> 'bootstrap' is a common term in computer science with many different >>>> meaning. If someone heard 'bootstrap' only used by the Debian project >>>> before, that that is their problem. I think I heard it first in some >>>> projects as the 'bootstrap.sh' shell script that takes care about >>>> initializing the build environment correctly. I might have been a bit >>>> confused when I heard that they also describe the first compilation of >>>> compiler by an other one written in a different language as >>>> bootstrapping, but that might just be because English is my second >>>> language. >>>> >>>>> It would be enough >>>>> to name it like 'base-rootfs'. That's for example, how 'buildchroot' >>>>> term was invented, to avoid intersections with 'sysroot' and >>>>> 'buildroot'. >>>> >>>> 'sysroot' and 'buildroot' are names that where invented by someone, >>>> 'bootstrap' is a word, like 'face' or 'book'. I don't automatically >>>> think of 'facebook' just because I read the word 'book' or 'face' in >>>> some other context. >>>> >>>> Personally I don't like to add another '*base*' thing to isar. We >>> >>> I didn't ask to use it, I only tried to describe the idea. Whatever else >>> could be used, but without intersections with Debian. > > So then why did you bring this up if you don't have an alternative > suggestion I should use instead? That just sounds like destructive > criticism. > > Since I don't think that there is an intersection in the nomenclatures > of Debian with the name 'isar-bootstrap' I choose to use this name. > > Or should I now start guessing what other name you might prefer? What is > this? > >>> >>>> already have 'isar-image-base' and 'dpkg-base'. And this recipe is >>>> should not be used as a 'base' to be included or inherited upon like >>>> the others. So 'base-roofs' as a recipe name does not fit IMO. >>>> >>>> (Also when I hear 'base' I think about numbers and get very confused ;) >>>> >>>> I still prefer 'isar-bootstrap' since its output should be what of some >>>> kind of distro-specific system bootstrap machanism, like >>>> debootstrap, >>>> febootstrap (now supermin), pacstrap, ..., generates. >>>> If the distro-specific bootstrap mechanism builds its 'bootstraped root >>>> file system' from a bunch of binary packages or directly from source is >>>> implementation and distro specific. So the distinction only makes sense >>>> in the upstream project but not here, since we are just using what the >>>> upstream distributions provides for general consumption. >>> >>> I asked the question, but you cut it, so I'm going to repost the whole >>> quote here: >>> >>> 8<-- >>> >>> AFAIK there are 2 kinds opinions in Debian community about bootstraping: >>> - Build from sources (DebianBootstrap, rebootstrap) >>> - Install from debs (debootstrap) >>> >>> My question, why it's so important to use initially ambiguous term, >>> which may lead to potential confusing for Debian users? >>> >>> 8<-- >> >> I hope we can settle on this topic quickly, specifically as this >> internal naming of classes are not really user-facing. > It's not only about internal class names, this series introduces 'do_bootstrap' task that is visible to user and will be a part of documentation. My position is simple: I read patches and if I've found something unclear, I'm trying to understand what was the reason going this way. Regarding bootstraping, I provided above 2 kinds of interpretation of this process in Debian, so this term is initially ambiguous. If you use deboostrap, it's ok to name tasks/recipes/etc respectively (do_debootstrap, ...). If in future this code will be refactored to support other rootfs creation mechanisms, it'll be very easy to rename these files/classes. Moreover, at the moment there is no other option for debootstrap, so it's only about something hypothetical in future, what could never happen. @Jan: I've got your opinion, so nothing else from me. > I really hope so as well. Arguing about internal names is very annoying. > @Claudius: For me is very annoying when my question is ignored after explicit re-posting. So let's be polite and keep conversations here free of emotions. Alex ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-15 8:58 ` Alexander Smirnov @ 2018-03-15 22:07 ` Claudius Heine 2018-03-16 5:49 ` Alexander Smirnov 0 siblings, 1 reply; 27+ messages in thread From: Claudius Heine @ 2018-03-15 22:07 UTC (permalink / raw) To: Alexander Smirnov, Claudius Heine, Jan Kiszka, isar-users [-- Attachment #1: Type: text/plain, Size: 7057 bytes --] Hi Alex, On Thu, 2018-03-15 at 11:58 +0300, Alexander Smirnov wrote: > @Claudius: > For me is very annoying when my question is ignored after explicit > re-posting. So let's be polite and keep conversations here free of > emotions. Read the following text without any emotions, apart from the paragraphs where I added the applicable ascii-emoji: First, I already answered your question indirectly here: >> If the distro-specific bootstrap mechanism builds its 'bootstraped >> root file system' from a bunch of binary packages or directly from >> source is implementation and distro specific. So the distinction >> only makes sense in the upstream project but not here, since we are >> just using what the upstream distributions provides for general >> consumption. Short: I don't agree with the presumptions in this question, therefor I cannot answer it sensible. And second I don't like responding to attempts at manipulation, like you did with this loaded question: > My question, why it's so important to use initially ambiguous term, > which may lead to potential confusing for Debian users? Maybe that is just a misunderstanding, but maybe we should go through it piece by piece and then look at it as a whole, so you can understand what I mean: > My question, why it's so important "why it's so important" reads accusatory, not neutral and together with the following parts of this sentence colors the rest of your question in a negative light. > to use initially ambiguous term, Here you are presuming my guilt by implying that I have chosen an 'ambiguous term' on purpose. I don't think 'isar-bootstrap' or 'bootstrap' is ambiguous in this context. Since generating an initial root file system for linux distributions is called 'bootstrapping the linux distribution' and that is exactly what is done here, so its not ambiguous for me. That debian has two methods of doing it also doesn't matter. Since that is an implementation detail and can be easily looked up. The end result is that we have some kind of minimal root file system that we can use to install our customization on top. > which may lead to potential confusing for Debian users? "may lead to potential confusing" Is it or is it not? That sounds like you are just making things up and spread FUD. It sound like "this code could potentially cause your computer to catch fire and explode". If I would have called something 'debootstrap' that formats your hard drive, then yes, I see that this causes confusion and even more. I don't see why a name like 'isar-bootstrap' or its containing task 'do_bootstrap' could confuse any Debian user that uses Isar. And even if that "may lead to potential confusing" them, it will clear up fast when they just take a look into the 'do_bootstrap' implementation. Also, since Isar is a completely different Project than Debian, people will have other expectations and might be positively surprised that something with 'bootstrap' in its name still does what they have expected it to do ;) So then look at the whole question again and try to answer it: > My question, why it's so important to use initially ambiguous term, > which may lead to potential confusing for Debian users? If I would nitpick then that isn't even a question, its a statement that this is your question. Also, how am I supposed to answer it? (Hold on, this us a rhetorical question and I don't expect an answer.) The answer is obvious. Its like questioning: Why do you think it is right to kill everyone? Maybe I should just reply to this questions and yours: That is not my intention. I'm not sure that this is what you wanted to hear, but your question can now be considered answered. Since that is done now, I follow with my responses to some other parts of your mail: > It's not only about internal class names, this series introduces > 'do_bootstrap' task that is visible to user and will be a part of > documentation. I don't expect many people building on top of isar-bootstrap, since that is now pretty much the core internals of isar. People will build their customization based on the output of isar-bootstrap, not isar- bootstrap itself. > If you use deboostrap, it's ok to name tasks/recipes/etc > respectively (do_debootstrap, ...). But in some cases it uses 'qemu-debootstrap' ;) (Nitpicky again, sorry) Now we have a general name for all the bootstrap mechanisms 'do_bootstrap' and then if the need arise we can change 'do_bootstrap' to some kind of 'virtual' task and have different implementations like 'do_bootstrap_deb_binary', 'do_bootstrap_deb_source', ... For now we have one, because we only support one bootstrapping mechanism or distribution. That is the reverse direction of your suggestion, but this way we have some kind of meta-tasks names already, and changing it later can be done more transparent. > My position is simple: I read patches and if I've found something > unclear, I'm trying to understand what was the reason going this way. That is a good position. And review generally really helps. Here are some suggestions for me to improve the current situation. First, please stay measured. Arguing a long time about minor things like variable names, while there might be many other places in the actual code that could be improved is not. For instance I did like your suggestion to add '--homedir' to gpg, more of that please. In general more suggestions with measurable improvements and less emotions like apprehensions about possibly confusing someone. Second, please more 'trying to understand' instead of forcing your view onto others via questions and or statements. Only after you understood the view point of others you can really critique them. So ask questions first (best in a neutral, open and honest way), after you understood the developers view point and code you can make suggestions for improvements while stating you objective reasons for it. Also appreciating and praising good code occasionally will help a lot spreading a positive and cooperative atmosphere in the project. :) Maybe take a look at [1] and especially [2]. I left some parts of your message out, but I hope that I have made myself implicitly clear how I stand on the other parts of your message. If something is still unclear, then please ask. Cheers, Claudius [1] https://mtlynch.io/human-code-reviews-1/ [2] https://mtlynch.io/human-code-reviews-2/ P.S. For me this kind of discussion now borders more on entertainment than work and I don't think that this is a good sign. ;) -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153 Keyserver: hkp://pool.sks-keyservers.net [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-15 22:07 ` Claudius Heine @ 2018-03-16 5:49 ` Alexander Smirnov 2018-03-28 6:00 ` Jan Kiszka 0 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-16 5:49 UTC (permalink / raw) To: Claudius Heine, Claudius Heine, Jan Kiszka, isar-users On 03/16/2018 01:07 AM, Claudius Heine wrote: > Hi Alex, > > On Thu, 2018-03-15 at 11:58 +0300, Alexander Smirnov wrote: >> @Claudius: >> For me is very annoying when my question is ignored after explicit >> re-posting. So let's be polite and keep conversations here free of >> emotions. > > Read the following text without any emotions, apart from the paragraphs > where I added the applicable ascii-emoji: > > First, I already answered your question indirectly here: > >>> If the distro-specific bootstrap mechanism builds its 'bootstraped >>> root file system' from a bunch of binary packages or directly from >>> source is implementation and distro specific. So the distinction >>> only makes sense in the upstream project but not here, since we are >>> just using what the upstream distributions provides for general >>> consumption. > > Short: I don't agree with the presumptions in this question, therefor I > cannot answer it sensible. > > And second I don't like responding to attempts at manipulation, like > you did with this loaded question: > >> My question, why it's so important to use initially ambiguous term, >> which may lead to potential confusing for Debian users? > > Maybe that is just a misunderstanding, but maybe we should go through > it piece by piece and then look at it as a whole, so you can understand > what I mean: > >> My question, why it's so important > > "why it's so important" reads accusatory, not neutral and together with > the following parts of this sentence colors the rest of your question > in a negative light. > >> to use initially ambiguous term, > > Here you are presuming my guilt by implying that I have chosen an > 'ambiguous term' on purpose. I don't think 'isar-bootstrap' or > 'bootstrap' is ambiguous in this context. Since generating an initial > root file system for linux distributions is called 'bootstrapping the > linux distribution' and that is exactly what is done here, so its not > ambiguous for me. > > That debian has two methods of doing it also doesn't matter. Since that > is an implementation detail and can be easily looked up. The end result > is that we have some kind of minimal root file system that we can use > to install our customization on top. > >> which may lead to potential confusing for Debian users? > > "may lead to potential confusing" Is it or is it not? That sounds like > you are just making things up and spread FUD. It sound like "this code > could potentially cause your computer to catch fire and explode". > > If I would have called something 'debootstrap' that formats your hard > drive, then yes, I see that this causes confusion and even more. > > I don't see why a name like 'isar-bootstrap' or its containing task > 'do_bootstrap' could confuse any Debian user that uses Isar. And even > if that "may lead to potential confusing" them, it will clear up fast > when they just take a look into the 'do_bootstrap' implementation. > > Also, since Isar is a completely different Project than Debian, people > will have other expectations and might be positively surprised that > something with 'bootstrap' in its name still does what they have > expected it to do ;) > > So then look at the whole question again and try to answer it: > >> My question, why it's so important to use initially ambiguous term, >> which may lead to potential confusing for Debian users? > > If I would nitpick then that isn't even a question, its a statement > that this is your question. Also, how am I supposed to answer it? (Hold > on, this us a rhetorical question and I don't expect an answer.) > The answer is obvious. Its like questioning: > > Why do you think it is right to kill everyone? > > Maybe I should just reply to this questions and yours: > > That is not my intention. > > I'm not sure that this is what you wanted to hear, but your question > can now be considered answered. > > Since that is done now, I follow with my responses to some other parts > of your mail: > >> It's not only about internal class names, this series introduces >> 'do_bootstrap' task that is visible to user and will be a part of >> documentation. > > I don't expect many people building on top of isar-bootstrap, since > that is now pretty much the core internals of isar. People will build > their customization based on the output of isar-bootstrap, not isar- > bootstrap itself. > >> If you use deboostrap, it's ok to name tasks/recipes/etc >> respectively (do_debootstrap, ...). > > But in some cases it uses 'qemu-debootstrap' ;) (Nitpicky again, sorry) > > Now we have a general name for all the bootstrap mechanisms > 'do_bootstrap' and then if the need arise we can change 'do_bootstrap' > to some kind of 'virtual' task and have different implementations like > 'do_bootstrap_deb_binary', 'do_bootstrap_deb_source', ... For now we > have one, because we only support one bootstrapping mechanism or > distribution. That is the reverse direction of your suggestion, but > this way we have some kind of meta-tasks names already, and changing it > later can be done more transparent. > >> My position is simple: I read patches and if I've found something >> unclear, I'm trying to understand what was the reason going this way. > > That is a good position. And review generally really helps. > Here are some suggestions for me to improve the current situation. > > First, please stay measured. Arguing a long time about minor things > like variable names, while there might be many other places in the > actual code that could be improved is not. > > For instance I did like your suggestion to add '--homedir' to gpg, more > of that please. > > In general more suggestions with measurable improvements and less > emotions like apprehensions about possibly confusing someone. > > Second, please more 'trying to understand' instead of forcing your view > onto others via questions and or statements. Have you tried to do this also? 1. Isar is oriented to *Debian* community, yes, it's stand-alone project, but the main focus is Debian users and Debian infrastructure. This is not my IMO, this is the fact of this project. Everything we do should comply with Debian terms and philosophy. 2. Debian bootstrapping (building from zero) is well-known problem in Debian community, which still has no complete solution. AFAIK this work is done manually. A dedicated project was started - rebootstrap, but it's still under development. 3. I consider Debian bootstrapping as a possible Isar feature after adding cross-compilation. This bootstrapping has nothing common with calling debootstrap. These bullets are the source of my concerns. So I proposed to rename the things because I realized that such change takes just a few minutes. But due to this topic becomes more complicated, I'd like to close this discussion for now. > Only after you understood the view point of others you can really critique them. So ask questions > first > (best in a neutral, open and honest way), Reading some of your previous mails I wonder what you mean here. Especially reading this statement after your comments written above sounds like a kind of double standards. Stopped reading here. Alex > after you understood the developers view point and code you can make suggestions for > improvements while stating you objective reasons for it. > Also appreciating and praising good code occasionally will help a lot > spreading a positive and cooperative atmosphere in the project. :) > > Maybe take a look at [1] and especially [2]. > > I left some parts of your message out, but I hope that I have made > myself implicitly clear how I stand on the other parts of your message. > If something is still unclear, then please ask. > > Cheers, > Claudius > > [1] https://mtlynch.io/human-code-reviews-1/ > [2] https://mtlynch.io/human-code-reviews-2/ > > P.S. For me this kind of discussion now borders more on entertainment > than work and I don't think that this is a good sign. ;) > ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-16 5:49 ` Alexander Smirnov @ 2018-03-28 6:00 ` Jan Kiszka 0 siblings, 0 replies; 27+ messages in thread From: Jan Kiszka @ 2018-03-28 6:00 UTC (permalink / raw) To: Alexander Smirnov, Claudius Heine, Claudius Heine, isar-users [-- Attachment #1.1: Type: text/plain, Size: 8202 bytes --] On 2018-03-16 06:49, Alexander Smirnov wrote: > On 03/16/2018 01:07 AM, Claudius Heine wrote: >> Hi Alex, >> >> On Thu, 2018-03-15 at 11:58 +0300, Alexander Smirnov wrote: >>> @Claudius: >>> For me is very annoying when my question is ignored after explicit >>> re-posting. So let's be polite and keep conversations here free of >>> emotions. >> >> Read the following text without any emotions, apart from the paragraphs >> where I added the applicable ascii-emoji: >> >> First, I already answered your question indirectly here: >> >>>> If the distro-specific bootstrap mechanism builds its 'bootstraped >>>> root file system' from a bunch of binary packages or directly from >>>> source is implementation and distro specific. So the distinction >>>> only makes sense in the upstream project but not here, since we are >>>> just using what the upstream distributions provides for general >>>> consumption. >> >> Short: I don't agree with the presumptions in this question, therefor I >> cannot answer it sensible. >> >> And second I don't like responding to attempts at manipulation, like >> you did with this loaded question: >> >>> My question, why it's so important to use initially ambiguous term, >>> which may lead to potential confusing for Debian users? >> >> Maybe that is just a misunderstanding, but maybe we should go through >> it piece by piece and then look at it as a whole, so you can understand >> what I mean: >> >>> My question, why it's so important >> >> "why it's so important" reads accusatory, not neutral and together with >> the following parts of this sentence colors the rest of your question >> in a negative light. >> >>> to use initially ambiguous term, >> >> Here you are presuming my guilt by implying that I have chosen an >> 'ambiguous term' on purpose. I don't think 'isar-bootstrap' or >> 'bootstrap' is ambiguous in this context. Since generating an initial >> root file system for linux distributions is called 'bootstrapping the >> linux distribution' and that is exactly what is done here, so its not >> ambiguous for me. >> >> That debian has two methods of doing it also doesn't matter. Since that >> is an implementation detail and can be easily looked up. The end result >> is that we have some kind of minimal root file system that we can use >> to install our customization on top. >> >>> which may lead to potential confusing for Debian users? >> >> "may lead to potential confusing" Is it or is it not? That sounds like >> you are just making things up and spread FUD. It sound like "this code >> could potentially cause your computer to catch fire and explode". >> >> If I would have called something 'debootstrap' that formats your hard >> drive, then yes, I see that this causes confusion and even more. >> >> I don't see why a name like 'isar-bootstrap' or its containing task >> 'do_bootstrap' could confuse any Debian user that uses Isar. And even >> if that "may lead to potential confusing" them, it will clear up fast >> when they just take a look into the 'do_bootstrap' implementation. >> >> Also, since Isar is a completely different Project than Debian, people >> will have other expectations and might be positively surprised that >> something with 'bootstrap' in its name still does what they have >> expected it to do ;) >> >> So then look at the whole question again and try to answer it: >> >>> My question, why it's so important to use initially ambiguous term, >>> which may lead to potential confusing for Debian users? >> >> If I would nitpick then that isn't even a question, its a statement >> that this is your question. Also, how am I supposed to answer it? (Hold >> on, this us a rhetorical question and I don't expect an answer.) >> The answer is obvious. Its like questioning: >> >> Why do you think it is right to kill everyone? >> >> Maybe I should just reply to this questions and yours: >> >> That is not my intention. >> >> I'm not sure that this is what you wanted to hear, but your question >> can now be considered answered. >> >> Since that is done now, I follow with my responses to some other parts >> of your mail: >> >>> It's not only about internal class names, this series introduces >>> 'do_bootstrap' task that is visible to user and will be a part of >>> documentation. >> >> I don't expect many people building on top of isar-bootstrap, since >> that is now pretty much the core internals of isar. People will build >> their customization based on the output of isar-bootstrap, not isar- >> bootstrap itself. >> >>> If you use deboostrap, it's ok to name tasks/recipes/etc >>> respectively (do_debootstrap, ...). >> >> But in some cases it uses 'qemu-debootstrap' ;) (Nitpicky again, sorry) >> >> Now we have a general name for all the bootstrap mechanisms >> 'do_bootstrap' and then if the need arise we can change 'do_bootstrap' >> to some kind of 'virtual' task and have different implementations like >> 'do_bootstrap_deb_binary', 'do_bootstrap_deb_source', ... For now we >> have one, because we only support one bootstrapping mechanism or >> distribution. That is the reverse direction of your suggestion, but >> this way we have some kind of meta-tasks names already, and changing it >> later can be done more transparent. >> >>> My position is simple: I read patches and if I've found something >>> unclear, I'm trying to understand what was the reason going this way. >> >> That is a good position. And review generally really helps. >> Here are some suggestions for me to improve the current situation. >> >> First, please stay measured. Arguing a long time about minor things >> like variable names, while there might be many other places in the >> actual code that could be improved is not. >> >> For instance I did like your suggestion to add '--homedir' to gpg, more >> of that please. >> >> In general more suggestions with measurable improvements and less >> emotions like apprehensions about possibly confusing someone. >> >> Second, please more 'trying to understand' instead of forcing your >> view > onto others via questions and or statements. > > Have you tried to do this also? > > 1. Isar is oriented to *Debian* community, yes, it's stand-alone > project, but the main focus is Debian users and Debian infrastructure. > This is not my IMO, this is the fact of this project. Everything we do > should comply with Debian terms and philosophy. > > 2. Debian bootstrapping (building from zero) is well-known problem in > Debian community, which still has no complete solution. AFAIK this work > is done manually. A dedicated project was started - rebootstrap, but > it's still under development. > > 3. I consider Debian bootstrapping as a possible Isar feature after > adding cross-compilation. This bootstrapping has nothing common with > calling debootstrap. > > These bullets are the source of my concerns. So I proposed to rename the > things because I realized that such change takes just a few minutes. But > due to this topic becomes more complicated, I'd like to close this > discussion for now. > >> Only after you understood the view point of others you can really >> critique them. So ask questions >> first >> (best in a neutral, open and honest way), > > Reading some of your previous mails I wonder what you mean here. > Especially reading this statement after your comments written above > sounds like a kind of double standards. Stopped reading here. > I hope this discussion is not blocking the progress of this important feature. Just the make it clear that naming is be far not as clear as it was put here: "Package: debootstrap Bootstrap a basic Debian system" "Package: cdebootstrap Bootstrap a Debian system" "Package: multistrap multiple repository bootstrap based on apt" So let's call this thing by its commonly used Debian name, bootstrap, and move on to more important topics. I'd really like to see the debootstrap migration finalized and merged, specifically as I need some cross-buildchroot on top of it. Thanks, Jan [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext 2018-03-08 7:53 ` Claudius Heine 2018-03-14 8:56 ` Alexander Smirnov @ 2018-03-14 14:04 ` Alexander Smirnov 2018-03-14 14:26 ` Claudius Heine 2 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-14 14:04 UTC (permalink / raw) To: claudius.heine.ext, isar-users; +Cc: Claudius Heine On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: > From: Claudius Heine <ch@denx.de> > > Since multistrap is deprecated for some years, it is required to change > to debootstrap. > > This patch introduces the 'isar-bootstrap' recipe that implement the > creation of a minimal base root file system using debootstrap. > > Signed-off-by: Claudius Heine <ch@denx.de> > --- > meta-isar/conf/distro/debian-jessie.conf | 15 +- > meta-isar/conf/distro/debian-jessie.list | 3 + > meta-isar/conf/distro/debian-stretch.conf | 11 +- > meta-isar/conf/distro/debian-stretch.list | 3 + > meta-isar/conf/distro/debian-wheezy.conf | 11 +- > meta-isar/conf/distro/debian-wheezy.list | 3 + > meta-isar/conf/distro/raspbian-jessie.conf | 10 +- > meta-isar/conf/distro/raspbian-jessie.list | 1 + > .../isar-bootstrap/files/isar-apt.conf | 2 + > meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 +++++++++++++++++++++ > 10 files changed, 247 insertions(+), 17 deletions(-) > create mode 100644 meta-isar/conf/distro/debian-jessie.list > create mode 100644 meta-isar/conf/distro/debian-stretch.list > create mode 100644 meta-isar/conf/distro/debian-wheezy.list > create mode 100644 meta-isar/conf/distro/raspbian-jessie.list > create mode 100644 meta/recipes-core/isar-bootstrap/files/isar-apt.conf > create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > > diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta-isar/conf/distro/debian-jessie.conf > index 3e62b14..df27fd7 100644 > --- a/meta-isar/conf/distro/debian-jessie.conf > +++ b/meta-isar/conf/distro/debian-jessie.conf > @@ -1,16 +1,19 @@ > -# Debian jessie multistrap configuration > +# Debian jessie distribution configuration > # > # This software is a part of ISAR. > # Copyright (C) 2015-2016 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/debian-jessie.list" > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > +DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 armmp \ > + armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 octeon \ > + orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k-ip32 \ > + s390x sb1-bcm91250a versatile" > + > +# No longer needed: > DISTRO_SUITE ?= "jessie" > DISTRO_COMPONENTS ?= "main contrib non-free" > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 armmp \ > - armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 octeon \ > - orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k-ip32 \ > - s390x sb1-bcm91250a versatile" > diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta-isar/conf/distro/debian-jessie.list > new file mode 100644 > index 0000000..be46a57 > --- /dev/null > +++ b/meta-isar/conf/distro/debian-jessie.list > @@ -0,0 +1,3 @@ > +deb http://ftp.de.debian.org/debian jessie main contrib non-free > +deb http://ftp.de.debian.org/debian jessie-updates main contrib non-free > +deb http://security.debian.org jessie/updates main contrib non-free > diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta-isar/conf/distro/debian-stretch.conf > index 5a85d52..13c4f94 100644 > --- a/meta-isar/conf/distro/debian-stretch.conf > +++ b/meta-isar/conf/distro/debian-stretch.conf > @@ -1,13 +1,16 @@ > # This software is a part of ISAR. > # Copyright (C) 2017 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/debian-stretch.list" > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > +DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 armmp \ > + armmp-lpae kirkwood loongson-3 marvell octeon orion5x powerpc64le \ > + rt-686-pae rt-amd64 s390x" > + > +# No longer needed: > DISTRO_SUITE ?= "stretch" > DISTRO_COMPONENTS ?= "main contrib non-free" > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 armmp \ > - armmp-lpae kirkwood loongson-3 marvell octeon orion5x powerpc64le \ > - rt-686-pae rt-amd64 s390x" > diff --git a/meta-isar/conf/distro/debian-stretch.list b/meta-isar/conf/distro/debian-stretch.list > new file mode 100644 > index 0000000..bfa5cf9 > --- /dev/null > +++ b/meta-isar/conf/distro/debian-stretch.list > @@ -0,0 +1,3 @@ > +deb http://ftp.de.debian.org/debian stretch main contrib non-free > +deb http://ftp.de.debian.org/debian stretch-updates main contrib non-free > +deb http://security.debian.org stretch/updates main contrib non-free > diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta-isar/conf/distro/debian-wheezy.conf > index f5ab6eb..2108f9b 100644 > --- a/meta-isar/conf/distro/debian-wheezy.conf > +++ b/meta-isar/conf/distro/debian-wheezy.conf > @@ -1,14 +1,17 @@ > -# Debian wheezy multistrap configuration > +# Debian wheezy distribution configuration > # > # This software is a part of ISAR. > # Copyright (C) 2015-2016 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list" > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > +DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile vexpress \ > + iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" > + > +# No longer needed: > DISTRO_SUITE ?= "wheezy" > DISTRO_COMPONENTS ?= "main contrib non-free" > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile vexpress \ > - iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" > diff --git a/meta-isar/conf/distro/debian-wheezy.list b/meta-isar/conf/distro/debian-wheezy.list > new file mode 100644 > index 0000000..0bfa98e > --- /dev/null > +++ b/meta-isar/conf/distro/debian-wheezy.list > @@ -0,0 +1,3 @@ > +deb http://ftp.de.debian.org/debian wheezy main contrib non-free > +deb http://ftp.de.debian.org/debian wheezy-updates main contrib non-free > +deb http://security.debian.org wheezy/updates main contrib non-free > diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-isar/conf/distro/raspbian-jessie.conf > index c2a867d..57d0e7d 100644 > --- a/meta-isar/conf/distro/raspbian-jessie.conf > +++ b/meta-isar/conf/distro/raspbian-jessie.conf > @@ -1,13 +1,17 @@ > -# Raspbian stable multistrap configuration > +# Raspbian stable distribution configuration > # > # This software is a part of ISAR. > # Copyright (C) 2015-2016 ilbers GmbH > > +DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list" > +DISTRO_APT_KEYS += "https://archive.raspbian.org/raspbian.public.key;sha256sum=ca59cd4f2bcbc3a1d41ba6815a02a8dc5c175467a59bd87edeac458f4a5345de" > +DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh" > +DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" > + > +# No longer needed: > DISTRO_SUITE ?= "jessie" > DISTRO_COMPONENTS ?= "main contrib non-free firmware" > DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian" > DISTRO_APT_SOURCE_SEC ?= "" > -DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh" > DISTRO_MULTICONF_BOOTSTRAP ?= "base" > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > -DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" > diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta-isar/conf/distro/raspbian-jessie.list > new file mode 100644 > index 0000000..792f9c6 > --- /dev/null > +++ b/meta-isar/conf/distro/raspbian-jessie.list > @@ -0,0 +1 @@ > +deb http://archive.raspbian.org/raspbian jessie main contrib non-free firmware > diff --git a/meta/recipes-core/isar-bootstrap/files/isar-apt.conf b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > new file mode 100644 > index 0000000..a62feb1 > --- /dev/null > +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > @@ -0,0 +1,2 @@ > +APT::Install-Recommends "0"; > +APT::Install-Suggests "0"; > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > new file mode 100644 > index 0000000..2ddefa4 > --- /dev/null > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > @@ -0,0 +1,205 @@ > +# Minimal debian root file system > +# > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2018 > +# > +# SPDX-License-Identifier: MIT > + > +Description = "Minimal debian root file system" > + > +LICENSE = "gpl-2.0" > +LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" > +FILESPATH_prepend := "${THISDIR}/files:" > +SRC_URI = "file://isar-apt.conf" > +PV = "1.0" > + > +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > +DEBOOTSTRAP ?= "" > +ROOTFSDIR = "${WORKDIR}/rootfs" > +APTPREFS = "${WORKDIR}/apt-preferences" > +APTSRCS = "${WORKDIR}/apt-sources" > +APTKEYFILES = "" > +APTKEYRING = "${WORKDIR}/apt-keyring.gpg" > +DEBOOTSTRAP_KEYRING = "" > + > +python () { > + from urllib.parse import urlparse > + > + debootstrap = d.getVar("DEBOOTSTRAP", True) > + if not debootstrap: > + target = d.getVar("DISTRO_ARCH", True) > + machine = os.uname()[4] > + m = { > + "x86_64": ["i386", "amd64"], > + "x86": ["i386"], > + } > + if machine not in m or target not in m[machine]: > + debootstrap = "qemu-debootstrap" > + else: > + debootstrap = "debootstrap" > + d.setVar("DEBOOTSTRAP", debootstrap) > + > + distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False) > + if distro_apt_keys: > + d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}") > + for key in distro_apt_keys.split(): > + url = urlparse(key) > + filename = os.path.basename(url.path) > + d.appendVar("SRC_URI", " %s" % key) > + d.appendVar("APTKEYFILES", " %s" % filename) > +} > + > +def aggregate_files(d, file_list, file_out): > + import shutil > + > + with open(file_out, "wb") as out_fd: > + for entry in file_list: > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "rb") as in_fd: > + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) > + out_fd.write("\n".encode()) > + > +def parse_aptsources_list_line(source_list_line): > + import re > + > + s = source_list_line.strip() > + > + if s.startswith("#"): > + return None > + > + type, s = re.split("\s+", s, maxsplit=1) > + if type not in ["deb", "deb-src"]: > + return None > + > + options = "" > + options_match = re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) > + if options_match: > + options = options_match.group(0).strip() > + s = s[options_match.end():] > + > + source, s = re.split("\s+", s, maxsplit=1) > + > + suite, s = re.split("\s+", s, maxsplit=1) > + > + components = " ".join(s.split()) > + > + return type, options, source, suite, components > + > +def get_distro_primary_source_entry(d): > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > + for entry in apt_sources_list: > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "r") as in_fd: > + for line in in_fd: > + parsed = parse_aptsources_list_line(line) > + if parsed: > + type, _, source, suite, components = parsed > + if type == "deb": > + return source, suite, components > + return "", "", "" > + > +def get_distro_source(d): > + return get_distro_primary_source_entry(d)[0] > + > +def get_distro_suite(d): > + return get_distro_primary_source_entry(d)[1] > + > +def get_distro_components_argument(d): > + components = get_distro_primary_source_entry(d)[2] > + if components and components.strip(): > + return "--components=%s" % ",".join(components.split()) > + else: > + return "" > + > +do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_generate_keyring[dirs] = "${WORKDIR}" > +do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > +do_generate_keyring() { > + if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then > + for keyfile in ${@d.getVar("APTKEYFILES", True)}; do > + gpg --no-default-keyring --keyring "${APTKEYRING}" --import "$keyfile" This code touches my private "~/.gnupg" folder's content, what is not desirable behavior. Isar should touch files *only* in "build/tmp" folder. gpg already supports changing of home directory using --homedir option. So I think we could create dedicated folder "${TMPDIR}/gpg" and store keys here. At the moment I'm testing if debootstrap supports custom gpg storage. Will come back with the results. Alex > + done > + fi > +} > +addtask generate_keyring before do_build after do_unpack > + > +do_apt_config_prepare[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_apt_config_prepare[vardeps] += "\ > + APTPREFS \ > + DISTRO_APT_PREFERENCES \ > + DEBDISTRONAME \ > + APTSRCS \ > + DISTRO_APT_SOURCES \ > + " > +python do_apt_config_prepare() { > + apt_preferences_out = d.getVar("APTPREFS", True) > + apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", True) or "" > + ).split() > + aggregate_files(d, apt_preferences_list, apt_preferences_out) > + > + apt_sources_out = d.getVar("APTSRCS", True) > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > + > + aggregate_files(d, apt_sources_list, apt_sources_out) > +} > +addtask apt_config_prepare before do_build after do_generate_keyring > + > +do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" > +do_bootstrap() { > + if [ -e "${ROOTFSDIR}" ]; then > + sudo umount -l "${ROOTFSDIR}/dev" || true > + sudo umount -l "${ROOTFSDIR}/proc" || true > + sudo rm -rf "${ROOTFSDIR}" > + fi > + E="${@bb.utils.export_proxies(d)}" > + sudo -E "${DEBOOTSTRAP}" --verbose \ > + --variant minbase \ > + --arch "${DISTRO_ARCH}" \ > + ${@get_distro_components_argument(d)} \ > + ${DEBOOTSTRAP_KEYRING} \ > + "${@get_distro_suite(d)}" \ > + "${ROOTFSDIR}" \ > + "${@get_distro_source(d)}" > +} > +addtask bootstrap before do_build after do_apt_config_prepare > + > +do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_apt_config_install() { > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" > + sudo install -v -m644 "${APTPREFS}" \ > + "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" > + sudo install -v -m644 "${APTSRCS}" \ > + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" > + sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list" > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" > + sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ > + "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" > +} > +addtask apt_config_install before do_build after do_bootstrap > + > +do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_apt_update() { > + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev > + sudo mount -t proc none ${ROOTFSDIR}/proc > + > + E="${@bb.utils.export_proxies(d)}" > + export DEBIAN_FRONTEND=noninteractive > + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ > + -o Debug::pkgProblemResolver=yes > +} > +addtask apt_update before do_build after do_apt_config_install > + > +do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}" > +do_deploy() { > + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" > +} > +addtask deploy before do_build after do_apt_update > + > +CLEANFUNCS = "clean_deploy" > +clean_deploy() { > + rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" > +} > -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 14:04 ` Alexander Smirnov @ 2018-03-14 14:26 ` Claudius Heine 2018-03-14 14:35 ` Alexander Smirnov 0 siblings, 1 reply; 27+ messages in thread From: Claudius Heine @ 2018-03-14 14:26 UTC (permalink / raw) To: Alexander Smirnov, claudius.heine.ext, isar-users [-- Attachment #1: Type: text/plain, Size: 20612 bytes --] Hi Alex, On Wed, 2018-03-14 at 17:04 +0300, Alexander Smirnov wrote: > > On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: > > From: Claudius Heine <ch@denx.de> > > > > Since multistrap is deprecated for some years, it is required to > > change > > to debootstrap. > > > > This patch introduces the 'isar-bootstrap' recipe that implement > > the > > creation of a minimal base root file system using debootstrap. > > > > Signed-off-by: Claudius Heine <ch@denx.de> > > --- > > meta-isar/conf/distro/debian-jessie.conf | 15 +- > > meta-isar/conf/distro/debian-jessie.list | 3 + > > meta-isar/conf/distro/debian-stretch.conf | 11 +- > > meta-isar/conf/distro/debian-stretch.list | 3 + > > meta-isar/conf/distro/debian-wheezy.conf | 11 +- > > meta-isar/conf/distro/debian-wheezy.list | 3 + > > meta-isar/conf/distro/raspbian-jessie.conf | 10 +- > > meta-isar/conf/distro/raspbian-jessie.list | 1 + > > .../isar-bootstrap/files/isar-apt.conf | 2 + > > meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 > > +++++++++++++++++++++ > > 10 files changed, 247 insertions(+), 17 deletions(-) > > create mode 100644 meta-isar/conf/distro/debian-jessie.list > > create mode 100644 meta-isar/conf/distro/debian-stretch.list > > create mode 100644 meta-isar/conf/distro/debian-wheezy.list > > create mode 100644 meta-isar/conf/distro/raspbian-jessie.list > > create mode 100644 meta/recipes-core/isar-bootstrap/files/isar- > > apt.conf > > create mode 100644 meta/recipes-core/isar-bootstrap/isar- > > bootstrap.bb > > > > diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta- > > isar/conf/distro/debian-jessie.conf > > index 3e62b14..df27fd7 100644 > > --- a/meta-isar/conf/distro/debian-jessie.conf > > +++ b/meta-isar/conf/distro/debian-jessie.conf > > @@ -1,16 +1,19 @@ > > -# Debian jessie multistrap configuration > > +# Debian jessie distribution configuration > > # > > # This software is a part of ISAR. > > # Copyright (C) 2015-2016 ilbers GmbH > > > > +DISTRO_APT_SOURCES += "conf/distro/debian-jessie.list" > > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > > +DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 > > armmp \ > > + armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 > > octeon \ > > + orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 > > r5k-ip32 \ > > + s390x sb1-bcm91250a versatile" > > + > > +# No longer needed: > > DISTRO_SUITE ?= "jessie" > > DISTRO_COMPONENTS ?= "main contrib non-free" > > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-secur > > ity" > > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > > -DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 > > armmp \ > > - armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 > > octeon \ > > - orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 > > r5k-ip32 \ > > - s390x sb1-bcm91250a versatile" > > diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta- > > isar/conf/distro/debian-jessie.list > > new file mode 100644 > > index 0000000..be46a57 > > --- /dev/null > > +++ b/meta-isar/conf/distro/debian-jessie.list > > @@ -0,0 +1,3 @@ > > +deb http://ftp.de.debian.org/debian class="Apple-tab-span" > > style="white-space:pre"> jessie main contrib non-free > > +deb http://ftp.de.debian.org/debian jessie-updates m > > ain contrib non-free > > +deb http://security.debian.org class="Apple-tab-span" > > style="white-space:pre"> jessie/updates main contrib non- > > free > > diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta- > > isar/conf/distro/debian-stretch.conf > > index 5a85d52..13c4f94 100644 > > --- a/meta-isar/conf/distro/debian-stretch.conf > > +++ b/meta-isar/conf/distro/debian-stretch.conf > > @@ -1,13 +1,16 @@ > > # This software is a part of ISAR. > > # Copyright (C) 2017 ilbers GmbH > > > > +DISTRO_APT_SOURCES += "conf/distro/debian-stretch.list" > > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > > +DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 > > armmp \ > > + armmp-lpae kirkwood loongson-3 marvell octeon orion5x > > powerpc64le \ > > + rt-686-pae rt-amd64 s390x" > > + > > +# No longer needed: > > DISTRO_SUITE ?= "stretch" > > DISTRO_COMPONENTS ?= "main contrib non-free" > > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-secur > > ity" > > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > > -DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 > > armmp \ > > - armmp-lpae kirkwood loongson-3 marvell octeon orion5x > > powerpc64le \ > > - rt-686-pae rt-amd64 s390x" > > diff --git a/meta-isar/conf/distro/debian-stretch.list b/meta- > > isar/conf/distro/debian-stretch.list > > new file mode 100644 > > index 0000000..bfa5cf9 > > --- /dev/null > > +++ b/meta-isar/conf/distro/debian-stretch.list > > @@ -0,0 +1,3 @@ > > +deb http://ftp.de.debian.org/debian stretch main > > contrib non-free > > +deb http://ftp.de.debian.org/debian stretch-updates > > main contrib non-free > > +deb http://security.debian.org class="Apple-tab-span" > > style="white-space:pre"> stretch/updates main contrib non- > > free > > diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta- > > isar/conf/distro/debian-wheezy.conf > > index f5ab6eb..2108f9b 100644 > > --- a/meta-isar/conf/distro/debian-wheezy.conf > > +++ b/meta-isar/conf/distro/debian-wheezy.conf > > @@ -1,14 +1,17 @@ > > -# Debian wheezy multistrap configuration > > +# Debian wheezy distribution configuration > > # > > # This software is a part of ISAR. > > # Copyright (C) 2015-2016 ilbers GmbH > > > > +DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list" > > +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > > +DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile > > vexpress \ > > + iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" > > + > > +# No longer needed: > > DISTRO_SUITE ?= "wheezy" > > DISTRO_COMPONENTS ?= "main contrib non-free" > > DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" > > DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-secur > > ity" > > -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" > > DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" > > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > > -DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile > > vexpress \ > > - iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" > > diff --git a/meta-isar/conf/distro/debian-wheezy.list b/meta- > > isar/conf/distro/debian-wheezy.list > > new file mode 100644 > > index 0000000..0bfa98e > > --- /dev/null > > +++ b/meta-isar/conf/distro/debian-wheezy.list > > @@ -0,0 +1,3 @@ > > +deb http://ftp.de.debian.org/debian wheezy main > > contrib non-free > > +deb http://ftp.de.debian.org/debian wheezy-updates m > > ain contrib non-free > > +deb http://security.debian.org class="Apple-tab-span" > > style="white-space:pre"> wheezy/updates main contrib non- > > free > > diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta- > > isar/conf/distro/raspbian-jessie.conf > > index c2a867d..57d0e7d 100644 > > --- a/meta-isar/conf/distro/raspbian-jessie.conf > > +++ b/meta-isar/conf/distro/raspbian-jessie.conf > > @@ -1,13 +1,17 @@ > > -# Raspbian stable multistrap configuration > > +# Raspbian stable distribution configuration > > # > > # This software is a part of ISAR. > > # Copyright (C) 2015-2016 ilbers GmbH > > > > +DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list" > > +DISTRO_APT_KEYS += "https://archive.raspbian.org/raspbian.public.k > > ey;sha256sum=ca59cd4f2bcbc3a1d41ba6815a02a8dc5c175467a59bd87edeac45 > > 8f4a5345de" > > +DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh" > > +DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" > > + > > +# No longer needed: > > DISTRO_SUITE ?= "jessie" > > DISTRO_COMPONENTS ?= "main contrib non-free firmware" > > DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian" > > DISTRO_APT_SOURCE_SEC ?= "" > > -DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh" > > DISTRO_MULTICONF_BOOTSTRAP ?= "base" > > DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" > > -DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" > > diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta- > > isar/conf/distro/raspbian-jessie.list > > new file mode 100644 > > index 0000000..792f9c6 > > --- /dev/null > > +++ b/meta-isar/conf/distro/raspbian-jessie.list > > @@ -0,0 +1 @@ > > +deb http://archive.raspbian.org/raspbian class="Apple-tab- > > span" style="white-space:pre"> jessie main contrib non-free > > firmware > > diff --git a/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > > b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > > new file mode 100644 > > index 0000000..a62feb1 > > --- /dev/null > > +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf > > @@ -0,0 +1,2 @@ > > +APT::Install-Recommends "0"; > > +APT::Install-Suggests "0"; > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > > new file mode 100644 > > index 0000000..2ddefa4 > > --- /dev/null > > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > > @@ -0,0 +1,205 @@ > > +# Minimal debian root file system > > +# > > +# This software is a part of ISAR. > > +# Copyright (c) Siemens AG, 2018 > > +# > > +# SPDX-License-Identifier: MIT > > + > > +Description = "Minimal debian root file system" > > + > > +LICENSE = "gpl-2.0" > > +LIC_FILES_CHKSUM = > > "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa9544 > > 99f7abaabaa882bbe" > > +FILESPATH_prepend := "${THISDIR}/files:" > > +SRC_URI = "file://isar-apt.conf" > > +PV = "1.0" > > + > > +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > +DEBOOTSTRAP ?= "" > > +ROOTFSDIR = "${WORKDIR}/rootfs" > > +APTPREFS = "${WORKDIR}/apt-preferences" > > +APTSRCS = "${WORKDIR}/apt-sources" > > +APTKEYFILES = "" > > +APTKEYRING = "${WORKDIR}/apt-keyring.gpg" > > +DEBOOTSTRAP_KEYRING = "" > > + > > +python () { > > + from urllib.parse import urlparse > > + > > + debootstrap = d.getVar("DEBOOTSTRAP", True) > > + if not debootstrap: > > + target = d.getVar("DISTRO_ARCH", True) > > + machine = os.uname()[4] > > + m = { > > + "x86_64": ["i386", "amd64"], > > + "x86": ["i386"], > > + } > > + if machine not in m or target not in m[machine]: > > + debootstrap = "qemu-debootstrap" > > + else: > > + debootstrap = "debootstrap" > > + d.setVar("DEBOOTSTRAP", debootstrap) > > + > > + distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False) > > + if distro_apt_keys: > > + d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}") > > + for key in distro_apt_keys.split(): > > + url = urlparse(key) > > + filename = os.path.basename(url.path) > > + d.appendVar("SRC_URI", " %s" % key) > > + d.appendVar("APTKEYFILES", " %s" % filename) > > +} > > + > > +def aggregate_files(d, file_list, file_out): > > + import shutil > > + > > + with open(file_out, "wb") as out_fd: > > + for entry in file_list: > > + entry_real = bb.parse.resolve_file(entry, d) > > + with open(entry_real, "rb") as in_fd: > > + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) > > + out_fd.write("\n".encode()) > > + > > +def parse_aptsources_list_line(source_list_line): > > + import re > > + > > + s = source_list_line.strip() > > + > > + if s.startswith("#"): > > + return None > > + > > + type, s = re.split("\s+", s, maxsplit=1) > > + if type not in ["deb", "deb-src"]: > > + return None > > + > > + options = "" > > + options_match = > > re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) > > + if options_match: > > + options = options_match.group(0).strip() > > + s = s[options_match.end():] > > + > > + source, s = re.split("\s+", s, maxsplit=1) > > + > > + suite, s = re.split("\s+", s, maxsplit=1) > > + > > + components = " ".join(s.split()) > > + > > + return type, options, source, suite, components > > + > > +def get_distro_primary_source_entry(d): > > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or > > "").split() > > + for entry in apt_sources_list: > > + entry_real = bb.parse.resolve_file(entry, d) > > + with open(entry_real, "r") as in_fd: > > + for line in in_fd: > > + parsed = parse_aptsources_list_line(line) > > + if parsed: > > + type, _, source, suite, components = parsed > > + if type == "deb": > > + return source, suite, components > > + return "", "", "" > > + > > +def get_distro_source(d): > > + return get_distro_primary_source_entry(d)[0] > > + > > +def get_distro_suite(d): > > + return get_distro_primary_source_entry(d)[1] > > + > > +def get_distro_components_argument(d): > > + components = get_distro_primary_source_entry(d)[2] > > + if components and components.strip(): > > + return "--components=%s" % ",".join(components.split()) > > + else: > > + return "" > > + > > +do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > +do_generate_keyring[dirs] = "${WORKDIR}" > > +do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > > +do_generate_keyring() { > > + if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then > > + for keyfile in ${@d.getVar("APTKEYFILES", True)}; do > > + gpg --no-default-keyring --keyring "${APTKEYRING}" -- > > import "$keyfile" > > This code touches my private "~/.gnupg" folder's content, what is > not > desirable behavior. Isar should touch files *only* in "build/tmp" > folder. Are you sure? $ find .gnupg -type f | sort | xargs b2sum > gpgsums.a $ gpg --no-default-keyring --keyring ./test1.db --import raspbian.public.key gpg: enabled debug flags: memstat gpg: keybox './test1.db' created gpg: key 9165938D90FDDD2E: public key "Mike Thompson (Raspberry Pi Debian armhf ARMv6+VFP) <mpthompson@gmail.com>" imported gpg: Total number processed: 1 gpg: imported: 1 gpg: keydb: handles=3 locks=2 parse=2 get=2 gpg: build=1 update=0 insert=1 delete=0 gpg: reset=1 found=2 not=2 cache=0 not=0 gpg: kid_not_found_cache: count=0 peak=0 flushes=0 gpg: sig_cache: total=6 cached=4 good=4 bad=0 gpg: random usage: poolsize=600 mixed=0 polls=0/0 added=0/0 outmix=0 getlvl1=0/0 getlvl2=0/0 gpg: rndjent stat: collector=0x0000000000000000 calls=0 bytes=0 gpg: secmem usage: 0/32768 bytes in 0 blocks $ find .gnupg -type f | sort | xargs b2sum > gpgsums.b $ diff -u gpgsums.a gpgsums.b [no output] I could add the '--homedir' parameter, but normally it shouldn't use it because I manually select a different keyring. Claudius > > gpg already supports changing of home directory using --homedir > option. > So I think we could create dedicated folder "${TMPDIR}/gpg" and > store > keys here. > > At the moment I'm testing if debootstrap supports custom gpg > storage. > Will come back with the results. > > Alex > > > + done > > + fi > > +} > > +addtask generate_keyring before do_build after do_unpack > > + > > +do_apt_config_prepare[stamp-extra-info] = "${DISTRO}- > > ${DISTRO_ARCH}" > > +do_apt_config_prepare[vardeps] += "\ > > + APTPREFS \ > > + DISTRO_APT_PREFERENCES \ > > + DEBDISTRONAME \ > > + APTSRCS \ > > + DISTRO_APT_SOURCES \ > > + " > > +python do_apt_config_prepare() { > > + apt_preferences_out = d.getVar("APTPREFS", True) > > + apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", > > True) or "" > > + ).split() > > + aggregate_files(d, apt_preferences_list, apt_preferences_out) > > + > > + apt_sources_out = d.getVar("APTSRCS", True) > > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or > > "").split() > > + > > + aggregate_files(d, apt_sources_list, apt_sources_out) > > +} > > +addtask apt_config_prepare before do_build after > > do_generate_keyring > > + > > +do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > +do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" > > +do_bootstrap() { > > + if [ -e "${ROOTFSDIR}" ]; then > > + sudo umount -l "${ROOTFSDIR}/dev" || true > > + sudo umount -l "${ROOTFSDIR}/proc" || true > > + sudo rm -rf "${ROOTFSDIR}" > > + fi > > + E="${@bb.utils.export_proxies(d)}" > > + sudo -E "${DEBOOTSTRAP}" --verbose \ > > + --variant minbase \ > > + --arch "${DISTRO_ARCH}" \ > > + ${@get_distro_components_argument(d)} > > \ > > + ${DEBOOTSTRAP_KEYRING} \ > > + "${@get_distro_suite(d)}" \ > > + "${ROOTFSDIR}" \ > > + "${@get_distro_source(d)}" > > +} > > +addtask bootstrap before do_build after do_apt_config_prepare > > + > > +do_apt_config_install[stamp-extra-info] = "${DISTRO}- > > ${DISTRO_ARCH}" > > +do_apt_config_install() { > > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" > > + sudo install -v -m644 "${APTPREFS}" \ > > + "${ROOTFSDIR}/etc/apt/preferences.d/boot > > strap" > > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" > > + sudo install -v -m644 "${APTSRCS}" \ > > + "${ROOTFSDIR}/etc/apt/sources.list.d/boo > > tstrap.list" > > + sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list" > > + sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" > > + sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ > > + "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar. > > conf" > > +} > > +addtask apt_config_install before do_build after do_bootstrap > > + > > +do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > +do_apt_update() { > > + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs > > ${ROOTFSDIR}/dev > > + sudo mount -t proc none ${ROOTFSDIR}/proc > > + > > + E="${@bb.utils.export_proxies(d)}" > > + export DEBIAN_FRONTEND=noninteractive > > + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > > + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y > > \ > > + -o > > Debug::pkgProblemResolver=yes > > +} > > +addtask apt_update before do_build after do_apt_config_install > > + > > +do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > +do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}" > > +do_deploy() { > > + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}- > > ${DISTRO_ARCH}" > > +} > > +addtask deploy before do_build after do_apt_update > > + > > +CLEANFUNCS = "clean_deploy" > > +clean_deploy() { > > + rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" > > +} > > > > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153 Keyserver: hkp://pool.sks-keyservers.net [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 14:26 ` Claudius Heine @ 2018-03-14 14:35 ` Alexander Smirnov 2018-03-14 16:13 ` Claudius Heine 0 siblings, 1 reply; 27+ messages in thread From: Alexander Smirnov @ 2018-03-14 14:35 UTC (permalink / raw) To: Claudius Heine, claudius.heine.ext, isar-users On 03/14/2018 05:26 PM, Claudius Heine wrote: > Hi Alex, > > On Wed, 2018-03-14 at 17:04 +0300, Alexander Smirnov wrote: >> >> On 03/07/2018 07:44 PM, claudius.heine.ext@siemens.com wrote: >>> From: Claudius Heine <ch@denx.de> >>> >>> Since multistrap is deprecated for some years, it is required to >>> change >>> to debootstrap. >>> >>> This patch introduces the 'isar-bootstrap' recipe that implement >>> the >>> creation of a minimal base root file system using debootstrap. >>> >>> Signed-off-by: Claudius Heine <ch@denx.de> >>> --- >>> meta-isar/conf/distro/debian-jessie.conf | 15 +- >>> meta-isar/conf/distro/debian-jessie.list | 3 + >>> meta-isar/conf/distro/debian-stretch.conf | 11 +- >>> meta-isar/conf/distro/debian-stretch.list | 3 + >>> meta-isar/conf/distro/debian-wheezy.conf | 11 +- >>> meta-isar/conf/distro/debian-wheezy.list | 3 + >>> meta-isar/conf/distro/raspbian-jessie.conf | 10 +- >>> meta-isar/conf/distro/raspbian-jessie.list | 1 + >>> .../isar-bootstrap/files/isar-apt.conf | 2 + >>> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 >>> +++++++++++++++++++++ >>> 10 files changed, 247 insertions(+), 17 deletions(-) >>> create mode 100644 meta-isar/conf/distro/debian-jessie.list >>> create mode 100644 meta-isar/conf/distro/debian-stretch.list >>> create mode 100644 meta-isar/conf/distro/debian-wheezy.list >>> create mode 100644 meta-isar/conf/distro/raspbian-jessie.list >>> create mode 100644 meta/recipes-core/isar-bootstrap/files/isar- >>> apt.conf >>> create mode 100644 meta/recipes-core/isar-bootstrap/isar- >>> bootstrap.bb >>> >>> diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta- >>> isar/conf/distro/debian-jessie.conf >>> index 3e62b14..df27fd7 100644 >>> --- a/meta-isar/conf/distro/debian-jessie.conf >>> +++ b/meta-isar/conf/distro/debian-jessie.conf >>> @@ -1,16 +1,19 @@ >>> -# Debian jessie multistrap configuration >>> +# Debian jessie distribution configuration >>> # >>> # This software is a part of ISAR. >>> # Copyright (C) 2015-2016 ilbers GmbH >>> >>> +DISTRO_APT_SOURCES += "conf/distro/debian-jessie.list" >>> +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" >>> +DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 >>> armmp \ >>> + armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 >>> octeon \ >>> + orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 >>> r5k-ip32 \ >>> + s390x sb1-bcm91250a versatile" >>> + >>> +# No longer needed: >>> DISTRO_SUITE ?= "jessie" >>> DISTRO_COMPONENTS ?= "main contrib non-free" >>> DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" >>> DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-secur >>> ity" >>> -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" >>> DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" >>> DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" >>> -DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 >>> armmp \ >>> - armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 >>> octeon \ >>> - orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 >>> r5k-ip32 \ >>> - s390x sb1-bcm91250a versatile" >>> diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta- >>> isar/conf/distro/debian-jessie.list >>> new file mode 100644 >>> index 0000000..be46a57 >>> --- /dev/null >>> +++ b/meta-isar/conf/distro/debian-jessie.list >>> @@ -0,0 +1,3 @@ >>> +deb http://ftp.de.debian.org/debian class="Apple-tab-span" >>> style="white-space:pre"> jessie main contrib non-free >>> +deb http://ftp.de.debian.org/debian jessie-updates m >>> ain contrib non-free >>> +deb http://security.debian.org class="Apple-tab-span" >>> style="white-space:pre"> jessie/updates main contrib non- >>> free >>> diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta- >>> isar/conf/distro/debian-stretch.conf >>> index 5a85d52..13c4f94 100644 >>> --- a/meta-isar/conf/distro/debian-stretch.conf >>> +++ b/meta-isar/conf/distro/debian-stretch.conf >>> @@ -1,13 +1,16 @@ >>> # This software is a part of ISAR. >>> # Copyright (C) 2017 ilbers GmbH >>> >>> +DISTRO_APT_SOURCES += "conf/distro/debian-stretch.list" >>> +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" >>> +DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 >>> armmp \ >>> + armmp-lpae kirkwood loongson-3 marvell octeon orion5x >>> powerpc64le \ >>> + rt-686-pae rt-amd64 s390x" >>> + >>> +# No longer needed: >>> DISTRO_SUITE ?= "stretch" >>> DISTRO_COMPONENTS ?= "main contrib non-free" >>> DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" >>> DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-secur >>> ity" >>> -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" >>> DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" >>> DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" >>> -DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 >>> armmp \ >>> - armmp-lpae kirkwood loongson-3 marvell octeon orion5x >>> powerpc64le \ >>> - rt-686-pae rt-amd64 s390x" >>> diff --git a/meta-isar/conf/distro/debian-stretch.list b/meta- >>> isar/conf/distro/debian-stretch.list >>> new file mode 100644 >>> index 0000000..bfa5cf9 >>> --- /dev/null >>> +++ b/meta-isar/conf/distro/debian-stretch.list >>> @@ -0,0 +1,3 @@ >>> +deb http://ftp.de.debian.org/debian stretch main >>> contrib non-free >>> +deb http://ftp.de.debian.org/debian stretch-updates >>> main contrib non-free >>> +deb http://security.debian.org class="Apple-tab-span" >>> style="white-space:pre"> stretch/updates main contrib non- >>> free >>> diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta- >>> isar/conf/distro/debian-wheezy.conf >>> index f5ab6eb..2108f9b 100644 >>> --- a/meta-isar/conf/distro/debian-wheezy.conf >>> +++ b/meta-isar/conf/distro/debian-wheezy.conf >>> @@ -1,14 +1,17 @@ >>> -# Debian wheezy multistrap configuration >>> +# Debian wheezy distribution configuration >>> # >>> # This software is a part of ISAR. >>> # Copyright (C) 2015-2016 ilbers GmbH >>> >>> +DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list" >>> +DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" >>> +DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile >>> vexpress \ >>> + iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" >>> + >>> +# No longer needed: >>> DISTRO_SUITE ?= "wheezy" >>> DISTRO_COMPONENTS ?= "main contrib non-free" >>> DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" >>> DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-secur >>> ity" >>> -DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" >>> DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" >>> DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" >>> -DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile >>> vexpress \ >>> - iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" >>> diff --git a/meta-isar/conf/distro/debian-wheezy.list b/meta- >>> isar/conf/distro/debian-wheezy.list >>> new file mode 100644 >>> index 0000000..0bfa98e >>> --- /dev/null >>> +++ b/meta-isar/conf/distro/debian-wheezy.list >>> @@ -0,0 +1,3 @@ >>> +deb http://ftp.de.debian.org/debian wheezy main >>> contrib non-free >>> +deb http://ftp.de.debian.org/debian wheezy-updates m >>> ain contrib non-free >>> +deb http://security.debian.org class="Apple-tab-span" >>> style="white-space:pre"> wheezy/updates main contrib non- >>> free >>> diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta- >>> isar/conf/distro/raspbian-jessie.conf >>> index c2a867d..57d0e7d 100644 >>> --- a/meta-isar/conf/distro/raspbian-jessie.conf >>> +++ b/meta-isar/conf/distro/raspbian-jessie.conf >>> @@ -1,13 +1,17 @@ >>> -# Raspbian stable multistrap configuration >>> +# Raspbian stable distribution configuration >>> # >>> # This software is a part of ISAR. >>> # Copyright (C) 2015-2016 ilbers GmbH >>> >>> +DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list" >>> +DISTRO_APT_KEYS += "https://archive.raspbian.org/raspbian.public.k >>> ey;sha256sum=ca59cd4f2bcbc3a1d41ba6815a02a8dc5c175467a59bd87edeac45 >>> 8f4a5345de" >>> +DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh" >>> +DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" >>> + >>> +# No longer needed: >>> DISTRO_SUITE ?= "jessie" >>> DISTRO_COMPONENTS ?= "main contrib non-free firmware" >>> DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian" >>> DISTRO_APT_SOURCE_SEC ?= "" >>> -DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh" >>> DISTRO_MULTICONF_BOOTSTRAP ?= "base" >>> DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" >>> -DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" >>> diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta- >>> isar/conf/distro/raspbian-jessie.list >>> new file mode 100644 >>> index 0000000..792f9c6 >>> --- /dev/null >>> +++ b/meta-isar/conf/distro/raspbian-jessie.list >>> @@ -0,0 +1 @@ >>> +deb http://archive.raspbian.org/raspbian class="Apple-tab- >>> span" style="white-space:pre"> jessie main contrib non-free >>> firmware >>> diff --git a/meta/recipes-core/isar-bootstrap/files/isar-apt.conf >>> b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf >>> new file mode 100644 >>> index 0000000..a62feb1 >>> --- /dev/null >>> +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt.conf >>> @@ -0,0 +1,2 @@ >>> +APT::Install-Recommends "0"; >>> +APT::Install-Suggests "0"; >>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> new file mode 100644 >>> index 0000000..2ddefa4 >>> --- /dev/null >>> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> @@ -0,0 +1,205 @@ >>> +# Minimal debian root file system >>> +# >>> +# This software is a part of ISAR. >>> +# Copyright (c) Siemens AG, 2018 >>> +# >>> +# SPDX-License-Identifier: MIT >>> + >>> +Description = "Minimal debian root file system" >>> + >>> +LICENSE = "gpl-2.0" >>> +LIC_FILES_CHKSUM = >>> "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa9544 >>> 99f7abaabaa882bbe" >>> +FILESPATH_prepend := "${THISDIR}/files:" >>> +SRC_URI = "file://isar-apt.conf" >>> +PV = "1.0" >>> + >>> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >>> +DEBOOTSTRAP ?= "" >>> +ROOTFSDIR = "${WORKDIR}/rootfs" >>> +APTPREFS = "${WORKDIR}/apt-preferences" >>> +APTSRCS = "${WORKDIR}/apt-sources" >>> +APTKEYFILES = "" >>> +APTKEYRING = "${WORKDIR}/apt-keyring.gpg" >>> +DEBOOTSTRAP_KEYRING = "" >>> + >>> +python () { >>> + from urllib.parse import urlparse >>> + >>> + debootstrap = d.getVar("DEBOOTSTRAP", True) >>> + if not debootstrap: >>> + target = d.getVar("DISTRO_ARCH", True) >>> + machine = os.uname()[4] >>> + m = { >>> + "x86_64": ["i386", "amd64"], >>> + "x86": ["i386"], >>> + } >>> + if machine not in m or target not in m[machine]: >>> + debootstrap = "qemu-debootstrap" >>> + else: >>> + debootstrap = "debootstrap" >>> + d.setVar("DEBOOTSTRAP", debootstrap) >>> + >>> + distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False) >>> + if distro_apt_keys: >>> + d.setVar("DEBOOTSTRAP_KEYRING", "--keyring ${APTKEYRING}") >>> + for key in distro_apt_keys.split(): >>> + url = urlparse(key) >>> + filename = os.path.basename(url.path) >>> + d.appendVar("SRC_URI", " %s" % key) >>> + d.appendVar("APTKEYFILES", " %s" % filename) >>> +} >>> + >>> +def aggregate_files(d, file_list, file_out): >>> + import shutil >>> + >>> + with open(file_out, "wb") as out_fd: >>> + for entry in file_list: >>> + entry_real = bb.parse.resolve_file(entry, d) >>> + with open(entry_real, "rb") as in_fd: >>> + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10) >>> + out_fd.write("\n".encode()) >>> + >>> +def parse_aptsources_list_line(source_list_line): >>> + import re >>> + >>> + s = source_list_line.strip() >>> + >>> + if s.startswith("#"): >>> + return None >>> + >>> + type, s = re.split("\s+", s, maxsplit=1) >>> + if type not in ["deb", "deb-src"]: >>> + return None >>> + >>> + options = "" >>> + options_match = >>> re.match("\[\s*(\S+=\S+(?=\s))*\s*(\S+=\S+)\s*\]\s+", s) >>> + if options_match: >>> + options = options_match.group(0).strip() >>> + s = s[options_match.end():] >>> + >>> + source, s = re.split("\s+", s, maxsplit=1) >>> + >>> + suite, s = re.split("\s+", s, maxsplit=1) >>> + >>> + components = " ".join(s.split()) >>> + >>> + return type, options, source, suite, components >>> + >>> +def get_distro_primary_source_entry(d): >>> + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or >>> "").split() >>> + for entry in apt_sources_list: >>> + entry_real = bb.parse.resolve_file(entry, d) >>> + with open(entry_real, "r") as in_fd: >>> + for line in in_fd: >>> + parsed = parse_aptsources_list_line(line) >>> + if parsed: >>> + type, _, source, suite, components = parsed >>> + if type == "deb": >>> + return source, suite, components >>> + return "", "", "" >>> + >>> +def get_distro_source(d): >>> + return get_distro_primary_source_entry(d)[0] >>> + >>> +def get_distro_suite(d): >>> + return get_distro_primary_source_entry(d)[1] >>> + >>> +def get_distro_components_argument(d): >>> + components = get_distro_primary_source_entry(d)[2] >>> + if components and components.strip(): >>> + return "--components=%s" % ",".join(components.split()) >>> + else: >>> + return "" >>> + >>> +do_generate_keyring[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>> +do_generate_keyring[dirs] = "${WORKDIR}" >>> +do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" >>> +do_generate_keyring() { >>> + if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then >>> + for keyfile in ${@d.getVar("APTKEYFILES", True)}; do >>> + gpg --no-default-keyring --keyring "${APTKEYRING}" -- >>> import "$keyfile" >> >> This code touches my private "~/.gnupg" folder's content, what is >> not >> desirable behavior. Isar should touch files *only* in "build/tmp" >> folder. > > Are you sure? > > $ find .gnupg -type f | sort | xargs b2sum > gpgsums.a > $ gpg --no-default-keyring --keyring ./test1.db --import > raspbian.public.key > gpg: enabled debug flags: memstat > gpg: keybox './test1.db' created > gpg: key 9165938D90FDDD2E: public key "Mike Thompson (Raspberry Pi > Debian armhf ARMv6+VFP) <mpthompson@gmail.com>" imported > gpg: Total number processed: 1 > gpg: imported: 1 > gpg: keydb: handles=3 locks=2 parse=2 get=2 > gpg: build=1 update=0 insert=1 delete=0 > gpg: reset=1 found=2 not=2 cache=0 not=0 > gpg: kid_not_found_cache: count=0 peak=0 flushes=0 > gpg: sig_cache: total=6 cached=4 good=4 bad=0 > gpg: random usage: poolsize=600 mixed=0 polls=0/0 added=0/0 > outmix=0 getlvl1=0/0 getlvl2=0/0 > gpg: rndjent stat: collector=0x0000000000000000 calls=0 bytes=0 > gpg: secmem usage: 0/32768 bytes in 0 blocks > $ find .gnupg -type f | sort | xargs b2sum > gpgsums.b > $ diff -u gpgsums.a gpgsums.b > [no output] > > I could add the '--homedir' parameter, but normally it shouldn't use it > because I manually select a different keyring. For now I have two facts: 1. Our Jenkins server has no home folder for build user, and I have the following error message: NOTE: Running task 46 of 424 (multiconfig:rpi-jessie:/workspace/git/isar_asmirnov_devel/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb:do_generate_keyring) NOTE: recipe isar-bootstrap-1.0-r0: task do_generate_keyring: Started ERROR: mc:rpi-jessie:isar-bootstrap-1.0-r0 do_generate_keyring: Function failed: do_generate_keyring (log file is located at /workspace/build/isar_asmirnov_devel/21/aa2f5faf35816611d0320ec1b615ddfc06886ea5/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_generate_keyring.25480) ERROR: Logfile of failure stored in: /workspace/build/isar_asmirnov_devel/21/aa2f5faf35816611d0320ec1b615ddfc06886ea5/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_generate_keyring.25480 Log data follows: | DEBUG: Executing shell function do_generate_keyring | gpg: fatal: can't create directory `/root/.gnupg': Permission denied | secmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536 | WARNING: exit code 2 from a shell command. | ERROR: Function failed: do_generate_keyring (log file is located at /workspace/build/isar_asmirnov_devel/21/aa2f5faf35816611d0320ec1b615ddfc06886ea5/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_generate_keyring.25480) NOTE: recipe isar-bootstrap-1.0-r0: task do_generate_keyring: Failed ERROR: Task (multiconfig:rpi-jessie:/workspace/git/isar_asmirnov_devel/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb:do_generate_keyring) failed with exit code '1' So the default HOMEDIR is /root and it tries to create something in it. 2. After building on my local machine, timestamp of files in ~/.gnugpg folder are updated. I could be wrong, but with --homedir, the second problem was gone. Alex > > Claudius > >> >> gpg already supports changing of home directory using --homedir >> option. >> So I think we could create dedicated folder "${TMPDIR}/gpg" and >> store >> keys here. >> >> At the moment I'm testing if debootstrap supports custom gpg >> storage. >> Will come back with the results. >> >> Alex >> >>> + done >>> + fi >>> +} >>> +addtask generate_keyring before do_build after do_unpack >>> + >>> +do_apt_config_prepare[stamp-extra-info] = "${DISTRO}- >>> ${DISTRO_ARCH}" >>> +do_apt_config_prepare[vardeps] += "\ >>> + APTPREFS \ >>> + DISTRO_APT_PREFERENCES \ >>> + DEBDISTRONAME \ >>> + APTSRCS \ >>> + DISTRO_APT_SOURCES \ >>> + " >>> +python do_apt_config_prepare() { >>> + apt_preferences_out = d.getVar("APTPREFS", True) >>> + apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", >>> True) or "" >>> + ).split() >>> + aggregate_files(d, apt_preferences_list, apt_preferences_out) >>> + >>> + apt_sources_out = d.getVar("APTSRCS", True) >>> + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or >>> "").split() >>> + >>> + aggregate_files(d, apt_sources_list, apt_sources_out) >>> +} >>> +addtask apt_config_prepare before do_build after >>> do_generate_keyring >>> + >>> +do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>> +do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" >>> +do_bootstrap() { >>> + if [ -e "${ROOTFSDIR}" ]; then >>> + sudo umount -l "${ROOTFSDIR}/dev" || true >>> + sudo umount -l "${ROOTFSDIR}/proc" || true >>> + sudo rm -rf "${ROOTFSDIR}" >>> + fi >>> + E="${@bb.utils.export_proxies(d)}" >>> + sudo -E "${DEBOOTSTRAP}" --verbose \ >>> + --variant minbase \ >>> + --arch "${DISTRO_ARCH}" \ >>> + ${@get_distro_components_argument(d)} >>> \ >>> + ${DEBOOTSTRAP_KEYRING} \ >>> + "${@get_distro_suite(d)}" \ >>> + "${ROOTFSDIR}" \ >>> + "${@get_distro_source(d)}" >>> +} >>> +addtask bootstrap before do_build after do_apt_config_prepare >>> + >>> +do_apt_config_install[stamp-extra-info] = "${DISTRO}- >>> ${DISTRO_ARCH}" >>> +do_apt_config_install() { >>> + sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" >>> + sudo install -v -m644 "${APTPREFS}" \ >>> + "${ROOTFSDIR}/etc/apt/preferences.d/boot >>> strap" >>> + sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" >>> + sudo install -v -m644 "${APTSRCS}" \ >>> + "${ROOTFSDIR}/etc/apt/sources.list.d/boo >>> tstrap.list" >>> + sudo rm -f "${ROOTFSDIR}/etc/apt/sources.list" >>> + sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" >>> + sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ >>> + "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar. >>> conf" >>> +} >>> +addtask apt_config_install before do_build after do_bootstrap >>> + >>> +do_apt_update[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>> +do_apt_update() { >>> + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs >>> ${ROOTFSDIR}/dev >>> + sudo mount -t proc none ${ROOTFSDIR}/proc >>> + >>> + E="${@bb.utils.export_proxies(d)}" >>> + export DEBIAN_FRONTEND=noninteractive >>> + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y >>> + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y >>> \ >>> + -o >>> Debug::pkgProblemResolver=yes >>> +} >>> +addtask apt_update before do_build after do_apt_config_install >>> + >>> +do_deploy[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>> +do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}" >>> +do_deploy() { >>> + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}- >>> ${DISTRO_ARCH}" >>> +} >>> +addtask deploy before do_build after do_apt_update >>> + >>> +CLEANFUNCS = "clean_deploy" >>> +clean_deploy() { >>> + rm -f "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}" >>> +} >>> >> >> -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 1/5] implement isar-bootstrap using debootstrap 2018-03-14 14:35 ` Alexander Smirnov @ 2018-03-14 16:13 ` Claudius Heine 0 siblings, 0 replies; 27+ messages in thread From: Claudius Heine @ 2018-03-14 16:13 UTC (permalink / raw) To: Alexander Smirnov, claudius.heine.ext, isar-users [-- Attachment #1: Type: text/plain, Size: 4519 bytes --] On Wed, 2018-03-14 at 17:35 +0300, Alexander Smirnov wrote: > On 03/14/2018 05:26 PM, Claudius Heine wrote: > > > > + > > > > +do_generate_keyring[stamp-extra-info] = "${DISTRO}- > > > > ${DISTRO_ARCH}" > > > > +do_generate_keyring[dirs] = "${WORKDIR}" > > > > +do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > > > > +do_generate_keyring() { > > > > + if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then > > > > + for keyfile in ${@d.getVar("APTKEYFILES", True)}; do > > > > + gpg --no-default-keyring --keyring "${APTKEYRING}" > > > > -- > > > > import "$keyfile" > > > > > > This code touches my private "~/.gnupg" folder's content, what is > > > not > > > desirable behavior. Isar should touch files *only* in "build/tmp" > > > folder. > > > > Are you sure? > > > > $ find .gnupg -type f | sort | xargs b2sum > gpgsums.a > > $ gpg --no-default-keyring --keyring ./test1.db --import > > raspbian.public.key > > gpg: enabled debug flags: memstat > > gpg: keybox './test1.db' created > > gpg: key 9165938D90FDDD2E: public key "Mike Thompson (Raspberry Pi > > Debian armhf ARMv6+VFP) <mpthompson@gmail.com>" imported > > gpg: Total number processed: 1 > > gpg: imported: 1 > > gpg: keydb: handles=3 locks=2 parse=2 get=2 > > gpg: build=1 update=0 insert=1 delete=0 > > gpg: reset=1 found=2 not=2 cache=0 not=0 > > gpg: kid_not_found_cache: count=0 peak=0 flushes=0 > > gpg: sig_cache: total=6 cached=4 good=4 bad=0 > > gpg: random usage: poolsize=600 mixed=0 polls=0/0 added=0/0 > > outmix=0 getlvl1=0/0 getlvl2=0/0 > > gpg: rndjent stat: collector=0x0000000000000000 calls=0 bytes=0 > > gpg: secmem usage: 0/32768 bytes in 0 blocks > > $ find .gnupg -type f | sort | xargs b2sum > gpgsums.b > > $ diff -u gpgsums.a gpgsums.b > > [no output] > > > > I could add the '--homedir' parameter, but normally it shouldn't > > use it > > because I manually select a different keyring. > > For now I have two facts: > > 1. Our Jenkins server has no home folder for build user, and I have > the > following error message: > > NOTE: Running task 46 of 424 > (multiconfig:rpi- > jessie:/workspace/git/isar_asmirnov_devel/meta/recipes-core/isar- > bootstrap/isar-bootstrap.bb:do_generate_keyring) > NOTE: recipe isar-bootstrap-1.0-r0: task do_generate_keyring: Started > ERROR: mc:rpi-jessie:isar-bootstrap-1.0-r0 do_generate_keyring: > Function > failed: do_generate_keyring (log file is located at > /workspace/build/isar_asmirnov_devel/21/aa2f5faf35816611d0320ec1b615d > dfc06886ea5/tmp/work/raspbian-jessie-armhf/isar- > bootstrap/temp/log.do_generate_keyring.25480) > ERROR: Logfile of failure stored in: > /workspace/build/isar_asmirnov_devel/21/aa2f5faf35816611d0320ec1b615d > dfc06886ea5/tmp/work/raspbian-jessie-armhf/isar- > bootstrap/temp/log.do_generate_keyring.25480 > Log data follows: > > DEBUG: Executing shell function do_generate_keyring > > gpg: fatal: can't create directory `/root/.gnupg': Permission > > denied > > secmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536 > > WARNING: exit code 2 from a shell command. > > ERROR: Function failed: do_generate_keyring (log file is located > > at > > /workspace/build/isar_asmirnov_devel/21/aa2f5faf35816611d0320ec1b615d > dfc06886ea5/tmp/work/raspbian-jessie-armhf/isar- > bootstrap/temp/log.do_generate_keyring.25480) > NOTE: recipe isar-bootstrap-1.0-r0: task do_generate_keyring: Failed > ERROR: Task > (multiconfig:rpi- > jessie:/workspace/git/isar_asmirnov_devel/meta/recipes-core/isar- > bootstrap/isar-bootstrap.bb:do_generate_keyring) > failed with exit code '1' > > So the default HOMEDIR is /root and it tries to create something in > it. > > 2. After building on my local machine, timestamp of files in > ~/.gnugpg > folder are updated. > > I could be wrong, but with --homedir, the second problem was gone. Ok, then it looks like gpg modifies the metadata of the directory. If that is easy to fix with the additional parameter, then I will do it on the next version of this patchset. Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153 Keyserver: hkp://pool.sks-keyservers.net [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v4 2/5] meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext @ 2018-03-07 16:44 ` claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 3/5] meta/buildchroot: switch to using isar-bootstrap claudius.heine.ext ` (3 subsequent siblings) 5 siblings, 0 replies; 27+ messages in thread From: claudius.heine.ext @ 2018-03-07 16:44 UTC (permalink / raw) To: isar-users; +Cc: Claudius Heine From: Claudius Heine <ch@denx.de> This patch introduces the 'isar-bootstrap-helper' bitbake class to support recipes that use the basic root file system created by isar-bootstrap. In this patch the function 'setup_root_file_system' is defined that allows to copy the root file system to the directory specified in the first parameter and installs additional packages specified by the next parameters. It also adds the local 'isar-apt' repo to the apt configuration and fetches its index. Signed-off-by: Claudius Heine <ch@denx.de> --- meta/classes/isar-bootstrap-helper.bbclass | 39 ++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 meta/classes/isar-bootstrap-helper.bbclass diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass new file mode 100644 index 0000000..a06116d --- /dev/null +++ b/meta/classes/isar-bootstrap-helper.bbclass @@ -0,0 +1,39 @@ +# Helper functions for using isar-bootstrap +# +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2018 +# +# SPDX-License-Identifier: MIT + +setup_root_file_system() { + ROOTFSDIR="$1" + shift + PACKAGES="$@" + + sudo cp -Trpfx \ + "${DEPLOY_DIR_IMAGE}/isar-bootstrap-${DISTRO}-${DISTRO_ARCH}/" \ + "$ROOTFSDIR" + + echo "deb file:///isar-apt ${DEBDISTRONAME} main" | \ + sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list" >/dev/null + + echo "Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000" | \ + sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null + + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs $ROOTFSDIR/dev + sudo mount -t proc none $ROOTFSDIR/proc + + # Install packages: + E="${@ bb.utils.export_proxies(d)}" + export DEBIAN_FRONTEND=noninteractive + sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \ + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ + -o Dir::Etc::sourceparts="-" \ + -o APT::Get::List-Cleanup="0" + sudo -E chroot "$ROOTFSDIR" \ + /usr/bin/apt-get install -y \ + --allow-unauthenticated \ + -o Debug::pkgProblemResolver=yes \ + $PACKAGES +} -- 2.16.1 ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v4 3/5] meta/buildchroot: switch to using isar-bootstrap 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 2/5] meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally claudius.heine.ext @ 2018-03-07 16:44 ` claudius.heine.ext 2018-03-08 8:18 ` Claudius Heine 2018-03-07 16:44 ` [PATCH v4 4/5] meta-isar/isar-image-base: " claudius.heine.ext ` (2 subsequent siblings) 5 siblings, 1 reply; 27+ messages in thread From: claudius.heine.ext @ 2018-03-07 16:44 UTC (permalink / raw) To: isar-users; +Cc: Claudius Heine From: Claudius Heine <ch@denx.de> This patch removes the multistrap dependency of the buildchroot recipe and changes it to use the 'isar-bootstrap' and 'isar-bootstrap-helper' instead. Signed-off-by: Claudius Heine <ch@denx.de> --- meta/recipes-devtools/buildchroot/buildchroot.bb | 59 +++++----------- meta/recipes-devtools/buildchroot/files/build.sh | 2 +- .../buildchroot/files/configscript.sh | 10 --- .../buildchroot/files/multistrap.conf.in | 37 ---------- meta/recipes-devtools/buildchroot/files/setup.sh | 82 ---------------------- 5 files changed, 18 insertions(+), 172 deletions(-) delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb index 2fc5297..b16e63a 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -8,13 +8,13 @@ DESCRIPTION = "Multistrap development filesystem" LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" -FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" -SRC_URI = "file://multistrap.conf.in \ - file://configscript.sh \ - file://setup.sh \ +FILESPATH_prepend := "${THISDIR}/files:" +SRC_URI = "file://configscript.sh \ file://build.sh" PV = "1.0" +inherit isar-bootstrap-helper + BUILDCHROOT_PREINSTALL ?= "gcc \ make \ build-essential \ @@ -31,51 +31,26 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" -do_build[depends] = "isar-apt:do_cache_config" +do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \ + ${BUILDCHROOT_DIR}/isar-apt \ + ${BUILDCHROOT_DIR}/downloads \ + ${BUILDCHROOT_DIR}/home/builder" +do_build[depends] = "isar-apt:do_cache_config isar-bootstrap:do_deploy" do_build() { - E="${@ bb.utils.export_proxies(d)}" - - # If buildchroot re-build is triggered, this ensure that we are always - # re-building a clean image: - sudo rm -rf ${BUILDCHROOT_DIR} - mkdir -p ${BUILDCHROOT_DIR}/isar-apt - mkdir -p ${BUILDCHROOT_DIR}/downloads - mkdir -p ${BUILDCHROOT_DIR}/dev - mkdir -p ${BUILDCHROOT_DIR}/proc - mkdir -p ${BUILDCHROOT_DIR}/sys - - chmod +x "${WORKDIR}/setup.sh" - chmod +x "${WORKDIR}/configscript.sh" - - # Multistrap accepts only relative path in configuration files, so get it: - cd ${TOPDIR} - WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))} - - # Adjust multistrap config - sed -e 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|g' \ - -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \ - -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \ - -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \ - -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \ - -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \ - -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \ - -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \ - -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \ - -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ - "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" - - sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt - sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc - sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads + CDIRS="${@d.expand(d.getVarFlags("do_build").get("root_cleandirs", ""))}" + if [ -n "$CDIRS" ]; then + sudo rm -rf $CDIRS + mkdir -p $CDIRS + fi - # Create root filesystem - sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" + setup_root_file_system "${BUILDCHROOT_DIR}" ${BUILDCHROOT_PREINSTALL} # Install package builder script + sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder" sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR} # Configure root filesystem + sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_DIR} sudo chroot ${BUILDCHROOT_DIR} /configscript.sh } diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh index 048f76f..06172f7 100644 --- a/meta/recipes-devtools/buildchroot/files/build.sh +++ b/meta/recipes-devtools/buildchroot/files/build.sh @@ -29,7 +29,7 @@ install_cmd="${install_cmd} --allow-unauthenticated" # APT::Get::List-Cleanup - do not erase obsolete packages list for # upstream in '/var/lib/apt/lists' apt-get update \ - -o Dir::Etc::sourcelist="sources.list.d/multistrap-isar-apt.list" \ + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ -o Dir::Etc::sourceparts="-" \ -o APT::Get::List-Cleanup="0" diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh index a5942b5..c2a599a 100644 --- a/meta/recipes-devtools/buildchroot/files/configscript.sh +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh @@ -31,13 +31,3 @@ debconf-set-selections <<END locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8 locales locales/default_environment_locale select en_US.UTF-8 END - -#set up non-interactive configuration -export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true -export LC_ALL=C LANGUAGE=C LANG=C - -#run pre installation script -/var/lib/dpkg/info/dash.preinst install - -#configuring packages -dpkg --configure -a diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in deleted file mode 100644 index 480a4b8..0000000 --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in +++ /dev/null @@ -1,37 +0,0 @@ -# This software is a part of ISAR. -# Copyright (C) 2015-2017 ilbers GmbH - -[General] -noauth=true -unpack=true -ignorenativearch=true -bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## -aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES## -configscript=##CONFIG_SCRIPT## -setupscript=##SETUP_SCRIPT## -hookdir=##DIR_HOOKS## - -[base] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE## -components=##DISTRO_COMPONENTS## -packages=##BUILDCHROOT_PREINSTALL## -omitdebsrc=true - -[updates] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE##-updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[security] -source=##DISTRO_APT_SOURCE_SEC## -suite=##DISTRO_SUITE##/updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[isar-apt] -source=file:///isar-apt -suite=isar -components=main -omitdebsrc=true diff --git a/meta/recipes-devtools/buildchroot/files/setup.sh b/meta/recipes-devtools/buildchroot/files/setup.sh deleted file mode 100755 index 1257739..0000000 --- a/meta/recipes-devtools/buildchroot/files/setup.sh +++ /dev/null @@ -1,82 +0,0 @@ -#!/bin/sh -# -# This software is a part of ISAR. -# Copyright (C) 2015-2016 ilbers GmbH -# -# Based on multistrap/examples/chroot.sh - -set -e - -# The script is called with the following arguments: -# $1 = $DIR - the top directory of the bootstrapped system -# $2 = $ARCH - the specified architecture, already checked with -# dpkg-architecture. -# setup.sh needs to be executable. - -TARGET=$1 -ARCH=$2 - -# Prevent daemons from starting in buildchroot -if [ -x "$TARGET/sbin/start-stop-daemon" ]; then - echo "initctl: Trying to prevent daemons from starting in $TARGET" - - # Disable start-stop-daemon - mv $TARGET/sbin/start-stop-daemon $TARGET/sbin/start-stop-daemon.REAL - cat > $TARGET/sbin/start-stop-daemon << EOF -#!/bin/sh -echo -echo Warning: Fake start-stop-daemon called, doing nothing -EOF - chmod 755 $TARGET/sbin/start-stop-daemon -fi - -if [ -x "$TARGET/sbin/initctl" ]; then - echo "start-stop-daemon: Trying to prevent daemons from starting in $TARGET" - - # Disable initctl - mv "$TARGET/sbin/initctl" "$TARGET/sbin/initctl.REAL" - cat > $TARGET/sbin/initctl << EOF -#!/bin/sh -echo -echo "Warning: Fake initctl called, doing nothing" -EOF - chmod 755 $TARGET/sbin/initctl -fi - -# Define sysvinit policy 101 to prevent daemons from starting in buildchroot -if [ -x "$TARGET/sbin/init" -a ! -f "$TARGET/usr/sbin/policy-rc.d" ]; then - echo "sysvinit: Using policy-rc.d to prevent daemons from starting in $TARGET" - - cat > $TARGET/usr/sbin/policy-rc.d << EOF -#!/bin/sh -echo "sysvinit: All runlevel operations denied by policy" >&2 -exit 101 -EOF - chmod a+x $TARGET/usr/sbin/policy-rc.d -fi - -case $ARCH in - armel|armhf) - qemu_arch=arm - ;; - arm64) - qemu_arch=aarch64 - ;; -esac - -# Install QEMU emulator to execute foreign binaries -if [ ! -x /usr/bin/qemu-${qemu_arch}-static ]; then - echo "qemu-${qemu_arch}-static binary not present, unable to execute target binaries" -else - sudo cp /usr/bin/qemu-${qemu_arch}-static ${TARGET}/usr/bin -fi - -# Set hostname -echo "isar" > $TARGET/etc/hostname - -# Create packages build folder -sudo install -d $TARGET/home/builder -sudo chmod -R a+rw $TARGET/home/builder - -# Install host networking settings -sudo cp /etc/resolv.conf $TARGET/etc -- 2.16.1 ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 3/5] meta/buildchroot: switch to using isar-bootstrap 2018-03-07 16:44 ` [PATCH v4 3/5] meta/buildchroot: switch to using isar-bootstrap claudius.heine.ext @ 2018-03-08 8:18 ` Claudius Heine 0 siblings, 0 replies; 27+ messages in thread From: Claudius Heine @ 2018-03-08 8:18 UTC (permalink / raw) To: isar-users; +Cc: Claudius Heine Hi, On 03/07/2018 05:44 PM, [ext] claudius.heine.ext@siemens.com wrote: > From: Claudius Heine <ch@denx.de> > > This patch removes the multistrap dependency of the buildchroot recipe > and changes it to use the 'isar-bootstrap' and 'isar-bootstrap-helper' > instead. > > Signed-off-by: Claudius Heine <ch@denx.de> > --- > meta/recipes-devtools/buildchroot/buildchroot.bb | 59 +++++----------- > meta/recipes-devtools/buildchroot/files/build.sh | 2 +- > .../buildchroot/files/configscript.sh | 10 --- > .../buildchroot/files/multistrap.conf.in | 37 ---------- > meta/recipes-devtools/buildchroot/files/setup.sh | 82 ---------------------- > 5 files changed, 18 insertions(+), 172 deletions(-) > delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in > delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh > > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb > index 2fc5297..b16e63a 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb > @@ -8,13 +8,13 @@ DESCRIPTION = "Multistrap development filesystem" > LICENSE = "gpl-2.0" > LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" > > -FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" > -SRC_URI = "file://multistrap.conf.in \ > - file://configscript.sh \ > - file://setup.sh \ > +FILESPATH_prepend := "${THISDIR}/files:" > +SRC_URI = "file://configscript.sh \ > file://build.sh" > PV = "1.0" > > +inherit isar-bootstrap-helper > + > BUILDCHROOT_PREINSTALL ?= "gcc \ > make \ > build-essential \ > @@ -31,51 +31,26 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > -do_build[depends] = "isar-apt:do_cache_config" > +do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \ > + ${BUILDCHROOT_DIR}/isar-apt \ > + ${BUILDCHROOT_DIR}/downloads \ > + ${BUILDCHROOT_DIR}/home/builder" > +do_build[depends] = "isar-apt:do_cache_config isar-bootstrap:do_deploy" > > do_build() { > - E="${@ bb.utils.export_proxies(d)}" > - > - # If buildchroot re-build is triggered, this ensure that we are always > - # re-building a clean image: > - sudo rm -rf ${BUILDCHROOT_DIR} > - mkdir -p ${BUILDCHROOT_DIR}/isar-apt > - mkdir -p ${BUILDCHROOT_DIR}/downloads > - mkdir -p ${BUILDCHROOT_DIR}/dev > - mkdir -p ${BUILDCHROOT_DIR}/proc > - mkdir -p ${BUILDCHROOT_DIR}/sys > - > - chmod +x "${WORKDIR}/setup.sh" > - chmod +x "${WORKDIR}/configscript.sh" > - > - # Multistrap accepts only relative path in configuration files, so get it: > - cd ${TOPDIR} > - WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))} > - > - # Adjust multistrap config > - sed -e 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|g' \ > - -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \ > - -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \ > - -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \ > - -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \ > - -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \ > - -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \ > - -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \ > - -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \ > - -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ > - "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" > - > - sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt > - sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev > - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > - sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads > + CDIRS="${@d.expand(d.getVarFlags("do_build").get("root_cleandirs", ""))}" > + if [ -n "$CDIRS" ]; then > + sudo rm -rf $CDIRS > + mkdir -p $CDIRS > + fi I introduced the 'root_cleandirs' flag in the hopes that these five lines could eventually move into base.bbclass and be available this way for all tasks that need it. That can be done in a later patch. Cheers, Claudius > > - # Create root filesystem > - sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" > + setup_root_file_system "${BUILDCHROOT_DIR}" ${BUILDCHROOT_PREINSTALL} > > # Install package builder script > + sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder" > sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR} > > # Configure root filesystem > + sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_DIR} > sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > } > diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh > index 048f76f..06172f7 100644 > --- a/meta/recipes-devtools/buildchroot/files/build.sh > +++ b/meta/recipes-devtools/buildchroot/files/build.sh > @@ -29,7 +29,7 @@ install_cmd="${install_cmd} --allow-unauthenticated" > # APT::Get::List-Cleanup - do not erase obsolete packages list for > # upstream in '/var/lib/apt/lists' > apt-get update \ > - -o Dir::Etc::sourcelist="sources.list.d/multistrap-isar-apt.list" \ > + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ > -o Dir::Etc::sourceparts="-" \ > -o APT::Get::List-Cleanup="0" > > diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh > index a5942b5..c2a599a 100644 > --- a/meta/recipes-devtools/buildchroot/files/configscript.sh > +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh > @@ -31,13 +31,3 @@ debconf-set-selections <<END > locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8 > locales locales/default_environment_locale select en_US.UTF-8 > END > - > -#set up non-interactive configuration > -export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true > -export LC_ALL=C LANGUAGE=C LANG=C > - > -#run pre installation script > -/var/lib/dpkg/info/dash.preinst install > - > -#configuring packages > -dpkg --configure -a > diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in > deleted file mode 100644 > index 480a4b8..0000000 > --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in > +++ /dev/null > @@ -1,37 +0,0 @@ > -# This software is a part of ISAR. > -# Copyright (C) 2015-2017 ilbers GmbH > - > -[General] > -noauth=true > -unpack=true > -ignorenativearch=true > -bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## > -aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES## > -configscript=##CONFIG_SCRIPT## > -setupscript=##SETUP_SCRIPT## > -hookdir=##DIR_HOOKS## > - > -[base] > -source=##DISTRO_APT_SOURCE## > -suite=##DISTRO_SUITE## > -components=##DISTRO_COMPONENTS## > -packages=##BUILDCHROOT_PREINSTALL## > -omitdebsrc=true > - > -[updates] > -source=##DISTRO_APT_SOURCE## > -suite=##DISTRO_SUITE##-updates > -components=##DISTRO_COMPONENTS## > -omitdebsrc=true > - > -[security] > -source=##DISTRO_APT_SOURCE_SEC## > -suite=##DISTRO_SUITE##/updates > -components=##DISTRO_COMPONENTS## > -omitdebsrc=true > - > -[isar-apt] > -source=file:///isar-apt > -suite=isar > -components=main > -omitdebsrc=true > diff --git a/meta/recipes-devtools/buildchroot/files/setup.sh b/meta/recipes-devtools/buildchroot/files/setup.sh > deleted file mode 100755 > index 1257739..0000000 > --- a/meta/recipes-devtools/buildchroot/files/setup.sh > +++ /dev/null > @@ -1,82 +0,0 @@ > -#!/bin/sh > -# > -# This software is a part of ISAR. > -# Copyright (C) 2015-2016 ilbers GmbH > -# > -# Based on multistrap/examples/chroot.sh > - > -set -e > - > -# The script is called with the following arguments: > -# $1 = $DIR - the top directory of the bootstrapped system > -# $2 = $ARCH - the specified architecture, already checked with > -# dpkg-architecture. > -# setup.sh needs to be executable. > - > -TARGET=$1 > -ARCH=$2 > - > -# Prevent daemons from starting in buildchroot > -if [ -x "$TARGET/sbin/start-stop-daemon" ]; then > - echo "initctl: Trying to prevent daemons from starting in $TARGET" > - > - # Disable start-stop-daemon > - mv $TARGET/sbin/start-stop-daemon $TARGET/sbin/start-stop-daemon.REAL > - cat > $TARGET/sbin/start-stop-daemon << EOF > -#!/bin/sh > -echo > -echo Warning: Fake start-stop-daemon called, doing nothing > -EOF > - chmod 755 $TARGET/sbin/start-stop-daemon > -fi > - > -if [ -x "$TARGET/sbin/initctl" ]; then > - echo "start-stop-daemon: Trying to prevent daemons from starting in $TARGET" > - > - # Disable initctl > - mv "$TARGET/sbin/initctl" "$TARGET/sbin/initctl.REAL" > - cat > $TARGET/sbin/initctl << EOF > -#!/bin/sh > -echo > -echo "Warning: Fake initctl called, doing nothing" > -EOF > - chmod 755 $TARGET/sbin/initctl > -fi > - > -# Define sysvinit policy 101 to prevent daemons from starting in buildchroot > -if [ -x "$TARGET/sbin/init" -a ! -f "$TARGET/usr/sbin/policy-rc.d" ]; then > - echo "sysvinit: Using policy-rc.d to prevent daemons from starting in $TARGET" > - > - cat > $TARGET/usr/sbin/policy-rc.d << EOF > -#!/bin/sh > -echo "sysvinit: All runlevel operations denied by policy" >&2 > -exit 101 > -EOF > - chmod a+x $TARGET/usr/sbin/policy-rc.d > -fi > - > -case $ARCH in > - armel|armhf) > - qemu_arch=arm > - ;; > - arm64) > - qemu_arch=aarch64 > - ;; > -esac > - > -# Install QEMU emulator to execute foreign binaries > -if [ ! -x /usr/bin/qemu-${qemu_arch}-static ]; then > - echo "qemu-${qemu_arch}-static binary not present, unable to execute target binaries" > -else > - sudo cp /usr/bin/qemu-${qemu_arch}-static ${TARGET}/usr/bin > -fi > - > -# Set hostname > -echo "isar" > $TARGET/etc/hostname > - > -# Create packages build folder > -sudo install -d $TARGET/home/builder > -sudo chmod -R a+rw $TARGET/home/builder > - > -# Install host networking settings > -sudo cp /etc/resolv.conf $TARGET/etc > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v4 4/5] meta-isar/isar-image-base: switch to using isar-bootstrap 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext ` (2 preceding siblings ...) 2018-03-07 16:44 ` [PATCH v4 3/5] meta/buildchroot: switch to using isar-bootstrap claudius.heine.ext @ 2018-03-07 16:44 ` claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 5/5] meta-isar/multiconfig: remove multistrap references claudius.heine.ext 2018-03-07 19:51 ` [PATCH v4 0/5] Debootstrap integration Jan Kiszka 5 siblings, 0 replies; 27+ messages in thread From: claudius.heine.ext @ 2018-03-07 16:44 UTC (permalink / raw) To: isar-users; +Cc: Claudius Heine From: Claudius Heine <ch@denx.de> This patch removes the multistrap dependency of the isar-image-base recipe and changes it to use the 'isar-bootstrap' and 'isar-bootstrap-helper' instead. Signed-off-by: Claudius Heine <ch@denx.de> --- .../recipes-core/images/files/download_dev-random | 24 ------- .../recipes-core/images/files/multistrap.conf.in | 38 ----------- meta-isar/recipes-core/images/files/setup.sh | 77 ---------------------- meta-isar/recipes-core/images/isar-image-base.bb | 64 ++++++------------ meta/classes/image.bbclass | 2 +- 5 files changed, 22 insertions(+), 183 deletions(-) delete mode 100644 meta-isar/recipes-core/images/files/download_dev-random delete mode 100644 meta-isar/recipes-core/images/files/multistrap.conf.in delete mode 100755 meta-isar/recipes-core/images/files/setup.sh diff --git a/meta-isar/recipes-core/images/files/download_dev-random b/meta-isar/recipes-core/images/files/download_dev-random deleted file mode 100644 index f7f5fe5..0000000 --- a/meta-isar/recipes-core/images/files/download_dev-random +++ /dev/null @@ -1,24 +0,0 @@ -#!/bin/sh -# -# This software is a part of ISAR. -# Copyright (C) 2017 ilbers GmbH - -set -e - -readonly ROOTFS="$1" - -if [ ! -c ${ROOTFS}/dev/random ]; then - rm -f ${ROOTFS}/dev/random - mknod "${ROOTFS}/dev/random" c 1 8 - chmod 666 "${ROOTFS}/dev/random" - chown 0:0 "${ROOTFS}/dev/random" -fi - -if [ ! -c ${ROOTFS}/dev/urandom ]; then - rm -f ${ROOTFS}/dev/urandom - mknod "${ROOTFS}/dev/urandom" c 1 9 - chmod 666 "${ROOTFS}/dev/urandom" - chown 0:0 "${ROOTFS}/dev/urandom" -fi - -exit 0 diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in deleted file mode 100644 index 432b6af..0000000 --- a/meta-isar/recipes-core/images/files/multistrap.conf.in +++ /dev/null @@ -1,38 +0,0 @@ -# This software is a part of ISAR. -# Copyright (C) 2015-2017 ilbers GmbH - -[General] -noauth=true -unpack=true -ignorenativearch=true -bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## Isar -aptsources=##DISTRO_MULTICONF_APTSOURCES## -configscript=##CONFIG_SCRIPT## -setupscript=##SETUP_SCRIPT## -hookdir=##DIR_HOOKS## - -[base] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE## -components=##DISTRO_COMPONENTS## -packages=##IMAGE_PREINSTALL## -omitdebsrc=true - -[updates] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE##-updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[security] -source=##DISTRO_APT_SOURCE_SEC## -suite=##DISTRO_SUITE##/updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[Isar] -packages=##IMAGE_INSTALL## -source=##DEPLOY_DIR_APT## -suite=##ISAR_DISTRO_SUITE## -components=main -omitdebsrc=true diff --git a/meta-isar/recipes-core/images/files/setup.sh b/meta-isar/recipes-core/images/files/setup.sh deleted file mode 100755 index 86c065c..0000000 --- a/meta-isar/recipes-core/images/files/setup.sh +++ /dev/null @@ -1,77 +0,0 @@ -#!/bin/sh -# -# This software is a part of ISAR. -# Copyright (C) 2015-2016 ilbers GmbH -# -# Based on multistrap/examples/chroot.sh - -set -e - -# The script is called with the following arguments: -# $1 = $DIR - the top directory of the bootstrapped system -# $2 = $ARCH - the specified architecture, already checked with -# dpkg-architecture. -# setup.sh needs to be executable. - -TARGET=$1 -ARCH=$2 - -# Prevent daemons from starting in postinstall during the initial "dpkg -# --configure -a" under QEMU on the build host -if [ -x "$TARGET/sbin/start-stop-daemon" ]; then - echo "initctl: Trying to prevent daemons from starting in $TARGET" - - # Disable start-stop-daemon - mv $TARGET/sbin/start-stop-daemon $TARGET/sbin/start-stop-daemon.REAL - cat > $TARGET/sbin/start-stop-daemon << EOF -#!/bin/sh -echo -echo Warning: Fake start-stop-daemon called, doing nothing -EOF - chmod 755 $TARGET/sbin/start-stop-daemon -fi - -if [ -x "$TARGET/sbin/initctl" ]; then - echo "start-stop-daemon: Trying to prevent daemons from starting in $TARGET" - - # Disable initctl - mv "$TARGET/sbin/initctl" "$TARGET/sbin/initctl.REAL" - cat > $TARGET/sbin/initctl << EOF -#!/bin/sh -echo -echo "Warning: Fake initctl called, doing nothing" -EOF - chmod 755 $TARGET/sbin/initctl -fi - -# Define sysvinit policy 101 to prevent daemons from starting in postinstall -# during the initial "dpkg --configure -a" under QEMU on the build host -if [ -x "$TARGET/sbin/init" -a ! -f "$TARGET/usr/sbin/policy-rc.d" ]; then - echo "sysvinit: Using policy-rc.d to prevent daemons from starting in $TARGET" - - cat > $TARGET/usr/sbin/policy-rc.d << EOF -#!/bin/sh -echo "sysvinit: All runlevel operations denied by policy" >&2 -exit 101 -EOF - chmod a+x $TARGET/usr/sbin/policy-rc.d -fi - -case $ARCH in - armel|armhf) - qemu_arch=arm - ;; - arm64) - qemu_arch=aarch64 - ;; -esac - -# Install QEMU emulator to execute foreign binaries -if [ ! -x /usr/bin/qemu-${qemu_arch}-static ]; then - echo "qemu-${qemu_arch}-static binary not present, unable to execute target binaries" -else - sudo cp /usr/bin/qemu-${qemu_arch}-static ${TARGET}/usr/bin -fi - -# Set hostname -echo "isar" > $TARGET/etc/hostname diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb index 80cb666..c4799d3 100644 --- a/meta-isar/recipes-core/images/isar-image-base.bb +++ b/meta-isar/recipes-core/images/isar-image-base.bb @@ -9,14 +9,12 @@ LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" FILESPATH =. "${LAYERDIR_isar}/recipes-core/images/files:" -SRC_URI = "file://multistrap.conf.in \ - file://${DISTRO_CONFIG_SCRIPT} \ - file://setup.sh \ - file://download_dev-random" +SRC_URI = "file://${DISTRO_CONFIG_SCRIPT}" PV = "1.0" inherit image +inherit isar-bootstrap-helper IMAGE_INSTALL += "linux-image-${KERNEL_NAME}" @@ -28,50 +26,30 @@ IMAGE_PREINSTALL += "apt \ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" -do_rootfs[dirs] = "${WORKDIR}/hooks_multistrap" +do_rootfs[root_cleandirs] = "${IMAGE_ROOTFS} \ + ${IMAGE_ROOTFS}/isar-apt" do_rootfs() { - E="${@ bb.utils.export_proxies(d)}" + CDIRS="${@d.expand(d.getVarFlags("do_rootfs").get("root_cleandirs", ""))}" + if [ -n "$CDIRS" ]; then + sudo rm -rf $CDIRS + mkdir -p $CDIRS + fi - sudo rm -rf ${IMAGE_ROOTFS} - - chmod +x "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" - chmod +x "${WORKDIR}/setup.sh" - install -m 755 "${WORKDIR}/download_dev-random" "${WORKDIR}/hooks_multistrap/" - - # Multistrap accepts only relative path in configuration files, so get it: - cd ${TOPDIR} - WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))} - - # Adjust multistrap config - sed -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \ - -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \ - -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \ - -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \ - -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \ - -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \ - -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \ - -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/${DISTRO_CONFIG_SCRIPT}|g' \ - -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \ - -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ - -e 's|##IMAGE_INSTALL##|${IMAGE_INSTALL}|g' \ - -e 's|##DEPLOY_DIR_APT##|copy:///${DEPLOY_DIR_APT}/${DISTRO}|g' \ - -e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \ - "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" - - # Do not use bitbake flag [dirs] here because this folder should have - # specific ownership. - [ ! -d ${IMAGE_ROOTFS}/proc ] && sudo install -d -o 0 -g 0 -m 555 ${IMAGE_ROOTFS}/proc - sudo mount -t proc none ${IMAGE_ROOTFS}/proc - - # Create root filesystem. We must use sudo -E here to preserve the environment - # because of proxy settings - sudo -E multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf" + setup_root_file_system "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL} # Configure root filesystem + sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}" sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} ${MACHINE_SERIAL} \ - ${BAUDRATE_TTY} ${ROOTFS_DEV} ${ROOTFS_TYPE} - sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}" + ${BAUDRATE_TTY} \ + ${ROOTFS_DEV} \ + ${ROOTFS_TYPE} - sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true + # Cleanup + sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}" + sudo rm "${IMAGE_ROOTFS}/etc/apt/sources.list.d/isar-apt.list" + sudo umount -l ${IMAGE_ROOTFS}/isar-apt + sudo rmdir ${IMAGE_ROOTFS}/isar-apt + sudo umount -l ${IMAGE_ROOTFS}/dev + sudo umount -l ${IMAGE_ROOTFS}/proc } diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 2c255f4..56038b3 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -19,7 +19,7 @@ INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')}" inherit ${IMAGE_TYPE} do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}" -do_rootfs[depends] = "isar-apt:do_cache_config" +do_rootfs[depends] = "isar-apt:do_cache_config isar-bootstrap:do_deploy" do_rootfs() { die "No root filesystem function defined, please implement in your recipe" -- 2.16.1 ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v4 5/5] meta-isar/multiconfig: remove multistrap references 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext ` (3 preceding siblings ...) 2018-03-07 16:44 ` [PATCH v4 4/5] meta-isar/isar-image-base: " claudius.heine.ext @ 2018-03-07 16:44 ` claudius.heine.ext 2018-03-07 19:51 ` [PATCH v4 0/5] Debootstrap integration Jan Kiszka 5 siblings, 0 replies; 27+ messages in thread From: claudius.heine.ext @ 2018-03-07 16:44 UTC (permalink / raw) To: isar-users; +Cc: Claudius Heine From: Claudius Heine <ch@denx.de> This patch removes the left-over references to multistrap from the configuration files. Signed-off-by: Claudius Heine <ch@denx.de> --- meta-isar/conf/distro/debian-jessie.conf | 8 -------- meta-isar/conf/distro/debian-stretch.conf | 8 -------- meta-isar/conf/distro/debian-wheezy.conf | 8 -------- meta-isar/conf/distro/raspbian-jessie.conf | 8 -------- meta-isar/conf/local.conf.sample | 4 ++-- meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +- meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +- meta-isar/conf/multiconfig/rpi-jessie.conf | 2 +- 8 files changed, 5 insertions(+), 37 deletions(-) diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta-isar/conf/distro/debian-jessie.conf index df27fd7..ea46be6 100644 --- a/meta-isar/conf/distro/debian-jessie.conf +++ b/meta-isar/conf/distro/debian-jessie.conf @@ -9,11 +9,3 @@ DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64 armmp \ armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3 octeon \ orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k-ip32 \ s390x sb1-bcm91250a versatile" - -# No longer needed: -DISTRO_SUITE ?= "jessie" -DISTRO_COMPONENTS ?= "main contrib non-free" -DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" -DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" -DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta-isar/conf/distro/debian-stretch.conf index 13c4f94..99d1737 100644 --- a/meta-isar/conf/distro/debian-stretch.conf +++ b/meta-isar/conf/distro/debian-stretch.conf @@ -6,11 +6,3 @@ DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64 armmp \ armmp-lpae kirkwood loongson-3 marvell octeon orion5x powerpc64le \ rt-686-pae rt-amd64 s390x" - -# No longer needed: -DISTRO_SUITE ?= "stretch" -DISTRO_COMPONENTS ?= "main contrib non-free" -DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" -DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" -DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta-isar/conf/distro/debian-wheezy.conf index 2108f9b..5b2e936 100644 --- a/meta-isar/conf/distro/debian-wheezy.conf +++ b/meta-isar/conf/distro/debian-wheezy.conf @@ -7,11 +7,3 @@ DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list" DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile vexpress \ iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x" - -# No longer needed: -DISTRO_SUITE ?= "wheezy" -DISTRO_COMPONENTS ?= "main contrib non-free" -DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian" -DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security" -DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security" -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-isar/conf/distro/raspbian-jessie.conf index 57d0e7d..72d6067 100644 --- a/meta-isar/conf/distro/raspbian-jessie.conf +++ b/meta-isar/conf/distro/raspbian-jessie.conf @@ -7,11 +7,3 @@ DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list" DISTRO_APT_KEYS += "https://archive.raspbian.org/raspbian.public.key;sha256sum=ca59cd4f2bcbc3a1d41ba6815a02a8dc5c175467a59bd87edeac458f4a5345de" DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh" DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv" - -# No longer needed: -DISTRO_SUITE ?= "jessie" -DISTRO_COMPONENTS ?= "main contrib non-free firmware" -DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian" -DISTRO_APT_SOURCE_SEC ?= "" -DISTRO_MULTICONF_BOOTSTRAP ?= "base" -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}" diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index 46bfae0..39875c1 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -23,10 +23,10 @@ MACHINE ??= "qemuarm" # # Multistrap Configuration Selection # -# You need to select a specific multistrap configuration which will used for both: +# You need to select a specific distribution configuration which will used for both: # generation of buildchroot environment and target root filesystem. # -# This sets the default multistrap configuration: +# This sets the default distribution configuration: DISTRO ??= "debian-jessie" DISTRO_ARCH ??= "armhf" diff --git a/meta-isar/conf/multiconfig/qemuarm-jessie.conf b/meta-isar/conf/multiconfig/qemuarm-jessie.conf index 74fb7d9..fe1d760 100644 --- a/meta-isar/conf/multiconfig/qemuarm-jessie.conf +++ b/meta-isar/conf/multiconfig/qemuarm-jessie.conf @@ -9,7 +9,7 @@ # This sets the default machine to be qemuarm in this configuration: MACHINE ?= "qemuarm" -# This sets the default multistrap configuration: +# This sets the default distribution configuration: DISTRO ?= "debian-jessie" DISTRO_ARCH ?= "armhf" diff --git a/meta-isar/conf/multiconfig/qemuarm-wheezy.conf b/meta-isar/conf/multiconfig/qemuarm-wheezy.conf index 11355a4..c573254 100644 --- a/meta-isar/conf/multiconfig/qemuarm-wheezy.conf +++ b/meta-isar/conf/multiconfig/qemuarm-wheezy.conf @@ -9,7 +9,7 @@ # This sets the default machine to be qemuarm in this configuration: MACHINE ?= "qemuarm" -# This sets the default multistrap configuration: +# This sets the default distribution configuration: DISTRO ?= "debian-wheezy" DISTRO_ARCH ?= "armhf" diff --git a/meta-isar/conf/multiconfig/rpi-jessie.conf b/meta-isar/conf/multiconfig/rpi-jessie.conf index 3f33731..fcc10d9 100644 --- a/meta-isar/conf/multiconfig/rpi-jessie.conf +++ b/meta-isar/conf/multiconfig/rpi-jessie.conf @@ -9,7 +9,7 @@ # This sets the default machine to be rpi in this configuration: MACHINE ?= "rpi" -# This sets the default multistrap configuration: +# This sets the default distribution configuration: DISTRO ?= "raspbian-jessie" DISTRO_ARCH ?= "armhf" -- 2.16.1 ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 0/5] Debootstrap integration 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext ` (4 preceding siblings ...) 2018-03-07 16:44 ` [PATCH v4 5/5] meta-isar/multiconfig: remove multistrap references claudius.heine.ext @ 2018-03-07 19:51 ` Jan Kiszka 2018-03-08 6:06 ` Jan Kiszka 5 siblings, 1 reply; 27+ messages in thread From: Jan Kiszka @ 2018-03-07 19:51 UTC (permalink / raw) To: [ext] claudius.heine.ext@siemens.com, isar-users; +Cc: Claudius Heine On 2018-03-07 17:44, [ext] claudius.heine.ext@siemens.com wrote: > From: Claudius Heine <ch@denx.de> > > Hi, > > sorry, just saw a small error, some changes to isar-bootstrap got into > the buildchroot patch. > > Cheers, > Claudius > > Changes from v3: > - A small modification to isar-bootstrap.bb snuck into the buildchroot > patch. > > Changes from v2: > - rebased to current next > - added missing newlines at the end of file > - use cleandirs with root permissions > this could be improved in some later patch series, so that this > feature can be shared. > - moved main source url of packages to apt source list files of the > distributions: > - Implemented some basic parsing of source.lists files in > isar-bootstrap. > - Made 'DISTRO_SUITE', 'DISTRO_APT_SOURCE' and 'DISTRO_COMPONENTS' > variables obsolete. > - implemented repo key fetching to support rasbian repositories > - now the clean task removes the symlink of isar-bootstrap deploy task > > Changes from v1: > - rebased to current next > - added unmounting of /dev and /proc at the end of do_rootfs in isar-image-base > This was necessary for image file creation, otherwise it tried to > package stuff from the host. > - added 'isar-bootstrap:do_deploy' dependency to do_rootfs task in > image.bbclass. > - Changed 'RFSDIR' variable name to 'ROOTFSDIR' in > isar-bootstrap-helper.bbclass to be consistent with the variable > name in isar-bootstrap.bb > - Moved 'isar-apt' apt-preference settings from isar-bootstrap.bb to > isar-bootstrap-helper.bbclass > - Removed '--no-install-recommends' parameters in favor of adding the > apt configuration in 'isar-bootstrap.bb' > - unmount and remove possible existing bootstraped directories before > bootstrapping in do_bootstrap task of 'isar-bootstrap.bb' > - changed from /dev ro bind mounts to mounting devtmpfs to /dev to be > consistent with the other places where /dev is mounted. > > > Claudius Heine (5): > implement isar-bootstrap using debootstrap > meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally > meta/buildchroot: switch to using isar-bootstrap > meta-isar/isar-image-base: switch to using isar-bootstrap > meta-isar/multiconfig: remove multistrap references > > meta-isar/conf/distro/debian-jessie.conf | 9 +- > meta-isar/conf/distro/debian-jessie.list | 3 + > meta-isar/conf/distro/debian-stretch.conf | 7 +- > meta-isar/conf/distro/debian-stretch.list | 3 + > meta-isar/conf/distro/debian-wheezy.conf | 9 +- > meta-isar/conf/distro/debian-wheezy.list | 3 + > meta-isar/conf/distro/raspbian-jessie.conf | 12 +- > meta-isar/conf/distro/raspbian-jessie.list | 1 + > meta-isar/conf/local.conf.sample | 4 +- > meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +- > meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +- > meta-isar/conf/multiconfig/rpi-jessie.conf | 2 +- > .../recipes-core/images/files/download_dev-random | 24 --- > .../recipes-core/images/files/multistrap.conf.in | 38 ---- > meta-isar/recipes-core/images/files/setup.sh | 77 -------- > meta-isar/recipes-core/images/isar-image-base.bb | 64 +++---- > meta/classes/image.bbclass | 2 +- > meta/classes/isar-bootstrap-helper.bbclass | 39 ++++ > .../isar-bootstrap/files/isar-apt.conf | 2 + > meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 +++++++++++++++++++++ > meta/recipes-devtools/buildchroot/buildchroot.bb | 59 ++---- > meta/recipes-devtools/buildchroot/files/build.sh | 2 +- > .../buildchroot/files/configscript.sh | 10 - > .../buildchroot/files/multistrap.conf.in | 37 ---- > meta/recipes-devtools/buildchroot/files/setup.sh | 82 --------- > 25 files changed, 310 insertions(+), 388 deletions(-) > create mode 100644 meta-isar/conf/distro/debian-jessie.list > create mode 100644 meta-isar/conf/distro/debian-stretch.list > create mode 100644 meta-isar/conf/distro/debian-wheezy.list > create mode 100644 meta-isar/conf/distro/raspbian-jessie.list > delete mode 100644 meta-isar/recipes-core/images/files/download_dev-random > delete mode 100644 meta-isar/recipes-core/images/files/multistrap.conf.in > delete mode 100755 meta-isar/recipes-core/images/files/setup.sh > create mode 100644 meta/classes/isar-bootstrap-helper.bbclass > create mode 100644 meta/recipes-core/isar-bootstrap/files/isar-apt.conf > create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in > delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh > FWIW, I've successfully built all CI targets locally just now. No time to test the result, though. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 0/5] Debootstrap integration 2018-03-07 19:51 ` [PATCH v4 0/5] Debootstrap integration Jan Kiszka @ 2018-03-08 6:06 ` Jan Kiszka 2018-03-08 8:14 ` Claudius Heine 0 siblings, 1 reply; 27+ messages in thread From: Jan Kiszka @ 2018-03-08 6:06 UTC (permalink / raw) To: [ext] claudius.heine.ext@siemens.com, isar-users; +Cc: Claudius Heine On 2018-03-07 20:51, [ext] Jan Kiszka wrote: > On 2018-03-07 17:44, [ext] claudius.heine.ext@siemens.com wrote: >> From: Claudius Heine <ch@denx.de> >> >> Hi, >> >> sorry, just saw a small error, some changes to isar-bootstrap got into >> the buildchroot patch. >> >> Cheers, >> Claudius >> >> Changes from v3: >> - A small modification to isar-bootstrap.bb snuck into the buildchroot >> patch. >> >> Changes from v2: >> - rebased to current next >> - added missing newlines at the end of file >> - use cleandirs with root permissions >> this could be improved in some later patch series, so that this >> feature can be shared. >> - moved main source url of packages to apt source list files of the >> distributions: >> - Implemented some basic parsing of source.lists files in >> isar-bootstrap. >> - Made 'DISTRO_SUITE', 'DISTRO_APT_SOURCE' and 'DISTRO_COMPONENTS' >> variables obsolete. >> - implemented repo key fetching to support rasbian repositories >> - now the clean task removes the symlink of isar-bootstrap deploy task >> >> Changes from v1: >> - rebased to current next >> - added unmounting of /dev and /proc at the end of do_rootfs in isar-image-base >> This was necessary for image file creation, otherwise it tried to >> package stuff from the host. >> - added 'isar-bootstrap:do_deploy' dependency to do_rootfs task in >> image.bbclass. >> - Changed 'RFSDIR' variable name to 'ROOTFSDIR' in >> isar-bootstrap-helper.bbclass to be consistent with the variable >> name in isar-bootstrap.bb >> - Moved 'isar-apt' apt-preference settings from isar-bootstrap.bb to >> isar-bootstrap-helper.bbclass >> - Removed '--no-install-recommends' parameters in favor of adding the >> apt configuration in 'isar-bootstrap.bb' >> - unmount and remove possible existing bootstraped directories before >> bootstrapping in do_bootstrap task of 'isar-bootstrap.bb' >> - changed from /dev ro bind mounts to mounting devtmpfs to /dev to be >> consistent with the other places where /dev is mounted. >> >> >> Claudius Heine (5): >> implement isar-bootstrap using debootstrap >> meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally >> meta/buildchroot: switch to using isar-bootstrap >> meta-isar/isar-image-base: switch to using isar-bootstrap >> meta-isar/multiconfig: remove multistrap references >> >> meta-isar/conf/distro/debian-jessie.conf | 9 +- >> meta-isar/conf/distro/debian-jessie.list | 3 + >> meta-isar/conf/distro/debian-stretch.conf | 7 +- >> meta-isar/conf/distro/debian-stretch.list | 3 + >> meta-isar/conf/distro/debian-wheezy.conf | 9 +- >> meta-isar/conf/distro/debian-wheezy.list | 3 + >> meta-isar/conf/distro/raspbian-jessie.conf | 12 +- >> meta-isar/conf/distro/raspbian-jessie.list | 1 + >> meta-isar/conf/local.conf.sample | 4 +- >> meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +- >> meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +- >> meta-isar/conf/multiconfig/rpi-jessie.conf | 2 +- >> .../recipes-core/images/files/download_dev-random | 24 --- >> .../recipes-core/images/files/multistrap.conf.in | 38 ---- >> meta-isar/recipes-core/images/files/setup.sh | 77 -------- >> meta-isar/recipes-core/images/isar-image-base.bb | 64 +++---- >> meta/classes/image.bbclass | 2 +- >> meta/classes/isar-bootstrap-helper.bbclass | 39 ++++ >> .../isar-bootstrap/files/isar-apt.conf | 2 + >> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 +++++++++++++++++++++ >> meta/recipes-devtools/buildchroot/buildchroot.bb | 59 ++---- >> meta/recipes-devtools/buildchroot/files/build.sh | 2 +- >> .../buildchroot/files/configscript.sh | 10 - >> .../buildchroot/files/multistrap.conf.in | 37 ---- >> meta/recipes-devtools/buildchroot/files/setup.sh | 82 --------- >> 25 files changed, 310 insertions(+), 388 deletions(-) >> create mode 100644 meta-isar/conf/distro/debian-jessie.list >> create mode 100644 meta-isar/conf/distro/debian-stretch.list >> create mode 100644 meta-isar/conf/distro/debian-wheezy.list >> create mode 100644 meta-isar/conf/distro/raspbian-jessie.list >> delete mode 100644 meta-isar/recipes-core/images/files/download_dev-random >> delete mode 100644 meta-isar/recipes-core/images/files/multistrap.conf.in >> delete mode 100755 meta-isar/recipes-core/images/files/setup.sh >> create mode 100644 meta/classes/isar-bootstrap-helper.bbclass >> create mode 100644 meta/recipes-core/isar-bootstrap/files/isar-apt.conf >> create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >> delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh >> > > FWIW, I've successfully built all CI targets locally just now. No time > to test the result, though. Images also run, and I was specifically able to build and run the next branch of jailhouse-images with these patches applied. Looks very good! And I also like "310 insertions(+), 388 deletions(-)" a lot. :) Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 0/5] Debootstrap integration 2018-03-08 6:06 ` Jan Kiszka @ 2018-03-08 8:14 ` Claudius Heine 2018-03-09 14:22 ` Jan Kiszka 0 siblings, 1 reply; 27+ messages in thread From: Claudius Heine @ 2018-03-08 8:14 UTC (permalink / raw) To: Jan Kiszka, isar-users; +Cc: Claudius Heine Hi Jan, On 03/08/2018 07:06 AM, Jan Kiszka wrote: > On 2018-03-07 20:51, [ext] Jan Kiszka wrote: >> On 2018-03-07 17:44, [ext] claudius.heine.ext@siemens.com wrote: >>> From: Claudius Heine <ch@denx.de> >>> >>> Hi, >>> >>> sorry, just saw a small error, some changes to isar-bootstrap got into >>> the buildchroot patch. >>> >>> Cheers, >>> Claudius >>> >>> Changes from v3: >>> - A small modification to isar-bootstrap.bb snuck into the buildchroot >>> patch. >>> >>> Changes from v2: >>> - rebased to current next >>> - added missing newlines at the end of file >>> - use cleandirs with root permissions >>> this could be improved in some later patch series, so that this >>> feature can be shared. >>> - moved main source url of packages to apt source list files of the >>> distributions: >>> - Implemented some basic parsing of source.lists files in >>> isar-bootstrap. >>> - Made 'DISTRO_SUITE', 'DISTRO_APT_SOURCE' and 'DISTRO_COMPONENTS' >>> variables obsolete. >>> - implemented repo key fetching to support rasbian repositories >>> - now the clean task removes the symlink of isar-bootstrap deploy task >>> >>> Changes from v1: >>> - rebased to current next >>> - added unmounting of /dev and /proc at the end of do_rootfs in isar-image-base >>> This was necessary for image file creation, otherwise it tried to >>> package stuff from the host. >>> - added 'isar-bootstrap:do_deploy' dependency to do_rootfs task in >>> image.bbclass. >>> - Changed 'RFSDIR' variable name to 'ROOTFSDIR' in >>> isar-bootstrap-helper.bbclass to be consistent with the variable >>> name in isar-bootstrap.bb >>> - Moved 'isar-apt' apt-preference settings from isar-bootstrap.bb to >>> isar-bootstrap-helper.bbclass >>> - Removed '--no-install-recommends' parameters in favor of adding the >>> apt configuration in 'isar-bootstrap.bb' >>> - unmount and remove possible existing bootstraped directories before >>> bootstrapping in do_bootstrap task of 'isar-bootstrap.bb' >>> - changed from /dev ro bind mounts to mounting devtmpfs to /dev to be >>> consistent with the other places where /dev is mounted. >>> >>> >>> Claudius Heine (5): >>> implement isar-bootstrap using debootstrap >>> meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally >>> meta/buildchroot: switch to using isar-bootstrap >>> meta-isar/isar-image-base: switch to using isar-bootstrap >>> meta-isar/multiconfig: remove multistrap references >>> >>> meta-isar/conf/distro/debian-jessie.conf | 9 +- >>> meta-isar/conf/distro/debian-jessie.list | 3 + >>> meta-isar/conf/distro/debian-stretch.conf | 7 +- >>> meta-isar/conf/distro/debian-stretch.list | 3 + >>> meta-isar/conf/distro/debian-wheezy.conf | 9 +- >>> meta-isar/conf/distro/debian-wheezy.list | 3 + >>> meta-isar/conf/distro/raspbian-jessie.conf | 12 +- >>> meta-isar/conf/distro/raspbian-jessie.list | 1 + >>> meta-isar/conf/local.conf.sample | 4 +- >>> meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +- >>> meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +- >>> meta-isar/conf/multiconfig/rpi-jessie.conf | 2 +- >>> .../recipes-core/images/files/download_dev-random | 24 --- >>> .../recipes-core/images/files/multistrap.conf.in | 38 ---- >>> meta-isar/recipes-core/images/files/setup.sh | 77 -------- >>> meta-isar/recipes-core/images/isar-image-base.bb | 64 +++---- >>> meta/classes/image.bbclass | 2 +- >>> meta/classes/isar-bootstrap-helper.bbclass | 39 ++++ >>> .../isar-bootstrap/files/isar-apt.conf | 2 + >>> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 +++++++++++++++++++++ >>> meta/recipes-devtools/buildchroot/buildchroot.bb | 59 ++---- >>> meta/recipes-devtools/buildchroot/files/build.sh | 2 +- >>> .../buildchroot/files/configscript.sh | 10 - >>> .../buildchroot/files/multistrap.conf.in | 37 ---- >>> meta/recipes-devtools/buildchroot/files/setup.sh | 82 --------- >>> 25 files changed, 310 insertions(+), 388 deletions(-) >>> create mode 100644 meta-isar/conf/distro/debian-jessie.list >>> create mode 100644 meta-isar/conf/distro/debian-stretch.list >>> create mode 100644 meta-isar/conf/distro/debian-wheezy.list >>> create mode 100644 meta-isar/conf/distro/raspbian-jessie.list >>> delete mode 100644 meta-isar/recipes-core/images/files/download_dev-random >>> delete mode 100644 meta-isar/recipes-core/images/files/multistrap.conf.in >>> delete mode 100755 meta-isar/recipes-core/images/files/setup.sh >>> create mode 100644 meta/classes/isar-bootstrap-helper.bbclass >>> create mode 100644 meta/recipes-core/isar-bootstrap/files/isar-apt.conf >>> create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in >>> delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh >>> >> >> FWIW, I've successfully built all CI targets locally just now. No time >> to test the result, though. > > Images also run, and I was specifically able to build and run the next > branch of jailhouse-images with these patches applied. Looks very good! > And I also like "310 insertions(+), 388 deletions(-)" a lot. :) Great that you like it :) But some features would need to be implemented or fixed again, like changing mirrors and setting hostname. So there is some regression happening here unfortunately. Also, I because I was busy tackling and fixing the big issues, there might still be some regressions or left over code in some places. I would try looking at the configscripts for both isar-image-base and buildchroot. I am not sure if everything there is still required or if I accidental deleted something very important from the setup scripts. I don't think so, but I haven't written this code or encountered any issues it fixes. But all those issues should be reasonable straight forward to fix, and if someone or myself follow the suggestions I just posted, then the code reduction might still happen. Cheers, Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v4 0/5] Debootstrap integration 2018-03-08 8:14 ` Claudius Heine @ 2018-03-09 14:22 ` Jan Kiszka 0 siblings, 0 replies; 27+ messages in thread From: Jan Kiszka @ 2018-03-09 14:22 UTC (permalink / raw) To: Claudius Heine, isar-users, Alexander Smirnov; +Cc: Claudius Heine On 2018-03-08 09:14, Claudius Heine wrote: > Hi Jan, > > On 03/08/2018 07:06 AM, Jan Kiszka wrote: >> On 2018-03-07 20:51, [ext] Jan Kiszka wrote: >>> On 2018-03-07 17:44, [ext] claudius.heine.ext@siemens.com wrote: >>>> From: Claudius Heine <ch@denx.de> >>>> >>>> Hi, >>>> >>>> sorry, just saw a small error, some changes to isar-bootstrap got into >>>> the buildchroot patch. >>>> >>>> Cheers, >>>> Claudius >>>> >>>> Changes from v3: >>>> - A small modification to isar-bootstrap.bb snuck into the >>>> buildchroot >>>> patch. >>>> >>>> Changes from v2: >>>> - rebased to current next >>>> - added missing newlines at the end of file >>>> - use cleandirs with root permissions >>>> this could be improved in some later patch series, so that this >>>> feature can be shared. >>>> - moved main source url of packages to apt source list files of the >>>> distributions: >>>> - Implemented some basic parsing of source.lists files in >>>> isar-bootstrap. >>>> - Made 'DISTRO_SUITE', 'DISTRO_APT_SOURCE' and 'DISTRO_COMPONENTS' >>>> variables obsolete. >>>> - implemented repo key fetching to support rasbian repositories >>>> - now the clean task removes the symlink of isar-bootstrap deploy >>>> task >>>> >>>> Changes from v1: >>>> - rebased to current next >>>> - added unmounting of /dev and /proc at the end of do_rootfs in >>>> isar-image-base >>>> This was necessary for image file creation, otherwise it tried to >>>> package stuff from the host. >>>> - added 'isar-bootstrap:do_deploy' dependency to do_rootfs task in >>>> image.bbclass. >>>> - Changed 'RFSDIR' variable name to 'ROOTFSDIR' in >>>> isar-bootstrap-helper.bbclass to be consistent with the variable >>>> name in isar-bootstrap.bb >>>> - Moved 'isar-apt' apt-preference settings from isar-bootstrap.bb to >>>> isar-bootstrap-helper.bbclass >>>> - Removed '--no-install-recommends' parameters in favor of adding >>>> the >>>> apt configuration in 'isar-bootstrap.bb' >>>> - unmount and remove possible existing bootstraped directories >>>> before >>>> bootstrapping in do_bootstrap task of 'isar-bootstrap.bb' >>>> - changed from /dev ro bind mounts to mounting devtmpfs to /dev >>>> to be >>>> consistent with the other places where /dev is mounted. >>>> >>>> >>>> Claudius Heine (5): >>>> implement isar-bootstrap using debootstrap >>>> meta/isar-bootstrap-helper.bbclass: handle rfs customization >>>> centrally >>>> meta/buildchroot: switch to using isar-bootstrap >>>> meta-isar/isar-image-base: switch to using isar-bootstrap >>>> meta-isar/multiconfig: remove multistrap references >>>> >>>> meta-isar/conf/distro/debian-jessie.conf | 9 +- >>>> meta-isar/conf/distro/debian-jessie.list | 3 + >>>> meta-isar/conf/distro/debian-stretch.conf | 7 +- >>>> meta-isar/conf/distro/debian-stretch.list | 3 + >>>> meta-isar/conf/distro/debian-wheezy.conf | 9 +- >>>> meta-isar/conf/distro/debian-wheezy.list | 3 + >>>> meta-isar/conf/distro/raspbian-jessie.conf | 12 +- >>>> meta-isar/conf/distro/raspbian-jessie.list | 1 + >>>> meta-isar/conf/local.conf.sample | 4 +- >>>> meta-isar/conf/multiconfig/qemuarm-jessie.conf | 2 +- >>>> meta-isar/conf/multiconfig/qemuarm-wheezy.conf | 2 +- >>>> meta-isar/conf/multiconfig/rpi-jessie.conf | 2 +- >>>> .../recipes-core/images/files/download_dev-random | 24 --- >>>> .../recipes-core/images/files/multistrap.conf.in | 38 ---- >>>> meta-isar/recipes-core/images/files/setup.sh | 77 -------- >>>> meta-isar/recipes-core/images/isar-image-base.bb | 64 +++---- >>>> meta/classes/image.bbclass | 2 +- >>>> meta/classes/isar-bootstrap-helper.bbclass | 39 ++++ >>>> .../isar-bootstrap/files/isar-apt.conf | 2 + >>>> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 205 >>>> +++++++++++++++++++++ >>>> meta/recipes-devtools/buildchroot/buildchroot.bb | 59 ++---- >>>> meta/recipes-devtools/buildchroot/files/build.sh | 2 +- >>>> .../buildchroot/files/configscript.sh | 10 - >>>> .../buildchroot/files/multistrap.conf.in | 37 ---- >>>> meta/recipes-devtools/buildchroot/files/setup.sh | 82 --------- >>>> 25 files changed, 310 insertions(+), 388 deletions(-) >>>> create mode 100644 meta-isar/conf/distro/debian-jessie.list >>>> create mode 100644 meta-isar/conf/distro/debian-stretch.list >>>> create mode 100644 meta-isar/conf/distro/debian-wheezy.list >>>> create mode 100644 meta-isar/conf/distro/raspbian-jessie.list >>>> delete mode 100644 >>>> meta-isar/recipes-core/images/files/download_dev-random >>>> delete mode 100644 >>>> meta-isar/recipes-core/images/files/multistrap.conf.in >>>> delete mode 100755 meta-isar/recipes-core/images/files/setup.sh >>>> create mode 100644 meta/classes/isar-bootstrap-helper.bbclass >>>> create mode 100644 >>>> meta/recipes-core/isar-bootstrap/files/isar-apt.conf >>>> create mode 100644 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>> delete mode 100644 >>>> meta/recipes-devtools/buildchroot/files/multistrap.conf.in >>>> delete mode 100755 meta/recipes-devtools/buildchroot/files/setup.sh >>>> >>> >>> FWIW, I've successfully built all CI targets locally just now. No time >>> to test the result, though. >> >> Images also run, and I was specifically able to build and run the next >> branch of jailhouse-images with these patches applied. Looks very good! >> And I also like "310 insertions(+), 388 deletions(-)" a lot. :) > > Great that you like it :) But some features would need to be implemented > or fixed again, like changing mirrors and setting hostname. So there is > some regression happening here unfortunately. > > Also, I because I was busy tackling and fixing the big issues, there > might still be some regressions or left over code in some places. I > would try looking at the configscripts for both isar-image-base and > buildchroot. I am not sure if everything there is still required or if I > accidental deleted something very important from the setup scripts. I > don't think so, but I haven't written this code or encountered any > issues it fixes. > > But all those issues should be reasonable straight forward to fix, and > if someone or myself follow the suggestions I just posted, then the code > reduction might still happen. Alex, feel free to pick up this series. I thing it would be very valuable for upcoming features like reproducible build or cross-buildchroot to have this refactoring merged first. As Claudius said, he is probably busy the next weeks with a more urgent topic. Thanks, Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2018-03-28 6:00 UTC | newest] Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-03-07 16:44 [PATCH v4 0/5] Debootstrap integration claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext 2018-03-08 7:53 ` Claudius Heine 2018-03-14 8:56 ` Alexander Smirnov 2018-03-14 9:52 ` Claudius Heine 2018-03-14 10:38 ` Alexander Smirnov 2018-03-14 14:14 ` Claudius Heine 2018-03-14 14:25 ` Alexander Smirnov 2018-03-14 18:53 ` Jan Kiszka 2018-03-15 8:05 ` Claudius Heine 2018-03-15 8:58 ` Alexander Smirnov 2018-03-15 22:07 ` Claudius Heine 2018-03-16 5:49 ` Alexander Smirnov 2018-03-28 6:00 ` Jan Kiszka 2018-03-14 14:04 ` Alexander Smirnov 2018-03-14 14:26 ` Claudius Heine 2018-03-14 14:35 ` Alexander Smirnov 2018-03-14 16:13 ` Claudius Heine 2018-03-07 16:44 ` [PATCH v4 2/5] meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 3/5] meta/buildchroot: switch to using isar-bootstrap claudius.heine.ext 2018-03-08 8:18 ` Claudius Heine 2018-03-07 16:44 ` [PATCH v4 4/5] meta-isar/isar-image-base: " claudius.heine.ext 2018-03-07 16:44 ` [PATCH v4 5/5] meta-isar/multiconfig: remove multistrap references claudius.heine.ext 2018-03-07 19:51 ` [PATCH v4 0/5] Debootstrap integration Jan Kiszka 2018-03-08 6:06 ` Jan Kiszka 2018-03-08 8:14 ` Claudius Heine 2018-03-09 14:22 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox