From: Claudius Heine <ch@denx.de>
To: claudius.heine.ext@siemens.com, isar-users@googlegroups.com
Subject: Re: [PATCH 1/5] implement isar-bootstrap using debootstrap
Date: Thu, 22 Feb 2018 09:11:17 +0100 [thread overview]
Message-ID: <1519287076.2473.97.camel@denx.de> (raw)
In-Reply-To: <20180221115003.11163-2-claudius.heine.ext@siemens.com>
[-- Attachment #1: Type: text/plain, Size: 13778 bytes --]
Hi,
ok, now with a fresh mind, I'll start reviewing my own patches :)
On Wed, 2018-02-21 at 12:49 +0100, claudius.heine.ext@siemens.com
wrote:
> From: Claudius Heine <ch@denx.de>
>
> Since multistrap is deprecated for some years, it is required to
> change
> to debootstrap.
>
> This patch introduces the 'isar-bootstrap' recipe that implement the
> creation of a minimal base root file system using debootstrap.
>
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
> meta-isar/conf/distro/debian-jessie.conf | 14 ++-
> meta-isar/conf/distro/debian-jessie.list | 2 +
> meta-isar/conf/distro/debian-stretch.conf | 12 +-
> meta-isar/conf/distro/debian-stretch.list | 2 +
> meta-isar/conf/distro/debian-wheezy.conf | 14 ++-
> meta-isar/conf/distro/debian-wheezy.list | 2 +
> meta-isar/conf/distro/raspbian-jessie.conf | 9 +-
> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 128
> +++++++++++++++++++++
> 8 files changed, 166 insertions(+), 17 deletions(-)
> create mode 100644 meta-isar/conf/distro/debian-jessie.list
> create mode 100644 meta-isar/conf/distro/debian-stretch.list
> create mode 100644 meta-isar/conf/distro/debian-wheezy.list
> create mode 100644 meta/recipes-core/isar-bootstrap/isar-
> bootstrap.bb
>
> diff --git a/meta-isar/conf/distro/debian-jessie.conf b/meta-
> isar/conf/distro/debian-jessie.conf
> index 3e62b14..d7efa63 100644
> --- a/meta-isar/conf/distro/debian-jessie.conf
> +++ b/meta-isar/conf/distro/debian-jessie.conf
> @@ -1,16 +1,20 @@
> -# Debian jessie multistrap configuration
> +# Debian jessie distribution configuration
> #
> # This software is a part of ISAR.
> # Copyright (C) 2015-2016 ilbers GmbH
>
> DISTRO_SUITE ?= "jessie"
> -DISTRO_COMPONENTS ?= "main contrib non-free"
> +# For bootstrap purposes still needed:
> DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian"
> -DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security
> "
> +DISTRO_COMPONENTS ?= "main contrib non-free"
> +DISTRO_APT_SOURCES += "conf/distro/debian-jessie.list"
> DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
> -DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
> -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> DISTRO_KERNELS ?= "486 4kc-malta 586 5kc-malta 686-pae amd64 arm64
> armmp \
> armmp-lpae ixp4xx kirkwood loongson-2e loongson-2f loongson-3
> octeon \
> orion5x powerpc powerpc-smp powerpc64 powerpc64le r4k-ip22 r5k-
> ip32 \
> s390x sb1-bcm91250a versatile"
> +
> +# No longer needed:
> +DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security
> "
> +DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
> +DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta-
> isar/conf/distro/debian-jessie.list
> new file mode 100644
> index 0000000..3b5cc59
> --- /dev/null
> +++ b/meta-isar/conf/distro/debian-jessie.list
> @@ -0,0 +1,2 @@
> +deb http://ftp.de.debian.org/debian jessie-updates mai
> n contrib non-free
> +deb http://security.debian.org class="Apple-tab-span"
> style="white-space:pre"> jessie/updates main contrib non-free
> diff --git a/meta-isar/conf/distro/debian-stretch.conf b/meta-
> isar/conf/distro/debian-stretch.conf
> index 5a85d52..26a08c5 100644
> --- a/meta-isar/conf/distro/debian-stretch.conf
> +++ b/meta-isar/conf/distro/debian-stretch.conf
> @@ -2,12 +2,16 @@
> # Copyright (C) 2017 ilbers GmbH
>
> DISTRO_SUITE ?= "stretch"
> -DISTRO_COMPONENTS ?= "main contrib non-free"
> +# For bootstrap purposes still needed:
> DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian"
> -DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security
> "
> +DISTRO_COMPONENTS ?= "main contrib non-free"
> +DISTRO_APT_SOURCES += "conf/distro/debian-stretch.list"
> DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
> -DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
> -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> DISTRO_KERNELS ?= "4kc-malta 586 5kc-malta 686 686-pae amd64 arm64
> armmp \
> armmp-lpae kirkwood loongson-3 marvell octeon orion5x
> powerpc64le \
> rt-686-pae rt-amd64 s390x"
> +
> +# No longer needed:
> +DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security
> "
> +DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
> +DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> diff --git a/meta-isar/conf/distro/debian-stretch.list b/meta-
> isar/conf/distro/debian-stretch.list
> new file mode 100644
> index 0000000..842a859
> --- /dev/null
> +++ b/meta-isar/conf/distro/debian-stretch.list
> @@ -0,0 +1,2 @@
> +deb http://ftp.de.debian.org/debian stretch-updates ma
> in contrib non-free
> +deb http://security.debian.org class="Apple-tab-span"
> style="white-space:pre"> stretch/updates main contrib non-free
> diff --git a/meta-isar/conf/distro/debian-wheezy.conf b/meta-
> isar/conf/distro/debian-wheezy.conf
> index f5ab6eb..de7a10f 100644
> --- a/meta-isar/conf/distro/debian-wheezy.conf
> +++ b/meta-isar/conf/distro/debian-wheezy.conf
> @@ -1,14 +1,18 @@
> -# Debian wheezy multistrap configuration
> +# Debian wheezy distribution configuration
> #
> # This software is a part of ISAR.
> # Copyright (C) 2015-2016 ilbers GmbH
>
> DISTRO_SUITE ?= "wheezy"
> -DISTRO_COMPONENTS ?= "main contrib non-free"
> +# For bootstrap purposes still needed:
> DISTRO_APT_SOURCE ?= "http://ftp.debian.org/debian"
> -DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security
> "
> +DISTRO_COMPONENTS ?= "main contrib non-free"
> +DISTRO_APT_SOURCES += "conf/distro/debian-wheezy.list"
> DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
> -DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
> -DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> DISTRO_KERNELS ?= "486 686-pae amd64 rt-686-pae rt-amd64 versatile
> vexpress \
> iop32x ixp4xx kirkwood mv78xx0 mx5 omap orion5x"
> +
> +# No longer needed:
> +DISTRO_APT_SOURCE_SEC ?= "http://security.debian.org/debian-security
> "
> +DISTRO_MULTICONF_BOOTSTRAP ?= "base updates security"
> +DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> diff --git a/meta-isar/conf/distro/debian-wheezy.list b/meta-
> isar/conf/distro/debian-wheezy.list
> new file mode 100644
> index 0000000..2dbb112
> --- /dev/null
> +++ b/meta-isar/conf/distro/debian-wheezy.list
> @@ -0,0 +1,2 @@
> +deb http://ftp.de.debian.org/debian wheezy-updates mai
> n contrib non-free
> +deb http://security.debian.org class="Apple-tab-span"
> style="white-space:pre"> wheezy/updates main contrib non-free
> diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-
> isar/conf/distro/raspbian-jessie.conf
> index c2a867d..0d66905 100644
> --- a/meta-isar/conf/distro/raspbian-jessie.conf
> +++ b/meta-isar/conf/distro/raspbian-jessie.conf
> @@ -1,13 +1,16 @@
> -# Raspbian stable multistrap configuration
> +# Raspbian stable distribution configuration
> #
> # This software is a part of ISAR.
> # Copyright (C) 2015-2016 ilbers GmbH
>
> DISTRO_SUITE ?= "jessie"
> +# For bootstrap purposes still needed:
> DISTRO_COMPONENTS ?= "main contrib non-free firmware"
> DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian"
> +DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh"
> +DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv"
> +
> +# No longer needed:
> DISTRO_APT_SOURCE_SEC ?= ""
> -DISTRO_CONFIG_SCRIPT ?= "raspbian-configscript.sh"
> DISTRO_MULTICONF_BOOTSTRAP ?= "base"
> DISTRO_MULTICONF_APTSOURCES ?= "${DISTRO_MULTICONF_BOOTSTRAP}"
> -DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv"
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> new file mode 100644
> index 0000000..0e9282b
> --- /dev/null
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> @@ -0,0 +1,128 @@
> +# Minimal debian root file system
> +#
> +# This software is a part of ISAR.
> +# Copyright (c) Siemens AG, 2018
> +#
> +# SPDX-License-Identifier: MIT
> +
> +Description = "Minimal debian root file system"
> +
> +LICENSE = "gpl-2.0"
> +LIC_FILES_CHKSUM =
> "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499
> f7abaabaa882bbe"
> +
> +PV = "1.0"
> +
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
> +DEBOOTSTRAP ?= ""
> +ROOTFSDIR = "${WORKDIR}/rootfs"
> +APTPREFS = "${WORKDIR}/apt-preferences"
> +APTSRCS = "${WORKDIR}/apt-sources"
> +
> +python () {
> + debootstrap = d.getVar("DEBOOTSTRAP", True)
> + if not debootstrap:
> + target = d.getVar("DISTRO_ARCH", True)
> + machine = os.uname()[4]
> + m = {
> + "x86_64": ["i386", "amd64"],
> + "x86": ["i386"],
> + }
> + if machine not in m or target not in m[machine]:
> + debootstrap = "qemu-debootstrap"
> + else:
> + debootstrap = "debootstrap"
> + d.setVar("DEBOOTSTRAP", debootstrap)
> +}
> +
> +def aggregate_files(d, file_list, file_out):
> + import shutil
> +
> + with open(file_out, "wb") as out_fd:
> + for entry in file_list:
> + entry_real = bb.parse.resolve_file(entry, d)
> + with open(entry_real, "rb") as in_fd:
> + shutil.copyfileobj(in_fd, out_fd, 1024*1024*10)
> + out_fd.write("\n".encode())
> +
> +def gen_components_argument(d):
> + components = d.getVar("DISTRO_COMPONENTS", True)
> + if components and components.strip():
> + return "--components=%s" % ",".join(components.split())
> + else:
> + return ""
> +
> +do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +do_bootstrap[dirs] = "${WORKDIR}"
> +do_bootstrap[vardeps] += "DISTRO_COMPONENTS"
> +do_bootstrap() {
> + E="${@bb.utils.export_proxies(d)}"
> + sudo -E "${DEBOOTSTRAP}" --verbose \
> + --variant minbase \
> + --arch "${DISTRO_ARCH}" \
> + ${@ gen_components_argument(d)} \
> + "${DISTRO_SUITE}" \
> + "${ROOTFSDIR}" \
> + "${DISTRO_APT_SOURCE}"
> +}
> +addtask bootstrap before do_build after do_unpack
> +
> +# TODO: add stamp with checksum of preferences and source files
> +# TODO: add mirror handling
> +do_apt_config_prepare[vardeps] += "\
> + APTPREFS \
> + DISTRO_APT_PREFERENCES \
> + DEBDISTRONAME \
> + APTSRCS \
> + DISTRO_APT_SOURCES \
> + "
> +python do_apt_config_prepare() {
> + import textwrap
> +
> + apt_preferences_out = d.getVar("APTPREFS", True)
> + apt_preferences_list = (d.getVar("DISTRO_APT_PREFERENCES", True)
> or ""
> + ).split()
> + aggregate_files(d, apt_preferences_list, apt_preferences_out)
> +
> + deb_distro_name = d.getVar("DEBDISTRONAME", True) or "isar"
> + with open(apt_preferences_out, "ab") as out_fd:
> + out_fd.write(textwrap.dedent("""\
> + Package: *
> + Pin: release n={}
> + Pin-Priority: 1000
> + """.format(deb_distro_name)).encode())
I will put this in 'isar-bootstrap-helper' instead, since there
DEBDISTRONAME is used already when adding the isar-apt repository.
Thats it for this patch,
Claudius
> +
> + apt_sources_out = d.getVar("APTSRCS", True)
> + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or
> "").split()
> +
> + aggregate_files(d, apt_sources_list, apt_sources_out)
> +}
> +addtask apt_config_prepare before do_build after do_bootstrap
> +
> +do_apt_config_install() {
> + sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d"
> + sudo install -v -m644 "${APTPREFS}" \
> + "${ROOTFSDIR}/etc/apt/preferences.d/bootst
> rap"
> + sudo mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d"
> + sudo install -v -m644 "${APTSRCS}" \
> + "${ROOTFSDIR}/etc/apt/sources.list.d/boots
> trap.list"
> +}
> +addtask apt_config_install before do_build after
> do_apt_config_prepare
> +
> +do_apt_update() {
> + E="${@bb.utils.export_proxies(d)}"
> + export DEBIAN_FRONTEND=noninteractive
> +
> + sudo mount --bind -o ro /dev ${ROOTFSDIR}/dev
> + sudo mount -t proc none ${ROOTFSDIR}/proc
> +
> + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
> + --no-install-recommends
> +}
> +addtask apt_update before do_build after do_apt_config_install
> +
> +do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
> +do_deploy() {
> + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-
> ${DISTRO_ARCH}"
> +}
> +addtask deploy before do_build after do_apt_update
> \ No newline at end of file
> --
> 2.16.1
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
Keyserver: hkp://pool.sks-keyservers.net
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2018-02-22 8:11 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-21 11:49 [PATCH 0/5] Debootstrap integration claudius.heine.ext
2018-02-21 11:49 ` [PATCH 1/5] implement isar-bootstrap using debootstrap claudius.heine.ext
2018-02-22 8:11 ` Claudius Heine [this message]
2018-02-21 11:50 ` [PATCH 2/5] meta/isar-bootstrap-helper.bbclass: handle rfs customization centrally claudius.heine.ext
2018-02-22 8:20 ` Claudius Heine
2018-02-21 11:50 ` [PATCH 3/5] meta/buildchroot: switch to using isar-bootstrap claudius.heine.ext
2018-02-21 11:50 ` [PATCH 4/5] meta-isar/isar-image-base: " claudius.heine.ext
2018-02-21 11:50 ` [PATCH 5/5] meta-isar/multiconfig: remove multistrap references claudius.heine.ext
2018-02-21 17:37 ` [PATCH 0/5] Debootstrap integration Jan Kiszka
2018-02-21 21:38 ` Claudius Heine
2018-02-22 8:20 ` Alexander Smirnov
2018-02-22 10:20 ` Claudius Heine
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1519287076.2473.97.camel@denx.de \
--to=ch@denx.de \
--cc=claudius.heine.ext@siemens.com \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox