* [PATCH v3 0/8] pre-processing pipeline and transient package replacement
@ 2019-05-20 12:46 claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline claudius.heine.ext
` (7 more replies)
0 siblings, 8 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Hi,
here is the version of this patchset that fixes the apt cache creation.
The issue was that the packages of the image where not copied to the `apt_cache`
directory when creating an image. This is needed when the `base-apt` repository
is created. Adding the `copy-package-cache` root file system feature to images
solved this.
Cheers,
Claudius
changes from v2:
- added 'copy-package-cache' to the ROOTFS_FEATURES in image.bbclass
changes from v1:
- rebased to current next
- added some more documentation
Claudius Heine (8):
split up isar-bootstrap helper and implement pre-process pipeline
meta: remove transient package support
meta/classes: add image-locales-extension class
meta/classes: add image-account-extension class
doc: update description of image customization
doc: some fixes
meta-isar: local.conf.sample: update root password and isar user
creation
RECIPE-API-CHANGELOG: update transient package removal + root password
RECIPE-API-CHANGELOG.md | 22 ++
doc/user_manual.md | 53 +++-
meta-isar/conf/local.conf.sample | 12 +-
.../recipes-app/example-raw/files/postinst | 13 -
meta/classes/image-account-extension.bbclass | 257 ++++++++++++++++++
meta/classes/image-locales-extension.bbclass | 97 +++++++
meta/classes/image.bbclass | 30 +-
meta/classes/isar-bootstrap-helper.bbclass | 143 ----------
meta/classes/rootfs.bbclass | 176 +++++++++++-
.../isar-bootstrap/isar-bootstrap.inc | 5 +
.../buildchroot/buildchroot-host.bb | 16 +-
.../buildchroot/buildchroot-target.bb | 2 -
.../buildchroot/buildchroot.inc | 26 +-
meta/recipes-devtools/sdkchroot/sdkchroot.bb | 48 ++--
.../isar-cfg-localepurge/files/postinst | 15 -
.../isar-cfg-localepurge.bb | 70 -----
.../isar-cfg-rootpw/files/postinst.tmpl | 19 --
.../isar-cfg-rootpw/isar-cfg-rootpw.bb | 19 --
18 files changed, 663 insertions(+), 360 deletions(-)
create mode 100644 meta/classes/image-account-extension.bbclass
create mode 100644 meta/classes/image-locales-extension.bbclass
delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass
delete mode 100644 meta/recipes-support/isar-cfg-localepurge/files/postinst
delete mode 100644 meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
delete mode 100644 meta/recipes-support/isar-cfg-rootpw/files/postinst.tmpl
delete mode 100644 meta/recipes-support/isar-cfg-rootpw/isar-cfg-rootpw.bb
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-23 12:35 ` Henning Schild
2019-05-20 12:46 ` [PATCH v3 2/8] meta: remove transient package support claudius.heine.ext
` (6 subsequent siblings)
7 siblings, 1 reply; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
This removes the isar-bootstrap-helper.bbclass and integrates its
functionality into the rootfs.bbclass.
It introduces the rootfs_install task that runs through multiple
functions from command lists in order to allow easy integration of
custom functions.
The introduced functions are, in general order of execution:
rootfs_prepare: copy the isar-bootstrap rootfs base to workdir
rootfs_configure_*: basic configuration of the root file system,
before any custom packages where installed. (`/dev`, `/proc`, etc.
are not mounted). Functions are added to `ROOTFS_CONFIGURE_COMMAND`.
rootfs_do_mount: mount standard directories and repositories into the
rootfs
do_rootfs_install_*: downloading and installing packages. Functions
are added to `ROOTFS_INSTALL_COMMAND`.
Signed-off-by: Claudius Heine <ch@denx.de>
---
meta/classes/image.bbclass | 24 +--
meta/classes/isar-bootstrap-helper.bbclass | 143 ---------------
meta/classes/rootfs.bbclass | 172 +++++++++++++++++-
.../isar-bootstrap/isar-bootstrap.inc | 5 +
.../buildchroot/buildchroot-host.bb | 16 +-
.../buildchroot/buildchroot-target.bb | 2 -
.../buildchroot/buildchroot.inc | 26 ++-
meta/recipes-devtools/sdkchroot/sdkchroot.bb | 48 ++---
8 files changed, 228 insertions(+), 208 deletions(-)
delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 3eb261f..107b272 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -53,8 +53,10 @@ image_do_mounts() {
buildchroot_do_mounts
}
-inherit isar-bootstrap-helper
-ROOTFS_FEATURES += "finalize-rootfs"
+ROOTFSDIR = "${IMAGE_ROOTFS}"
+ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs"
+ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
+
inherit rootfs
inherit image-sdk-extension
inherit image-cache-extension
@@ -121,8 +123,10 @@ python set_image_size () {
d.setVarFlag('ROOTFS_SIZE', 'export', '1')
}
-do_image_gen_fstab() {
- cat > ${WORKDIR}/fstab << EOF
+ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
+image_configure_fstab[weight] = "2"
+image_configure_fstab() {
+ sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
# Begin /etc/fstab
/dev/root / auto defaults 0 0
proc /proc proc nosuid,noexec,nodev 0 0
@@ -134,18 +138,6 @@ devtmpfs /dev devtmpfs mode=0755,nosuid 0 0
# End /etc/fstab
EOF
}
-addtask image_gen_fstab before do_rootfs_install
-
-do_rootfs_install[depends] = "isar-apt:do_cache_config isar-bootstrap-target:do_bootstrap"
-do_rootfs_install[deptask] = "do_deploy_deb"
-do_rootfs_install[root_cleandirs] = "${IMAGE_ROOTFS} \
- ${IMAGE_ROOTFS}/isar-apt"
-do_rootfs_install() {
- setup_root_file_system --clean --keep-apt-cache \
- --fstab "${WORKDIR}/fstab" \
- "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
-}
-addtask rootfs_install before do_build after do_unpack
do_copy_boot_files[dirs] = "${DEPLOY_DIR_IMAGE}"
do_copy_boot_files() {
diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
deleted file mode 100644
index 8612be2..0000000
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ /dev/null
@@ -1,143 +0,0 @@
-# Helper functions for using isar-bootstrap
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-IMAGE_TRANSIENT_PACKAGES ??= ""
-
-def reverse_bb_array(d, varname):
- array = d.getVar(varname, True)
- if array is None:
- return None
- array = reversed(array.split())
- return " ".join(i for i in array)
-
-
-update_etc_os_release() {
- OS_RELEASE_BUILD_ID=""
- OS_RELEASE_VARIANT=""
- while true; do
- case "$1" in
- --build-id) OS_RELEASE_BUILD_ID=$2; shift ;;
- --variant) OS_RELEASE_VARIANT=$2; shift ;;
- -*) bbfatal "$0: invalid option specified: $1" ;;
- *) break ;;
- esac
- shift
- done
- ROOTFSDIR="$1"
-
- if [ -n "${OS_RELEASE_BUILD_ID}" ]; then
- sudo sed -i '/^BUILD_ID=.*/d' ${ROOTFSDIR}/etc/os-release
- echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \
- sudo tee -a ${ROOTFSDIR}/etc/os-release
- fi
- if [ -n "${OS_RELEASE_VARIANT}" ]; then
- sudo sed -i '/^VARIANT=.*/d' ${ROOTFSDIR}/etc/os-release
- echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \
- sudo tee -a ${ROOTFSDIR}/etc/os-release
- fi
-}
-
-setup_root_file_system() {
- CLEAN=""
- COPYISARAPT=""
- FSTAB=""
- ROOTFS_ARCH="${DISTRO_ARCH}"
- ROOTFS_DISTRO="${DISTRO}"
- while true; do
- case "$1" in
- --clean) CLEAN=1 ;;
- --copyisarapt) COPYISARAPT=1 ;;
- --fstab) FSTAB=$2; shift ;;
- --host-arch) ROOTFS_ARCH="$2-$ROOTFS_ARCH"; shift ;;
- --host-distro) ROOTFS_DISTRO="$2"; shift ;;
- --keep-apt-cache) KEEP_APT_CACHE=1 ;;
- -*) bbfatal "$0: invalid option specified: $1" ;;
- *) break ;;
- esac
- shift
- done
- ROOTFSDIR="$1"
- shift
- PACKAGES="$@"
- APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
- CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
-
- sudo cp -Trpfx \
- "${DEPLOY_DIR_BOOTSTRAP}/$ROOTFS_DISTRO-$ROOTFS_ARCH/" \
- "$ROOTFSDIR"
- [ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee "$ROOTFSDIR/etc/fstab"
-
- echo "deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main" | \
- sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list" >/dev/null
-
- echo "Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000" | \
- sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
-
- if [ ${COPYISARAPT} ]; then
- sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt
- else
- sudo mount --bind ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt
- fi
-
- if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then
- sudo mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt
- fi
-
- sudo mount --rbind /dev ${ROOTFSDIR}/dev
- sudo mount --make-rslave ${ROOTFSDIR}/dev
- sudo mount -t proc none $ROOTFSDIR/proc
- sudo mount --rbind /sys ${ROOTFSDIR}/sys
- sudo mount --make-rslave ${ROOTFSDIR}/sys
-
- # Refresh /etc/resolv.conf
- sudo cp -L /etc/resolv.conf ${ROOTFSDIR}/etc
-
- # Install packages:
- E="${@ bb.utils.export_proxies(d)}"
- export DEBIAN_FRONTEND=noninteractive
- # To avoid Perl locale warnings:
- export LANG=C
- export LANGUAGE=C
- export LC_ALL=C
- sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \
- -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
- -o Dir::Etc::sourceparts="-" \
- -o APT::Get::List-Cleanup="0"
- # Add multiarch for cross-target
- if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
- sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH}
- sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
- fi
- sudo -E chroot "$ROOTFSDIR" \
- /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
- ${IMAGE_TRANSIENT_PACKAGES}
- [ ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
- sudo -E chroot "$ROOTFSDIR" \
- /usr/bin/apt-get ${APT_ARGS} $PACKAGES
- for pkg in ${IMAGE_TRANSIENT_PACKAGES}; do
- sudo -E chroot "$ROOTFSDIR" \
- /usr/bin/apt-get ${APT_ARGS} $pkg
- done
- for pkg in ${@reverse_bb_array(d, "IMAGE_TRANSIENT_PACKAGES") or ""}; do
- sudo -E chroot "$ROOTFSDIR" \
- /usr/bin/apt-get purge --yes $pkg
- done
- if [ ${CLEAN} ]; then
- if [ ${KEEP_APT_CACHE} -eq 1 ]; then
- mkdir -p ${WORKDIR}/apt_cache
- sudo find ${ROOTFSDIR}/var/cache/apt/archives \
- -maxdepth 1 -name '*.deb' -execdir /bin/mv -t ${WORKDIR}/apt_cache '{}' '+'
- sudo chown -R $(whoami) ${WORKDIR}/apt_cache
- fi
- sudo -E chroot "$ROOTFSDIR" \
- /usr/bin/apt-get autoremove --purge --yes
- sudo -E chroot "$ROOTFSDIR" \
- /usr/bin/apt-get clean
- sudo "$ROOTFSDIR/chroot-setup.sh" "cleanup" "$ROOTFSDIR"
- sudo rm -rf "$ROOTFSDIR/chroot-setup.sh" "$ROOTFSDIR/var/lib/apt/lists/"*
- fi
-}
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 45b0350..0fc8ba8 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -1,14 +1,178 @@
# This software is a part of ISAR.
# Copyright (c) Siemens AG, 2019
+ROOTFS_ARCH ?= "${DISTRO_ARCH}"
+ROOTFS_DISTRO ?= "${DISTRO}"
+ROOTFS_PACKAGES ?= ""
+
# Features of the rootfs creation:
# available features are:
+# 'deploy-package-cache' - copy the package cache ${WORKDIR}/apt_cache
+# 'clean-package-cache' - delete package cache from rootfs
# 'finalize-rootfs' - delete files needed to chroot into the rootfs
ROOTFS_FEATURES ?= ""
+ROOTFS_APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
+
+ROOTFS_CLEAN_FILES="/etc/hostname /etc/resolv.conf"
+
+# Useful environment variables:
+export E = "${@ bb.utils.export_proxies(d)}"
+export DEBIAN_FRONTEND = "noninteractive"
+# To avoid Perl locale warnings:
+export LANG = "C"
+export LANGUAGE = "C"
+export LC_ALL = "C"
+
+rootfs_do_mounts[weight] = "3"
+rootfs_do_mounts() {
+ sudo -s <<'EOSUDO'
+ mountpoint -q '${ROOTFSDIR}/dev' || \
+ mount --rbind /dev '${ROOTFSDIR}/dev'
+ mount --make-rslave '${ROOTFSDIR}/dev'
+ mountpoint -q '${ROOTFSDIR}/proc' || \
+ mount -t proc none '${ROOTFSDIR}/proc'
+ mountpoint -q '${ROOTFSDIR}/sys' || \
+ mount --rbind /sys '${ROOTFSDIR}/sys'
+ mount --make-rslave '${ROOTFSDIR}/sys'
+
+ # Mount isar-apt if the directory does not exist or if it is empty
+ # This prevents overwriting something that was copied there
+ if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \
+ [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1 | wc -l)" = "0" ]
+ then
+ mkdir -p '${ROOTFSDIR}/isar-apt'
+ mountpoint -q '${ROOTFSDIR}/isar-apt' || \
+ mount --bind '${REPO_ISAR_DIR}/${DISTRO}' '${ROOTFSDIR}/isar-apt'
+ fi
+
+ # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set
+ if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ]
+ then
+ mkdir -p '${ROOTFSDIR}/base-apt'
+ mountpoint -q '${ROOTFSDIR}/base-apt' || \
+ mount --bind '${REPO_BASE_DIR}' '${ROOTFSDIR}/base-apt'
+ fi
+
+EOSUDO
+}
+
+rootfs_do_qemu() {
+ if [ '${@repr(d.getVar('ROOTFS_ARCH') == d.getVar('HOST_ARCH'))}' = 'False' ]
+ then
+ test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \
+ sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
+ fi
+}
+
+BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}-${DISTRO_ARCH}"
+BOOTSTRAP_SRC_${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}"
+
+rootfs_prepare[weight] = "25"
+rootfs_prepare(){
+ sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}'
+}
+
+ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt"
+rootfs_configure_isar_apt[weight] = "2"
+rootfs_configure_isar_apt() {
+ sudo -s <<'EOSUDO'
+
+ mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
+ echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \
+ '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
+
+ mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
+ cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar'
+Package: *
+Pin: release n=${DEBDISTRONAME}
+Pin-Priority: 1000
+EOF
+EOSUDO
+}
+
+ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update"
+rootfs_install_pkgs_update[weight] = "5"
+rootfs_install_pkgs_update() {
+ sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
+ -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
+ -o Dir::Etc::sourceparts="-" \
+ -o APT::Get::List-Cleanup="0"
+}
+ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf"
+rootfs_install_resolvconf[weight] = "1"
+rootfs_install_resolvconf() {
+ sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
+}
+
+ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download"
+rootfs_install_pkgs_download[weight] = "600"
+rootfs_install_pkgs_download() {
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
+}
+
+ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
+rootfs_install_clean_files[weight] = "2"
+rootfs_install_clean_files() {
+ sudo -E chroot '${ROOTFSDIR}' \
+ /bin/rm -f ${ROOTFS_CLEAN_FILES}
+}
+
+ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_install"
+rootfs_install_pkgs_install[weight] = "8000"
+rootfs_install_pkgs_install() {
+ sudo -E chroot "${ROOTFSDIR}" \
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
+}
+
+do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
+do_rootfs_install[vardeps] = "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}"
+do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build isar-apt:do_cache_config"
+do_rootfs_install[deptask] = "do_deploy_deb"
+python do_rootfs_install() {
+ configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or "").split()
+ install_cmds = (d.getVar("ROOTFS_INSTALL_COMMAND", True) or "").split()
+
+ # Mount after configure commands, so that they have time to copy
+ # 'isar-apt' (sdkchroot):
+ cmds = ['rootfs_prepare'] + configure_cmds + ['rootfs_do_mounts'] + install_cmds
+
+ stage_weights = [int(d.getVarFlag(i, 'weight', True) or "20")
+ for i in cmds]
+
+ progress_reporter = bb.progress.MultiStageProgressReporter(d, stage_weights)
+
+ for cmd in cmds:
+ progress_reporter.next_stage()
+ bb.build.exec_func(cmd, d)
+ progress_reporter.finish()
+}
+addtask rootfs_install before do_rootfs_postprocess after do_unpack
+
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'copy-package-cache', 'rootfs_postprocess_copy_package_cache', '', d)}"
+rootfs_postprocess_copy_package_cache() {
+ mkdir -p '${WORKDIR}/apt_cache'
+ sudo find '${ROOTFSDIR}/var/cache/apt/archives' \
+ -maxdepth 1 -name '*.deb' -execdir /bin/mv -t '${WORKDIR}/apt_cache' '{}' '+'
+ me="$(id -u):$(id -g)"
+ sudo chown -R "$me" '${WORKDIR}/apt_cache'
+}
+
+ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"
+rootfs_postprocess_clean_package_cache() {
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/bin/apt-get clean
+ sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
+}
+
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs', 'rootfs_postprocess_finalize', '', d)}"
rootfs_postprocess_finalize() {
sudo -s <<'EOSUDO'
+ test -e "${ROOTFSDIR}/chroot-setup.sh" && \
+ "${ROOTFSDIR}/chroot-setup.sh" "cleanup" "${ROOTFSDIR}"
+ rm -f "${ROOTFSDIR}/chroot-setup.sh"
+
test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" && \
find "${ROOTFSDIR}/usr/bin" \
-maxdepth 1 -name 'qemu-*-static' -type f -delete
@@ -42,6 +206,12 @@ EOSUDO
do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
python do_rootfs_postprocess() {
+ # Take care that its correctly mounted:
+ bb.build.exec_func('rootfs_do_mounts', d)
+ # Take care that qemu-*-static is available, since it could have been
+ # removed on a previous execution of this task:
+ bb.build.exec_func('rootfs_do_qemu', d)
+
cmds = d.getVar("ROOTFS_POSTPROCESS_COMMAND")
if cmds is None or not cmds.strip():
return
@@ -49,7 +219,7 @@ python do_rootfs_postprocess() {
for cmd in cmds:
bb.build.exec_func(cmd, d)
}
-addtask rootfs_postprocess before do_rootfs after do_rootfs_install
+addtask rootfs_postprocess before do_rootfs
python do_rootfs() {
"""Virtual task"""
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index b740c57..f8741c2 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -327,6 +327,11 @@ isar_bootstrap() {
mount --make-rslave ${ROOTFSDIR}/sys
export DEBIAN_FRONTEND=noninteractive
+
+ if [ ${IS_HOST} ]; then
+ chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH}
+ fi
+
chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
-o Debug::pkgProblemResolver=yes
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
index 73cd548..121992c 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
@@ -7,6 +7,8 @@ DESCRIPTION = "Isar development filesystem for host"
PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
require buildchroot.inc
+ROOTFS_ARCH = "${HOST_ARCH}"
+ROOTFS_DISTRO = "${HOST_DISTRO}"
BUILDCHROOT_PREINSTALL ?= "make \
debhelper \
@@ -18,15 +20,5 @@ BUILDCHROOT_PREINSTALL ?= "make \
automake \
devscripts \
equivs \
- libc6:${DISTRO_ARCH}"
-
-# According to the wiki page:
-# https://wiki.debian.org/CrossToolchains
-BUILDCHROOT_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf \
- crossbuild-essential-armhf"
-BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \
- crossbuild-essential-arm64"
-
-
-PARAMS = "--host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}'"
-do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap"
+ libc6:${DISTRO_ARCH} \
+ crossbuild-essential-${DISTRO_ARCH}"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb b/meta/recipes-devtools/buildchroot/buildchroot-target.bb
index 20f4d23..5a01258 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot-target.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot-target.bb
@@ -21,5 +21,3 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
devscripts \
equivs \
adduser"
-
-do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-target:do_bootstrap"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
index ca9ce51..40fc8b0 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
@@ -13,18 +13,25 @@ SRC_URI = "file://configscript.sh \
file://deps.sh"
PV = "1.0"
-inherit isar-bootstrap-helper
+inherit rootfs
BUILDCHROOT_DIR = "${WORKDIR}/rootfs"
+ROOTFSDIR = "${BUILDCHROOT_DIR}"
+ROOTFS_PACKAGES = "${BUILDCHROOT_PREINSTALL}"
+ROOTFS_CLEAN_FILES = ""
+
+rootfs_do_mounts_append() {
+ sudo -s <<'EOSUDO'
+ mkdir -p '${BUILDCHROOT_DIR}/downloads'
+ mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \
+ mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads'
+EOSUDO
+}
-do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
- ${BUILDCHROOT_DIR}/isar-apt \
- ${BUILDCHROOT_DIR}/downloads \
- ${BUILDCHROOT_DIR}/home/builder"
-
-do_build() {
- setup_root_file_system ${PARAMS} ${BUILDCHROOT_DIR} ${BUILDCHROOT_PREINSTALL}
-
+ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files"
+buildchroot_install_files() {
+ sudo mkdir -p "${BUILDCHROOT_DIR}/home/builder"
+ sudo mkdir -p "${BUILDCHROOT_DIR}/isar-apt"
# Install package builder script
sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder"
sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar
@@ -38,5 +45,4 @@ do_build() {
GROUP_ID=$(id -g)
sudo chroot ${BUILDCHROOT_DIR} /configscript.sh $USER_ID $GROUP_ID
- sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads
}
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
index 49aeb47..57d52ba 100644
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
@@ -12,10 +12,25 @@ SRC_URI = " \
file://configscript.sh \
file://README.sdk"
PV = "0.1"
-
-inherit isar-bootstrap-helper
PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
+TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
+TOOLCHAIN_${HOST_ARCH} = "build-essential"
+TOOLCHAIN_i386 = "build-essential"
+
+inherit rootfs
+ROOTFS_ARCH = "${HOST_ARCH}"
+ROOTFS_DISTRO = "${HOST_DISTRO}"
+ROOTFSDIR = "${S}"
+ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}"
+ROOTFS_FEATURES += "copy-package-cache"
+
+python() {
+ if d.getVar("HOST_ARCH") not in ['i386', 'amd64']:
+ raise bb.parse.SkipRecipe("SDK doesn't support {} as host".format(
+ d.getVar("ROOTFS_ARCH")))
+}
+
SDKCHROOT_PREINSTALL := "debhelper \
autotools-dev \
dpkg \
@@ -28,29 +43,14 @@ SDKCHROOT_PREINSTALL := "debhelper \
S = "${WORKDIR}/rootfs"
-do_build[dirs] = "${DEPLOY_DIR_IMAGE}"
-do_build[root_cleandirs] = "${S} \
- ${S}/isar-apt"
-
-do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap"
-
-do_build() {
-
- if [ ${HOST_DISTRO} != "debian-stretch" ]; then
- bbfatal "SDK doesn't support ${HOST_DISTRO}"
- fi
- if [ ${HOST_ARCH} != "i386" -a ${HOST_ARCH} != "amd64" ]; then
- bbfatal "SDK doesn't support ${HOST_ARCH} as host"
- fi
-
- if [ ${HOST_ARCH} = ${DISTRO_ARCH} -o ${DISTRO_ARCH} = "i386" ]; then
- packages="${SDKCHROOT_PREINSTALL} build-essential"
- else
- packages="${SDKCHROOT_PREINSTALL} crossbuild-essential-${DISTRO_ARCH}"
- fi
-
- setup_root_file_system --copyisarapt --host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}' "${S}" $packages
+ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir"
+rootfs_configure_isar_apt_dir() {
+ # Copy isar-apt instead of mounting:
+ sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
+}
+ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
+sdkchroot_install_files() {
# Configure root filesystem
sudo install -m 644 ${WORKDIR}/README.sdk ${S}
sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 2/8] meta: remove transient package support
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 3/8] meta/classes: add image-locales-extension class claudius.heine.ext
` (5 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Transient packages where meant for configuring images via packages,
which proved to not be fit for purpose.
Packages are meant to be shared across images, configuring it this way
would mean that every possible configuration permutation would need to
result in a separate package just so that no image accidentally installs
a configuration package meant for a different build/image.
Instead images need to be configured while its specific root file system
is created using the new root file system creation pipeline.
Signed-off-by: Claudius Heine <ch@denx.de>
---
meta/classes/image.bbclass | 4 +-
.../isar-cfg-localepurge/files/postinst | 15 ----
.../isar-cfg-localepurge.bb | 70 -------------------
.../isar-cfg-rootpw/files/postinst.tmpl | 19 -----
.../isar-cfg-rootpw/isar-cfg-rootpw.bb | 19 -----
5 files changed, 1 insertion(+), 126 deletions(-)
delete mode 100644 meta/recipes-support/isar-cfg-localepurge/files/postinst
delete mode 100644 meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
delete mode 100644 meta/recipes-support/isar-cfg-rootpw/files/postinst.tmpl
delete mode 100644 meta/recipes-support/isar-cfg-rootpw/isar-cfg-rootpw.bb
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 107b272..99eea92 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -36,9 +36,7 @@ def cfg_script(d):
FILESPATH =. "${LAYERDIR_core}/conf/distro:"
SRC_URI += "${@ cfg_script(d) }"
-DEPENDS += "${IMAGE_INSTALL} ${IMAGE_TRANSIENT_PACKAGES}"
-
-IMAGE_TRANSIENT_PACKAGES += "isar-cfg-localepurge isar-cfg-rootpw"
+DEPENDS += "${IMAGE_INSTALL}"
ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_core} describe --tags --dirty --match 'v[0-9].[0-9]*'"
ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
diff --git a/meta/recipes-support/isar-cfg-localepurge/files/postinst b/meta/recipes-support/isar-cfg-localepurge/files/postinst
deleted file mode 100644
index 9dfd6f9..0000000
--- a/meta/recipes-support/isar-cfg-localepurge/files/postinst
+++ /dev/null
@@ -1,15 +0,0 @@
-#!/bin/sh
-set -e
-
-cat /usr/lib/isar-cfg-localepurge/locale.gen \
- >> /etc/locale.gen
-cat /usr/lib/isar-cfg-localepurge/locale.default \
- > /etc/default/locale
-cat /usr/lib/isar-cfg-localepurge/locale.nopurge \
- > /etc/locale.nopurge
-
-debconf-set-selections /usr/lib/isar-cfg-localepurge/locale.debconf
-
-dpkg-reconfigure -f noninteractive locales
-
-localepurge
diff --git a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
deleted file mode 100644
index 62b4b2d..0000000
--- a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
+++ /dev/null
@@ -1,70 +0,0 @@
-# This software is a part of ISAR.
-
-DESCRIPTION = "Isar configuration package for locale and localepurge"
-MAINTAINER = "isar-users <isar-users@googlegroups.com>"
-DEBIAN_DEPENDS = "localepurge"
-
-SRC_URI = "file://postinst"
-
-inherit dpkg-raw
-
-LOCALE_GEN ?= "en_US.UTF-8 UTF-8\n\
- en_US ISO-8859-1\n"
-LOCALE_DEFAULT ?= "en_US.UTF-8"
-
-def get_locale_gen(d):
- locale_gen = d.getVar("LOCALE_GEN", True) or ""
- return '\n'.join(sorted(set(i.strip()
- for i in locale_gen.split('\\n')
- if i.strip())))
-
-def get_dc_locale_gen(d):
- locale_gen = d.getVar("LOCALE_GEN", True) or ""
- return ', '.join(sorted(set(i.strip()
- for i in locale_gen.split('\\n')
- if i.strip())))
-
-def get_nopurge(d):
- locale_gen = d.getVar("LOCALE_GEN", True) or ""
- return '\n'.join(sorted(set(i.strip()
- for j in locale_gen.split('\\n')
- if j.strip()
- for i in (j.split()[0].split("_")[0],
- j.split()[0].split(".")[0],
- j.split()[0]))))
-
-do_gen_config[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-do_gen_config() {
- cat<<-__EOF__ > ${WORKDIR}/locale.gen
- ${@get_locale_gen(d)}
- __EOF__
- cat<<-__EOF__ > ${WORKDIR}/locale.debconf
- locales locales/locales_to_be_generated multiselect ${@get_dc_locale_gen(d)}
- locales locales/default_environment_locale select ${LOCALE_DEFAULT}
- __EOF__
- cat<<-__EOF__ > ${WORKDIR}/locale.default
- LANG=${LOCALE_DEFAULT}
- __EOF__
- cat<<-__EOF__ > ${WORKDIR}/locale.nopurge
- #USE_DPKG
- MANDELETE
- DONTBOTHERNEWLOCALE
- #SHOWFREEDSPACE
- #QUICKNDIRTYCALC
- #VERBOSE
- ${@get_nopurge(d)}
- __EOF__
-}
-addtask gen_config after do_unpack before do_install
-
-do_install() {
- install -v -d ${D}/usr/lib/${PN}
- install -v -m 644 ${WORKDIR}/locale.debconf \
- ${D}/usr/lib/${PN}/locale.debconf
- install -v -m 644 ${WORKDIR}/locale.gen \
- ${D}/usr/lib/${PN}/locale.gen
- install -v -m 644 ${WORKDIR}/locale.default \
- ${D}/usr/lib/${PN}/locale.default
- install -v -m 644 ${WORKDIR}/locale.nopurge \
- ${D}/usr/lib/${PN}/locale.nopurge
-}
diff --git a/meta/recipes-support/isar-cfg-rootpw/files/postinst.tmpl b/meta/recipes-support/isar-cfg-rootpw/files/postinst.tmpl
deleted file mode 100644
index ca08a41..0000000
--- a/meta/recipes-support/isar-cfg-rootpw/files/postinst.tmpl
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/bin/sh
-set -e
-
-if ! grep -q 'root:\*:' /etc/shadow; then
- echo "ERROR:isar-cfg-rootpw: root password was set by a different package" >&2
- exit -1
-fi
-
-if [ -n '${CFG_ROOT_PW}' ]; then
- echo 'root:${CFG_ROOT_PW}' | chpasswd -e
-else
- passwd -d root
-fi
-
-if [ '${CFG_ROOT_LOCKED}' = "1" ]; then
- # Lock the account after setting the password, since unlocking it at some
- # point later would set it to the back to the previous one.
- passwd -l root
-fi
diff --git a/meta/recipes-support/isar-cfg-rootpw/isar-cfg-rootpw.bb b/meta/recipes-support/isar-cfg-rootpw/isar-cfg-rootpw.bb
deleted file mode 100644
index adee3b5..0000000
--- a/meta/recipes-support/isar-cfg-rootpw/isar-cfg-rootpw.bb
+++ /dev/null
@@ -1,19 +0,0 @@
-# This software is a part of ISAR.
-
-DESCRIPTION = "Isar configuration package for root password"
-MAINTAINER = "isar-users <isar-users@googlegroups.com>"
-DEBIAN_DEPENDS = "passwd"
-
-SRC_URI = "file://postinst.tmpl"
-
-TEMPLATE_FILES = "postinst.tmpl"
-TEMPLATE_VARS = "CFG_ROOT_PW CFG_ROOT_LOCKED"
-
-CFG_ROOT_PW ??= ""
-CFG_ROOT_LOCKED ??= "0"
-
-inherit dpkg-raw
-
-do_install() {
- echo "intentionally left blank"
-}
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 3/8] meta/classes: add image-locales-extension class
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 2/8] meta: remove transient package support claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 4/8] meta/classes: add image-account-extension class claudius.heine.ext
` (4 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
This class allows to configure the locales of the image and also uses
'localepurge' to remove unwanted locales.
The locale can be configured the same way as with the
isar-cfg-localepurge package, via the `LOCALE_GEN` and `LOCALE_DEFAULT`
variables.
Signed-off-by: Claudius Heine <ch@denx.de>
---
meta/classes/image-locales-extension.bbclass | 97 ++++++++++++++++++++
meta/classes/image.bbclass | 1 +
meta/classes/rootfs.bbclass | 4 +
3 files changed, 102 insertions(+)
create mode 100644 meta/classes/image-locales-extension.bbclass
diff --git a/meta/classes/image-locales-extension.bbclass b/meta/classes/image-locales-extension.bbclass
new file mode 100644
index 0000000..3c0758f
--- /dev/null
+++ b/meta/classes/image-locales-extension.bbclass
@@ -0,0 +1,97 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass for setting locales and purging unneeded
+# ones.
+
+LOCALE_GEN ?= "en_US.UTF-8 UTF-8\n\
+ en_US ISO-8859-1\n"
+LOCALE_DEFAULT ?= "en_US.UTF-8"
+
+def get_locale_gen(d, sep='\n'):
+ locale_gen = d.getVar("LOCALE_GEN", True) or ""
+ return sep.join(sorted(set(i.strip()
+ for i in locale_gen.split('\\n')
+ if i.strip())))
+
+def get_nopurge(d):
+ locale_gen = d.getVar("LOCALE_GEN", True) or ""
+ return '\n'.join(sorted(set(i.strip()
+ for j in locale_gen.split('\\n')
+ if j.strip()
+ for i in (j.split()[0].split("_")[0],
+ j.split()[0].split(".")[0],
+ j.split()[0]))))
+
+ROOTFS_INSTALL_COMMAND_BEFORE_CLEAN += "image_install_localepurge_download"
+image_install_localepurge_download[weight] = "40"
+image_install_localepurge_download() {
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only localepurge
+}
+
+ROOTFS_INSTALL_COMMAND += "image_install_localepurge_install"
+image_install_localepurge_install[weight] = "700"
+image_install_localepurge_install() {
+
+ # Generate locale and localepurge configuration:
+ cat<<__EOF__ > ${WORKDIR}/locale.gen
+${@get_locale_gen(d)}
+__EOF__
+ cat<<__EOF__ > ${WORKDIR}/locale.debconf
+locales locales/locales_to_be_generated multiselect ${@get_locale_gen(d, ', ')}
+locales locales/default_environment_locale select ${LOCALE_DEFAULT}
+__EOF__
+ cat<<__EOF__ > ${WORKDIR}/locale.default
+LANG=${LOCALE_DEFAULT}
+__EOF__
+ cat<<__EOF__ > ${WORKDIR}/locale.nopurge
+#USE_DPKG
+MANDELETE
+DONTBOTHERNEWLOCALE
+#SHOWFREEDSPACE
+#QUICKNDIRTYCALC
+#VERBOSE
+${@get_nopurge(d)}
+__EOF__
+
+ # Install configuration into image:
+ sudo -E -s <<'EOSUDO'
+ cat '${WORKDIR}/locale.gen' >> '${ROOTFSDIR}/etc/locale.gen'
+ cat '${WORKDIR}/locale.default' > '${ROOTFSDIR}/etc/default/locale'
+ cat '${WORKDIR}/locale.nopurge' > '${ROOTFSDIR}/etc/locale.nopurge'
+ cat '${WORKDIR}/locale.debconf' > '${ROOTFSDIR}/tmp/locale.debconf'
+
+ # Enter image and trigger locales config and localepurge:
+ chroot '${ROOTFSDIR}' /bin/sh <<'EOSH'
+ localepurge_state='i'
+ if dpkg -s localepurge 2>/dev/null >&2
+ then
+ echo 'localepurge was installed (leaving it installed later)'
+ else
+ localepurge_state='p'
+ echo 'localepurge was not installed (removing it later)'
+ apt-get ${ROOTFS_APT_ARGS} localepurge
+ fi
+
+ echo 'running locale debconf-set-selections'
+ debconf-set-selections /tmp/locale.debconf
+ rm -f '/tmp/locale.debconf'
+
+ echo 'reconfigure locales'
+ dpkg-reconfigure -f noninteractive locales
+
+ echo 'running localepurge'
+ localepurge
+
+ if [ "$localepurge_state" = 'p' ]
+ then
+ echo removing localepurge...
+ apt-get purge --yes localepurge
+ apt-get autoremove --purge --yes
+ fi
+EOSH
+EOSUDO
+}
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 99eea92..dce6638 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -60,6 +60,7 @@ inherit image-sdk-extension
inherit image-cache-extension
inherit image-tools-extension
inherit image-postproc-extension
+inherit image-locales-extension
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 0fc8ba8..5736eee 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -99,6 +99,7 @@ rootfs_install_pkgs_update() {
-o Dir::Etc::sourceparts="-" \
-o APT::Get::List-Cleanup="0"
}
+
ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf"
rootfs_install_resolvconf[weight] = "1"
rootfs_install_resolvconf() {
@@ -112,6 +113,9 @@ rootfs_install_pkgs_download() {
/usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
}
+ROOTFS_INSTALL_COMMAND_BEFORE_CLEAN ??= ""
+ROOTFS_INSTALL_COMMAND += "${ROOTFS_INSTALL_COMMAND_BEFORE_CLEAN}"
+
ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
rootfs_install_clean_files[weight] = "2"
rootfs_install_clean_files() {
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 4/8] meta/classes: add image-account-extension class
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
` (2 preceding siblings ...)
2019-05-20 12:46 ` [PATCH v3 3/8] meta/classes: add image-locales-extension class claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 5/8] doc: update description of image customization claudius.heine.ext
` (3 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
This class allows to configure user and group accounts of the image.
Groups or users that should be configured/created are added into the
`GROUPS` or `USERS` variable.
The configuration itself is then added to each groups or users
`GROUP_<groupname>` or `USER_<username>` flags.
The flags available for groups are `gid` and `flags`. The `flags`
variable contains some additional options for the group. With this patch
only `system` is supported for groups, allowing to create groups with
`groupadd` with the `--system` parameter.
The flags available for users are `password`, `expire`, `inactive`,
`uid`, `gid`, `comment`, `home`, `shell`, `groups` and `flags`. The
additional flags for users are `no-create-home`, `create-home`, `system`
and `allow-empty-password`.
Signed-off-by: Claudius Heine <ch@denx.de>
---
meta/classes/image-account-extension.bbclass | 257 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
2 files changed, 258 insertions(+)
create mode 100644 meta/classes/image-account-extension.bbclass
diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/image-account-extension.bbclass
new file mode 100644
index 0000000..22754da
--- /dev/null
+++ b/meta/classes/image-account-extension.bbclass
@@ -0,0 +1,257 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass for creating user accounts and groups.
+
+USERS ??= ""
+
+#USERS += "root"
+#USER_root[password] = "" # Encrypted password
+#USER_root[expire] = ""
+#USER_root[inactive] = ""
+#USER_root[uid] = ""
+#USER_root[gid] = "" # If first character is a number: gid, otherwise groupname
+#USER_root[comment] = "The ultimate root user"
+#USER_root[home] = "/home/root"
+#USER_root[shell] = "/bin/sh"
+#USER_root[groups] = "audio video"
+#USER_root[flags] = "no-create-home create-home system allow-empty-password"
+
+GROUPS ??= ""
+
+#GROUPS += "root"
+#GROUP_root[gid] = ""
+#GROUP_root[flags] = "system"
+
+def gen_accounts_array(d, listname, entryname, flags, verb_flags=None):
+ from itertools import chain
+
+ entries = (d.getVar(listname, True) or "").split()
+ return " ".join(
+ ":".join(
+ chain(
+ (entry,),
+ (
+ (",".join(
+ (
+ d.getVarFlag(entryname + "_" + entry, flag, True) or ""
+ ).split()
+ ) if flag not in (verb_flags or []) else (
+ d.getVarFlag(entryname + "_" + entry, flag, True) or ""
+ )).replace(":","=")
+ for flag in flags
+ ),
+ )
+ )
+ for entry in entries
+ )
+
+# List of space separated entries, where each entry has the format:
+# username:encryptedpassword:expiredate:inactivenumber:userid:groupid:comment:homedir:shell:group1,group2:flag1,flag2
+IMAGE_ACCOUNTS_USERS =+ "${@gen_accounts_array(d, 'USERS', 'USER', ['password', 'expire', 'inactive', 'uid', 'gid', 'comment', 'home', 'shell', 'groups', 'flags'], ['password', 'comment', 'home', 'shell'])}"
+
+# List of space separated entries, where each entry has the format:
+# groupname:groupid:flag1,flag2
+IMAGE_ACCOUNTS_GROUPS =+ "${@gen_accounts_array(d, 'GROUPS', 'GROUP', ['gid', 'flags'])}"
+
+ROOTFS_CONFIGURE_COMMAND += "image_configure_accounts"
+image_configure_accounts[weight] = "3"
+image_configure_accounts() {
+ # Create groups
+ # Add space to the end of the list:
+ list='${@" ".join(d.getVar('IMAGE_ACCOUNTS_GROUPS', True).split())} '
+ while true; do
+ # Pop first group entry:
+ list_rest="${list#*:*:* }"
+ entry="${list%%${list_rest}}"
+ list="${list_rest}"
+
+ if [ -z "${entry}" ]; then
+ break
+ fi
+
+ # Add colon to the end of the entry and remove trailing space:
+ entry="${entry% }:"
+
+ # Decode entries:
+ name="${entry%%:*}"
+ entry="${entry#${name}:}"
+
+ gid="${entry%%:*}"
+ entry="${entry#${gid}:}"
+
+ flags="${entry%%:*}"
+ entry="${entry#${flags}:}"
+
+ flags=",${flags}," # Needed for searching for substrings
+
+ # Check if user already exists:
+ if grep -q "^${name}:" '${ROOTFSDIR}/etc/group'; then
+ exists="y"
+ else
+ exists="n"
+ fi
+
+ # Create arguments:
+ set -- # clear arguments
+
+ if [ -n "$gid" ]; then
+ set -- "$@" --gid "$gid"
+ fi
+
+ if [ "n" = "$exists" ]; then
+ if [ "${flags}" != "${flags%*,system,*}" ]; then
+ set -- "$@" --system
+ fi
+ fi
+
+ # Create or modify groups:
+ if [ "y" = "$exists" ]; then
+ if [ -z "$@" ]; then
+ echo "Do not execute groupmod (no changes)."
+ else
+ echo "Execute groupmod with \"$@\" for \"$name\""
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/sbin/groupmod "$@" "$name"
+ fi
+ else
+ echo "Execute groupadd with \"$@\" for \"$name\""
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/sbin/groupadd "$@" "$name"
+ fi
+ done
+
+ # Create users
+ list='${@" ".join(d.getVar('IMAGE_ACCOUNTS_USERS', True).split())} '
+ while true; do
+ # Pop first user entry:
+ list_rest="${list#*:*:*:*:*:*:*:*:*:*:* }"
+ entry="${list%%${list_rest}}"
+ list="${list_rest}"
+
+ if [ -z "${entry}" ]; then
+ break
+ fi
+
+ # Add colon to the end of the entry and remove trailing space:
+ entry="${entry% }:"
+
+ # Decode entries:
+ name="${entry%%:*}"
+ entry="${entry#${name}:}"
+
+ password="${entry%%:*}"
+ entry="${entry#${password}:}"
+
+ expire="${entry%%:*}"
+ entry="${entry#${expire}:}"
+
+ inactive="${entry%%:*}"
+ entry="${entry#${inactive}:}"
+
+ uid="${entry%%:*}"
+ entry="${entry#${uid}:}"
+
+ gid="${entry%%:*}"
+ entry="${entry#${gid}:}"
+
+ comment="${entry%%:*}"
+ entry="${entry#${comment}:}"
+
+ home="${entry%%:*}"
+ entry="${entry#${home}:}"
+
+ shell="${entry%%:*}"
+ entry="${entry#${shell}:}"
+
+ groups="${entry%%:*}"
+ entry="${entry#${groups}:}"
+
+ flags="${entry%%:*}"
+ entry="${entry#${flags}:}"
+
+ flags=",${flags}," # Needed for searching for substrings
+
+ # Check if user already exists:
+ if grep -q "^${name}:" '${ROOTFSDIR}/etc/passwd'; then
+ exists="y"
+ else
+ exists="n"
+ fi
+
+ # Create arguments:
+ set -- # clear arguments
+
+ if [ -n "$expire" ]; then
+ set -- "$@" --expiredate "$expire"
+ fi
+
+ if [ -n "$inactive" ]; then
+ set -- "$@" --inactive "$inactive"
+ fi
+
+ if [ -n "$uid" ]; then
+ set -- "$@" --uid "$uid"
+ fi
+
+ if [ -n "$gid" ]; then
+ set -- "$@" --gid "$gid"
+ fi
+
+ if [ -n "$comment" ]; then
+ set -- "$@" --comment "$comment"
+ fi
+
+ if [ -n "$home" ]; then
+ if [ "y" = "$exists" ]; then
+ set -- "$@" --home "$home" --move-home
+ else
+ set -- "$@" --home-dir "$home"
+ fi
+ fi
+
+ if [ -n "$shell" ]; then
+ set -- "$@" --shell "$shell"
+ fi
+
+ if [ -n "$groups" ]; then
+ set -- "$@" --groups "$groups"
+ fi
+
+ if [ "n" = "$exists" ]; then
+ if [ "${flags}" != "${flags%*,system,*}" ]; then
+ set -- "$@" --system
+ fi
+ if [ "${flags}" != "${flags%*,no-create-home,*}" ]; then
+ set -- "$@" --no-create-home
+ else
+ if [ "${flags}" != "${flags%*,create-home,*}" ]; then
+ set -- "$@" --create-home
+ fi
+ fi
+ fi
+
+ # Create or modify users:
+ if [ "y" = "$exists" ]; then
+ if [ -z "$@" ]; then
+ echo "Do not execute usermod (no changes)."
+ else
+ echo "Execute usermod with \"$@\" for \"$name\""
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/sbin/usermod "$@" "$name"
+ fi
+ else
+ echo "Execute useradd with \"$@\" for \"$name\""
+ sudo -E chroot '${ROOTFSDIR}' \
+ /usr/sbin/useradd "$@" "$name"
+ fi
+
+ # Set password:
+ if [ -n "$password" -o "${flags}" != "${flags%*,allow-empty-password,*}" ]; then
+ printf '%s:%s' "$name" "$password" | sudo chroot '${ROOTFSDIR}' \
+ /usr/sbin/chpasswd -e
+ fi
+ done
+}
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index dce6638..ef7983b 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -61,6 +61,7 @@ inherit image-cache-extension
inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
+inherit image-account-extension
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 5/8] doc: update description of image customization
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
` (3 preceding siblings ...)
2019-05-20 12:46 ` [PATCH v3 4/8] meta/classes: add image-account-extension class claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 6/8] doc: some fixes claudius.heine.ext
` (2 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Signed-off-by: Claudius Heine <ch@denx.de>
---
doc/user_manual.md | 51 ++++++++++++++++++++++++++++++++++++----------
1 file changed, 40 insertions(+), 11 deletions(-)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index dbbe6f6..4c9f4ae 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -314,16 +314,12 @@ Some other variables include:
- `IMAGE_INSTALL` - The list of custom packages to build and install to target image, please refer to relative chapter for more information.
- `BB_NUMBER_THREADS` - The number of `bitbake` jobs that can be run in parallel. Please set this option according your host CPU cores number.
- - `LOCALE_GEN` - A `\n` seperated list of `/etc/locale.gen` entries desired on the target.
- - `LOCALE_DEFAULT` - The default locale used for the `LANG` and `LANGUAGE` variable in `/etc/locale`.
- `HOST_DISTRO` - The distro to use for SDK root filesystem (so far limited only to `debian-stretch`). This variable is optional.
- `HOST_ARCH` - The Debian architecture of SDK root filesystem (e.g., `amd64`). By default set to current Debian host architecture. This variable is optional.
- `HOST_DISTRO_APT_SOURCES` - List of apt source files for SDK root filesystem. This variable is optional.
- `HOST_DISTRO_APT_PREFERENCES` - List of apt preference files for SDK root filesystem. This variable is optional.
- `DISTRO_APT_PREMIRRORS` - The preferred mirror (append it to the default URI in the format `ftp.debian.org my.preferred.mirror`. This variable is optional.
- `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt repos for apt installation after bootstrapping
- - `CFG_ROOT_PW` - The encrypted root password to be set. To encrypt password use `mkpasswd`. You find `mkpasswd` in the `whois` package of Debian. If the variable is empty, root login is passwordless.
- - `CFG_ROOT_LOCKED` - If set to `1` the root account will be locked.
---
@@ -496,17 +492,50 @@ Isar contains additional image type classes that can be used as reference:
## Customize and configure image
-Customization and configuration of an image should be done via packages, see below.
+Customization and configuration of an image can be done in two ways:
+
+ 1. Creating and adding a configuration package to `IMAGE_INSTALL`, or
+ 2. Changing the bitbake variables of the image recipe.
+
+In cases where configuration is not image specific, does not contain any secrets and can be shared between images, creating and adding a configuration package to `IMAGE_INSTALL` is the right option. This should be the case with most product specific configuration files.
+
+In cases where the configuration would contain secrets like user passwords, that would be world readable in `postinst`, etc. script files, some image extensions where created, that allow customization of those options from within the image recipe using bitbake variables. (e.g. user and group management and locale settings)
+
+### Locale configuration
+
+Two variables can be used to configure the locale installed on a image:
+
+ - `LOCALE_GEN` - A `\n` seperated list of `/etc/locale.gen` entries desired on the target.
+ - `LOCALE_DEFAULT` - The default locale used for the `LANG` and `LANGUAGE` variable in `/etc/locale`.
+
+### User and group configuration
+
+Groups can be created or modified using the `GROUPS` and `GROUP_<groupname>` variable or their flags.
+
+The `GROUPS` variable contains a space separated list of group names that should be modified or created. Each entry of this variable should have a corresponding `GROUP_<groupname>` variable.
-Adding those configuration packages to the image can be done in two ways:
+The `GROUP_<groupname>` variable contains the settings of a group named `groupname` in its flags. The following flags can be used:
- 1. Simply adding the package to `IMAGE_INSTALL`, like any other isar created package, or
- 2. Adding the package to `IMAGE_TRANSIENT_PACKAGES`.
+ - `gid` - The numeric group id.
+ - `flags` - A list of additional flags of the group. Those are the currently recognized flags:
+ - `system` - The group is created using the `--system` parameter.
-In most cases adding the configuration package to `IMAGE_INSTALL` is the right option.
+The `USERS` and `USER_<username>` variable works similar to the `GROUPS` and `GROUP_<groupname>` variable. The difference are the accepted flags of the `USER_<username>` variable. It accepts the following flags:
-In cases were the configuration script of the package has some external dependencies, that should not be part of the final image, then `IMAGE_TRANSIENT_PACKAGES` is the right option.
-Packages in the `IMAGE_TRANSIENT_PACKAGES` variable are installed to the image and purged in the next step. If such a configuration package deploys file as part of their content, then those files will be removed as well.
+ - `password` - The crypt(3) encrypted password. To encrypt a password use for example `mkpasswd` or `openssl passwd -6`. You can find `mkpasswd` in the `whois` package of Debian.
+ - `expire` - A `YYYY-MM-DD` formatted date on which the user account will be disabled. (see useradd(8))
+ - `inactive` - The number of days after a password expires until the account is permanently disabled. (see useradd(8))
+ - `uid` - The numeric user id.
+ - `gid` - The numeric group id or group name of this users initial login group.
+ - `comment` - This users comment field. Commonly the following format `full name,room number,work phone number,home phone number,other entry`.
+ - `home` - This users home directory
+ - `shell` - This users login shell
+ - `groups` - A space separated list of groups this user is a member of.
+ - `flags` - A list of additional flags of the user:
+ - `no-create-home` - `useradd` will be called with `-M` to prevent creation of the users home directory.
+ - `create-home` - `useradd` will be called with `-m` to force creation of the users home directory.
+ - `system` - `useradd` will be called with `--system`.
+ - `allow-empty-password` - Even if the `password` flag is empty, it will still be set. This results in a login without password.
---
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 6/8] doc: some fixes
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
` (4 preceding siblings ...)
2019-05-20 12:46 ` [PATCH v3 5/8] doc: update description of image customization claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 7/8] meta-isar: local.conf.sample: update root password and isar user creation claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 8/8] RECIPE-API-CHANGELOG: update transient package removal + root password claudius.heine.ext
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Signed-off-by: Claudius Heine <ch@denx.de>
---
doc/user_manual.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 4c9f4ae..ed181aa 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -314,7 +314,7 @@ Some other variables include:
- `IMAGE_INSTALL` - The list of custom packages to build and install to target image, please refer to relative chapter for more information.
- `BB_NUMBER_THREADS` - The number of `bitbake` jobs that can be run in parallel. Please set this option according your host CPU cores number.
- - `HOST_DISTRO` - The distro to use for SDK root filesystem (so far limited only to `debian-stretch`). This variable is optional.
+ - `HOST_DISTRO` - The distro to use for SDK root filesystem. This variable is optional.
- `HOST_ARCH` - The Debian architecture of SDK root filesystem (e.g., `amd64`). By default set to current Debian host architecture. This variable is optional.
- `HOST_DISTRO_APT_SOURCES` - List of apt source files for SDK root filesystem. This variable is optional.
- `HOST_DISTRO_APT_PREFERENCES` - List of apt preference files for SDK root filesystem. This variable is optional.
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 7/8] meta-isar: local.conf.sample: update root password and isar user creation
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
` (5 preceding siblings ...)
2019-05-20 12:46 ` [PATCH v3 6/8] doc: some fixes claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 8/8] RECIPE-API-CHANGELOG: update transient package removal + root password claudius.heine.ext
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Signed-off-by: Claudius Heine <ch@denx.de>
---
meta-isar/conf/local.conf.sample | 12 +++++++++++-
meta-isar/recipes-app/example-raw/files/postinst | 13 -------------
2 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 86e2384..b025a7f 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -174,4 +174,14 @@ ISAR_CROSS_COMPILE ?= "0"
# Password was encrypted using following command:
# mkpasswd -m sha512crypt -R 10000
# mkpasswd is part of the 'whois' package of Debian
-CFG_ROOT_PW ?= "$6$rounds=10000$RXeWrnFmkY$DtuS/OmsAS2cCEDo0BF5qQsizIrq6jPgXnwv3PHqREJeKd1sXdHX/ayQtuQWVDHe0KIO0/sVH8dvQm1KthF0d/"
+USERS += "root"
+USER_root[password] ??= "$6$rounds=10000$RXeWrnFmkY$DtuS/OmsAS2cCEDo0BF5qQsizIrq6jPgXnwv3PHqREJeKd1sXdHX/ayQtuQWVDHe0KIO0/sVH8dvQm1KthF0d/"
+
+GROUPS += "isar"
+GROUP_isar[flags] = "system"
+
+USERS += "isar"
+USER_isar[gid] = "isar"
+USER_isar[home] = "/var/lib/isar"
+USER_isar[comment] = "My isar user"
+USER_isar[flags] = "system create-home"
diff --git a/meta-isar/recipes-app/example-raw/files/postinst b/meta-isar/recipes-app/example-raw/files/postinst
index f48d993..416ff34 100644
--- a/meta-isar/recipes-app/example-raw/files/postinst
+++ b/meta-isar/recipes-app/example-raw/files/postinst
@@ -2,17 +2,4 @@
set -e
-if ! getent group isar >/dev/null; then
- addgroup --quiet --system isar
-fi
-
-if ! getent passwd isar >/dev/null; then
- useradd --system --gid isar --create-home \
- --home /var/lib/isar --no-user-group \
- --comment "My isar user" \
- isar
-fi
-
-chown -R isar:isar /var/lib/isar
-
echo "isar" > /etc/hostname
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 8/8] RECIPE-API-CHANGELOG: update transient package removal + root password
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
` (6 preceding siblings ...)
2019-05-20 12:46 ` [PATCH v3 7/8] meta-isar: local.conf.sample: update root password and isar user creation claudius.heine.ext
@ 2019-05-20 12:46 ` claudius.heine.ext
7 siblings, 0 replies; 11+ messages in thread
From: claudius.heine.ext @ 2019-05-20 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Signed-off-by: Claudius Heine <ch@denx.de>
---
RECIPE-API-CHANGELOG.md | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index d389d99..bbef1a3 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -154,3 +154,25 @@ Changes in v0.8
The content of `isar-image.bbclass` was moved to the `image.bbclass` file.
Recipes that inherit `isar-image` should be modified to inherit from `image`
instead.
+
+### Transient package support was removed
+
+The `LOCALE_GEN` and `LOCALE_DEFAULT` variables are now handled by the
+`image-locales-extension` class within the image recipe.
+
+Setting of the root password can now be done by the `image-account-extension`
+class within the image recipe. To set the root password to empty, you can
+use this code snippet:
+
+```
+USERS += "root"
+USER_root[password] = ""
+USER_root[flags] = "allow-empty-password"
+```
+
+Otherwise set a encrypted root password like this:
+
+```
+USERS += "root"
+USER_root[password] = "$6$rounds=10000$RXeWrnFmkY$DtuS/OmsAS2cCEDo0BF5qQsizIrq6jPgXnwv3PHqREJeKd1sXdHX/ayQtuQWVDHe0KIO0/sVH8dvQm1KthF0d/"
+```
--
2.20.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline
2019-05-20 12:46 ` [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline claudius.heine.ext
@ 2019-05-23 12:35 ` Henning Schild
2019-05-23 12:46 ` Claudius Heine
0 siblings, 1 reply; 11+ messages in thread
From: Henning Schild @ 2019-05-23 12:35 UTC (permalink / raw)
To: [ext] claudius.heine.ext@siemens.com; +Cc: isar-users, Claudius Heine
I was looking at the series and did not really find the place where the
IMAGE_TRANSIENT_PACKAGES features get removed. Just that p2 does not
append to the variable anymore.
For the first patch i would suggest a split. Splittling refactoring
changes from real functional ones ... I assume this patch does a
combination of both and the feature is dropped in here.
On top i would suggest a bbwarn if IMAGE_TRANSIENT_PACKAGES is
non-empty. That will give users of the feature a clue, maybe without
having to read the API changelog.
Henning
Am Mon, 20 May 2019 14:46:11 +0200
schrieb "[ext] claudius.heine.ext@siemens.com"
<claudius.heine.ext@siemens.com>:
> From: Claudius Heine <ch@denx.de>
>
> This removes the isar-bootstrap-helper.bbclass and integrates its
> functionality into the rootfs.bbclass.
>
> It introduces the rootfs_install task that runs through multiple
> functions from command lists in order to allow easy integration of
> custom functions.
>
> The introduced functions are, in general order of execution:
>
> rootfs_prepare: copy the isar-bootstrap rootfs base to workdir
> rootfs_configure_*: basic configuration of the root file system,
> before any custom packages where installed. (`/dev`, `/proc`, etc.
> are not mounted). Functions are added to
> `ROOTFS_CONFIGURE_COMMAND`. rootfs_do_mount: mount standard
> directories and repositories into the rootfs
> do_rootfs_install_*: downloading and installing packages. Functions
> are added to `ROOTFS_INSTALL_COMMAND`.
>
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
> meta/classes/image.bbclass | 24 +--
> meta/classes/isar-bootstrap-helper.bbclass | 143 ---------------
> meta/classes/rootfs.bbclass | 172
> +++++++++++++++++- .../isar-bootstrap/isar-bootstrap.inc |
> 5 + .../buildchroot/buildchroot-host.bb | 16 +-
> .../buildchroot/buildchroot-target.bb | 2 -
> .../buildchroot/buildchroot.inc | 26 ++-
> meta/recipes-devtools/sdkchroot/sdkchroot.bb | 48 ++---
> 8 files changed, 228 insertions(+), 208 deletions(-)
> delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 3eb261f..107b272 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -53,8 +53,10 @@ image_do_mounts() {
> buildchroot_do_mounts
> }
>
> -inherit isar-bootstrap-helper
> -ROOTFS_FEATURES += "finalize-rootfs"
> +ROOTFSDIR = "${IMAGE_ROOTFS}"
> +ROOTFS_FEATURES += "copy-package-cache clean-package-cache
> finalize-rootfs" +ROOTFS_PACKAGES += "${IMAGE_PREINSTALL}
> ${IMAGE_INSTALL}" +
> inherit rootfs
> inherit image-sdk-extension
> inherit image-cache-extension
> @@ -121,8 +123,10 @@ python set_image_size () {
> d.setVarFlag('ROOTFS_SIZE', 'export', '1')
> }
>
> -do_image_gen_fstab() {
> - cat > ${WORKDIR}/fstab << EOF
> +ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
> +image_configure_fstab[weight] = "2"
> +image_configure_fstab() {
> + sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
> # Begin /etc/fstab
> /dev/root / auto
> defaults 0 0
> proc /proc proc
> nosuid,noexec,nodev 0 0 @@ -134,18 +138,6 @@
> devtmpfs /dev devtmpfs
> mode=0755,nosuid 0 0 # End /etc/fstab EOF }
> -addtask image_gen_fstab before do_rootfs_install
> -
> -do_rootfs_install[depends] = "isar-apt:do_cache_config
> isar-bootstrap-target:do_bootstrap" -do_rootfs_install[deptask] =
> "do_deploy_deb" -do_rootfs_install[root_cleandirs] = "${IMAGE_ROOTFS}
> \
> - ${IMAGE_ROOTFS}/isar-apt"
> -do_rootfs_install() {
> - setup_root_file_system --clean --keep-apt-cache \
> - --fstab "${WORKDIR}/fstab" \
> - "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
> -}
> -addtask rootfs_install before do_build after do_unpack
>
> do_copy_boot_files[dirs] = "${DEPLOY_DIR_IMAGE}"
> do_copy_boot_files() {
> diff --git a/meta/classes/isar-bootstrap-helper.bbclass
> b/meta/classes/isar-bootstrap-helper.bbclass deleted file mode 100644
> index 8612be2..0000000
> --- a/meta/classes/isar-bootstrap-helper.bbclass
> +++ /dev/null
> @@ -1,143 +0,0 @@
> -# Helper functions for using isar-bootstrap
> -#
> -# This software is a part of ISAR.
> -# Copyright (c) Siemens AG, 2018
> -#
> -# SPDX-License-Identifier: MIT
> -
> -IMAGE_TRANSIENT_PACKAGES ??= ""
> -
> -def reverse_bb_array(d, varname):
> - array = d.getVar(varname, True)
> - if array is None:
> - return None
> - array = reversed(array.split())
> - return " ".join(i for i in array)
> -
> -
> -update_etc_os_release() {
> - OS_RELEASE_BUILD_ID=""
> - OS_RELEASE_VARIANT=""
> - while true; do
> - case "$1" in
> - --build-id) OS_RELEASE_BUILD_ID=$2; shift ;;
> - --variant) OS_RELEASE_VARIANT=$2; shift ;;
> - -*) bbfatal "$0: invalid option specified: $1" ;;
> - *) break ;;
> - esac
> - shift
> - done
> - ROOTFSDIR="$1"
> -
> - if [ -n "${OS_RELEASE_BUILD_ID}" ]; then
> - sudo sed -i '/^BUILD_ID=.*/d' ${ROOTFSDIR}/etc/os-release
> - echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \
> - sudo tee -a ${ROOTFSDIR}/etc/os-release
> - fi
> - if [ -n "${OS_RELEASE_VARIANT}" ]; then
> - sudo sed -i '/^VARIANT=.*/d' ${ROOTFSDIR}/etc/os-release
> - echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \
> - sudo tee -a ${ROOTFSDIR}/etc/os-release
> - fi
> -}
> -
> -setup_root_file_system() {
> - CLEAN=""
> - COPYISARAPT=""
> - FSTAB=""
> - ROOTFS_ARCH="${DISTRO_ARCH}"
> - ROOTFS_DISTRO="${DISTRO}"
> - while true; do
> - case "$1" in
> - --clean) CLEAN=1 ;;
> - --copyisarapt) COPYISARAPT=1 ;;
> - --fstab) FSTAB=$2; shift ;;
> - --host-arch) ROOTFS_ARCH="$2-$ROOTFS_ARCH"; shift ;;
> - --host-distro) ROOTFS_DISTRO="$2"; shift ;;
> - --keep-apt-cache) KEEP_APT_CACHE=1 ;;
> - -*) bbfatal "$0: invalid option specified: $1" ;;
> - *) break ;;
> - esac
> - shift
> - done
> - ROOTFSDIR="$1"
> - shift
> - PACKAGES="$@"
> - APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
> - CLEAN_FILES="${ROOTFSDIR}/etc/hostname
> ${ROOTFSDIR}/etc/resolv.conf" -
> - sudo cp -Trpfx \
> - "${DEPLOY_DIR_BOOTSTRAP}/$ROOTFS_DISTRO-$ROOTFS_ARCH/" \
> - "$ROOTFSDIR"
> - [ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee
> "$ROOTFSDIR/etc/fstab" -
> - echo "deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main"
> | \
> - sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list"
> >/dev/null -
> - echo "Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority:
> 1000" | \
> - sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
> -
> - if [ ${COPYISARAPT} ]; then
> - sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt
> - else
> - sudo mount --bind ${REPO_ISAR_DIR}/${DISTRO}
> $ROOTFSDIR/isar-apt
> - fi
> -
> - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then
> - sudo mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt
> - fi
> -
> - sudo mount --rbind /dev ${ROOTFSDIR}/dev
> - sudo mount --make-rslave ${ROOTFSDIR}/dev
> - sudo mount -t proc none $ROOTFSDIR/proc
> - sudo mount --rbind /sys ${ROOTFSDIR}/sys
> - sudo mount --make-rslave ${ROOTFSDIR}/sys
> -
> - # Refresh /etc/resolv.conf
> - sudo cp -L /etc/resolv.conf ${ROOTFSDIR}/etc
> -
> - # Install packages:
> - E="${@ bb.utils.export_proxies(d)}"
> - export DEBIAN_FRONTEND=noninteractive
> - # To avoid Perl locale warnings:
> - export LANG=C
> - export LANGUAGE=C
> - export LC_ALL=C
> - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \
> - -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
> - -o Dir::Etc::sourceparts="-" \
> - -o APT::Get::List-Cleanup="0"
> - # Add multiarch for cross-target
> - if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
> - sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture
> ${DISTRO_ARCH}
> - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
> - fi
> - sudo -E chroot "$ROOTFSDIR" \
> - /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
> - ${IMAGE_TRANSIENT_PACKAGES}
> - [ ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
> - sudo -E chroot "$ROOTFSDIR" \
> - /usr/bin/apt-get ${APT_ARGS} $PACKAGES
> - for pkg in ${IMAGE_TRANSIENT_PACKAGES}; do
> - sudo -E chroot "$ROOTFSDIR" \
> - /usr/bin/apt-get ${APT_ARGS} $pkg
> - done
> - for pkg in ${@reverse_bb_array(d, "IMAGE_TRANSIENT_PACKAGES") or
> ""}; do
> - sudo -E chroot "$ROOTFSDIR" \
> - /usr/bin/apt-get purge --yes $pkg
> - done
> - if [ ${CLEAN} ]; then
> - if [ ${KEEP_APT_CACHE} -eq 1 ]; then
> - mkdir -p ${WORKDIR}/apt_cache
> - sudo find ${ROOTFSDIR}/var/cache/apt/archives \
> - -maxdepth 1 -name '*.deb' -execdir /bin/mv -t
> ${WORKDIR}/apt_cache '{}' '+'
> - sudo chown -R $(whoami) ${WORKDIR}/apt_cache
> - fi
> - sudo -E chroot "$ROOTFSDIR" \
> - /usr/bin/apt-get autoremove --purge --yes
> - sudo -E chroot "$ROOTFSDIR" \
> - /usr/bin/apt-get clean
> - sudo "$ROOTFSDIR/chroot-setup.sh" "cleanup" "$ROOTFSDIR"
> - sudo rm -rf "$ROOTFSDIR/chroot-setup.sh"
> "$ROOTFSDIR/var/lib/apt/lists/"*
> - fi
> -}
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index 45b0350..0fc8ba8 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -1,14 +1,178 @@
> # This software is a part of ISAR.
> # Copyright (c) Siemens AG, 2019
>
> +ROOTFS_ARCH ?= "${DISTRO_ARCH}"
> +ROOTFS_DISTRO ?= "${DISTRO}"
> +ROOTFS_PACKAGES ?= ""
> +
> # Features of the rootfs creation:
> # available features are:
> +# 'deploy-package-cache' - copy the package cache
> ${WORKDIR}/apt_cache +# 'clean-package-cache' - delete package cache
> from rootfs # 'finalize-rootfs' - delete files needed to chroot into
> the rootfs ROOTFS_FEATURES ?= ""
>
> +ROOTFS_APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
> +
> +ROOTFS_CLEAN_FILES="/etc/hostname /etc/resolv.conf"
> +
> +# Useful environment variables:
> +export E = "${@ bb.utils.export_proxies(d)}"
> +export DEBIAN_FRONTEND = "noninteractive"
> +# To avoid Perl locale warnings:
> +export LANG = "C"
> +export LANGUAGE = "C"
> +export LC_ALL = "C"
> +
> +rootfs_do_mounts[weight] = "3"
> +rootfs_do_mounts() {
> + sudo -s <<'EOSUDO'
> + mountpoint -q '${ROOTFSDIR}/dev' || \
> + mount --rbind /dev '${ROOTFSDIR}/dev'
> + mount --make-rslave '${ROOTFSDIR}/dev'
> + mountpoint -q '${ROOTFSDIR}/proc' || \
> + mount -t proc none '${ROOTFSDIR}/proc'
> + mountpoint -q '${ROOTFSDIR}/sys' || \
> + mount --rbind /sys '${ROOTFSDIR}/sys'
> + mount --make-rslave '${ROOTFSDIR}/sys'
> +
> + # Mount isar-apt if the directory does not exist or if it is
> empty
> + # This prevents overwriting something that was copied there
> + if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \
> + [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1
> | wc -l)" = "0" ]
> + then
> + mkdir -p '${ROOTFSDIR}/isar-apt'
> + mountpoint -q '${ROOTFSDIR}/isar-apt' || \
> + mount --bind '${REPO_ISAR_DIR}/${DISTRO}'
> '${ROOTFSDIR}/isar-apt'
> + fi
> +
> + # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set
> + if
> [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}"
> = 'True' ]
> + then
> + mkdir -p '${ROOTFSDIR}/base-apt'
> + mountpoint -q '${ROOTFSDIR}/base-apt' || \
> + mount --bind '${REPO_BASE_DIR}'
> '${ROOTFSDIR}/base-apt'
> + fi
> +
> +EOSUDO
> +}
> +
> +rootfs_do_qemu() {
> + if [ '${@repr(d.getVar('ROOTFS_ARCH') ==
> d.getVar('HOST_ARCH'))}' = 'False' ]
> + then
> + test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \
> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static'
> '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
> + fi
> +}
> +
> +BOOTSTRAP_SRC =
> "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}-${DISTRO_ARCH}"
> +BOOTSTRAP_SRC_${ROOTFS_ARCH} =
> "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +
> +rootfs_prepare[weight] = "25" +rootfs_prepare(){
> + sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}'
> +}
> +
> +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt"
> +rootfs_configure_isar_apt[weight] = "2"
> +rootfs_configure_isar_apt() {
> + sudo -s <<'EOSUDO'
> +
> + mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
> + echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main'
> > \
> + '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
> +
> + mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
> + cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar'
> +Package: *
> +Pin: release n=${DEBDISTRONAME}
> +Pin-Priority: 1000
> +EOF
> +EOSUDO
> +}
> +
> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update"
> +rootfs_install_pkgs_update[weight] = "5"
> +rootfs_install_pkgs_update() {
> + sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
> + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
> + -o Dir::Etc::sourceparts="-" \
> + -o APT::Get::List-Cleanup="0"
> +}
> +ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf"
> +rootfs_install_resolvconf[weight] = "1"
> +rootfs_install_resolvconf() {
> + sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
> +}
> +
> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download"
> +rootfs_install_pkgs_download[weight] = "600"
> +rootfs_install_pkgs_download() {
> + sudo -E chroot '${ROOTFSDIR}' \
> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only
> ${ROOTFS_PACKAGES} +}
> +
> +ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if
> (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
> +rootfs_install_clean_files[weight] = "2"
> +rootfs_install_clean_files() {
> + sudo -E chroot '${ROOTFSDIR}' \
> + /bin/rm -f ${ROOTFS_CLEAN_FILES}
> +}
> +
> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_install"
> +rootfs_install_pkgs_install[weight] = "8000"
> +rootfs_install_pkgs_install() {
> + sudo -E chroot "${ROOTFSDIR}" \
> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
> +}
> +
> +do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
> +do_rootfs_install[vardeps] = "${ROOTFS_CONFIGURE_COMMAND}
> ${ROOTFS_INSTALL_COMMAND}" +do_rootfs_install[depends] =
> "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') ==
> d.getVar('DISTRO_ARCH') else 'host'}:do_build
> isar-apt:do_cache_config" +do_rootfs_install[deptask] =
> "do_deploy_deb" +python do_rootfs_install() {
> + configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or
> "").split()
> + install_cmds = (d.getVar("ROOTFS_INSTALL_COMMAND", True) or
> "").split() +
> + # Mount after configure commands, so that they have time to copy
> + # 'isar-apt' (sdkchroot):
> + cmds = ['rootfs_prepare'] + configure_cmds +
> ['rootfs_do_mounts'] + install_cmds +
> + stage_weights = [int(d.getVarFlag(i, 'weight', True) or "20")
> + for i in cmds]
> +
> + progress_reporter = bb.progress.MultiStageProgressReporter(d,
> stage_weights) +
> + for cmd in cmds:
> + progress_reporter.next_stage()
> + bb.build.exec_func(cmd, d)
> + progress_reporter.finish()
> +}
> +addtask rootfs_install before do_rootfs_postprocess after do_unpack
> +
> +ROOTFS_POSTPROCESS_COMMAND +=
> "${@bb.utils.contains('ROOTFS_FEATURES', 'copy-package-cache',
> 'rootfs_postprocess_copy_package_cache', '', d)}"
> +rootfs_postprocess_copy_package_cache() {
> + mkdir -p '${WORKDIR}/apt_cache'
> + sudo find '${ROOTFSDIR}/var/cache/apt/archives' \
> + -maxdepth 1 -name '*.deb' -execdir /bin/mv -t
> '${WORKDIR}/apt_cache' '{}' '+'
> + me="$(id -u):$(id -g)"
> + sudo chown -R "$me" '${WORKDIR}/apt_cache'
> +}
> +
> +ROOTFS_POSTPROCESS_COMMAND +=
> "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache',
> 'rootfs_postprocess_clean_package_cache', '', d)}"
> +rootfs_postprocess_clean_package_cache() {
> + sudo -E chroot '${ROOTFSDIR}' \
> + /usr/bin/apt-get clean
> + sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
> +}
> +
> ROOTFS_POSTPROCESS_COMMAND +=
> "${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs',
> 'rootfs_postprocess_finalize', '', d)}" rootfs_postprocess_finalize()
> { sudo -s <<'EOSUDO'
> + test -e "${ROOTFSDIR}/chroot-setup.sh" && \
> + "${ROOTFSDIR}/chroot-setup.sh" "cleanup" "${ROOTFSDIR}"
> + rm -f "${ROOTFSDIR}/chroot-setup.sh"
> +
> test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" && \
> find "${ROOTFSDIR}/usr/bin" \
> -maxdepth 1 -name 'qemu-*-static' -type f -delete
> @@ -42,6 +206,12 @@ EOSUDO
>
> do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
> python do_rootfs_postprocess() {
> + # Take care that its correctly mounted:
> + bb.build.exec_func('rootfs_do_mounts', d)
> + # Take care that qemu-*-static is available, since it could have
> been
> + # removed on a previous execution of this task:
> + bb.build.exec_func('rootfs_do_qemu', d)
> +
> cmds = d.getVar("ROOTFS_POSTPROCESS_COMMAND")
> if cmds is None or not cmds.strip():
> return
> @@ -49,7 +219,7 @@ python do_rootfs_postprocess() {
> for cmd in cmds:
> bb.build.exec_func(cmd, d)
> }
> -addtask rootfs_postprocess before do_rootfs after do_rootfs_install
> +addtask rootfs_postprocess before do_rootfs
>
> python do_rootfs() {
> """Virtual task"""
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
> b740c57..f8741c2 100644 ---
> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -327,6
> +327,11 @@ isar_bootstrap() { mount --make-rslave ${ROOTFSDIR}/sys
>
> export DEBIAN_FRONTEND=noninteractive
> +
> + if [ ${IS_HOST} ]; then
> + chroot "${ROOTFSDIR}" /usr/bin/dpkg
> --add-architecture ${DISTRO_ARCH}
> + fi
> +
> chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
> -o Debug::pkgProblemResolver=yes
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index
> 73cd548..121992c 100644 ---
> a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++
> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -7,6 +7,8
> @@ DESCRIPTION = "Isar development filesystem for host" PF =
> "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
> require buildchroot.inc
> +ROOTFS_ARCH = "${HOST_ARCH}"
> +ROOTFS_DISTRO = "${HOST_DISTRO}"
>
> BUILDCHROOT_PREINSTALL ?= "make \
> debhelper \
> @@ -18,15 +20,5 @@ BUILDCHROOT_PREINSTALL ?= "make \
> automake \
> devscripts \
> equivs \
> - libc6:${DISTRO_ARCH}"
> -
> -# According to the wiki page:
> -# https://wiki.debian.org/CrossToolchains
> -BUILDCHROOT_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf
> \
> - crossbuild-essential-armhf"
> -BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \
> - crossbuild-essential-arm64"
> -
> -
> -PARAMS = "--host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}'"
> -do_build[depends] = "isar-apt:do_cache_config
> isar-bootstrap-host:do_bootstrap"
> + libc6:${DISTRO_ARCH} \
> + crossbuild-essential-${DISTRO_ARCH}"
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb
> b/meta/recipes-devtools/buildchroot/buildchroot-target.bb index
> 20f4d23..5a01258 100644 ---
> a/meta/recipes-devtools/buildchroot/buildchroot-target.bb +++
> b/meta/recipes-devtools/buildchroot/buildchroot-target.bb @@ -21,5
> +21,3 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ devscripts \
> equivs \
> adduser"
> -
> -do_build[depends] = "isar-apt:do_cache_config
> isar-bootstrap-target:do_bootstrap" diff --git
> a/meta/recipes-devtools/buildchroot/buildchroot.inc
> b/meta/recipes-devtools/buildchroot/buildchroot.inc index
> ca9ce51..40fc8b0 100644 ---
> a/meta/recipes-devtools/buildchroot/buildchroot.inc +++
> b/meta/recipes-devtools/buildchroot/buildchroot.inc @@ -13,18 +13,25
> @@ SRC_URI = "file://configscript.sh \ file://deps.sh" PV = "1.0"
>
> -inherit isar-bootstrap-helper
> +inherit rootfs
>
> BUILDCHROOT_DIR = "${WORKDIR}/rootfs"
> +ROOTFSDIR = "${BUILDCHROOT_DIR}"
> +ROOTFS_PACKAGES = "${BUILDCHROOT_PREINSTALL}"
> +ROOTFS_CLEAN_FILES = ""
> +
> +rootfs_do_mounts_append() {
> + sudo -s <<'EOSUDO'
> + mkdir -p '${BUILDCHROOT_DIR}/downloads'
> + mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \
> + mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads'
> +EOSUDO
> +}
>
> -do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
> - ${BUILDCHROOT_DIR}/isar-apt \
> - ${BUILDCHROOT_DIR}/downloads \
> - ${BUILDCHROOT_DIR}/home/builder"
> -
> -do_build() {
> - setup_root_file_system ${PARAMS} ${BUILDCHROOT_DIR}
> ${BUILDCHROOT_PREINSTALL} -
> +ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files"
> +buildchroot_install_files() {
> + sudo mkdir -p "${BUILDCHROOT_DIR}/home/builder"
> + sudo mkdir -p "${BUILDCHROOT_DIR}/isar-apt"
> # Install package builder script
> sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder"
> sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar
> @@ -38,5 +45,4 @@ do_build() {
> GROUP_ID=$(id -g)
> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh $USER_ID
> $GROUP_ID
> - sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads
> }
> diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> b/meta/recipes-devtools/sdkchroot/sdkchroot.bb index 49aeb47..57d52ba
> 100644 --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
> @@ -12,10 +12,25 @@ SRC_URI = " \
> file://configscript.sh \
> file://README.sdk"
> PV = "0.1"
> -
> -inherit isar-bootstrap-helper
> PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
>
> +TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
> +TOOLCHAIN_${HOST_ARCH} = "build-essential"
> +TOOLCHAIN_i386 = "build-essential"
> +
> +inherit rootfs
> +ROOTFS_ARCH = "${HOST_ARCH}"
> +ROOTFS_DISTRO = "${HOST_DISTRO}"
> +ROOTFSDIR = "${S}"
> +ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}"
> +ROOTFS_FEATURES += "copy-package-cache"
> +
> +python() {
> + if d.getVar("HOST_ARCH") not in ['i386', 'amd64']:
> + raise bb.parse.SkipRecipe("SDK doesn't support {} as
> host".format(
> + d.getVar("ROOTFS_ARCH")))
> +}
> +
> SDKCHROOT_PREINSTALL := "debhelper \
> autotools-dev \
> dpkg \
> @@ -28,29 +43,14 @@ SDKCHROOT_PREINSTALL := "debhelper \
>
> S = "${WORKDIR}/rootfs"
>
> -do_build[dirs] = "${DEPLOY_DIR_IMAGE}"
> -do_build[root_cleandirs] = "${S} \
> - ${S}/isar-apt"
> -
> -do_build[depends] = "isar-apt:do_cache_config
> isar-bootstrap-host:do_bootstrap" -
> -do_build() {
> -
> - if [ ${HOST_DISTRO} != "debian-stretch" ]; then
> - bbfatal "SDK doesn't support ${HOST_DISTRO}"
> - fi
> - if [ ${HOST_ARCH} != "i386" -a ${HOST_ARCH} != "amd64" ]; then
> - bbfatal "SDK doesn't support ${HOST_ARCH} as host"
> - fi
> -
> - if [ ${HOST_ARCH} = ${DISTRO_ARCH} -o ${DISTRO_ARCH} = "i386" ];
> then
> - packages="${SDKCHROOT_PREINSTALL} build-essential"
> - else
> - packages="${SDKCHROOT_PREINSTALL}
> crossbuild-essential-${DISTRO_ARCH}"
> - fi
> -
> - setup_root_file_system --copyisarapt --host-arch '${HOST_ARCH}'
> --host-distro '${HOST_DISTRO}' "${S}" $packages
> +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir"
> +rootfs_configure_isar_apt_dir() {
> + # Copy isar-apt instead of mounting:
> + sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
> +}
>
> +ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
> +sdkchroot_install_files() {
> # Configure root filesystem
> sudo install -m 644 ${WORKDIR}/README.sdk ${S}
> sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline
2019-05-23 12:35 ` Henning Schild
@ 2019-05-23 12:46 ` Claudius Heine
0 siblings, 0 replies; 11+ messages in thread
From: Claudius Heine @ 2019-05-23 12:46 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Claudius Heine
Hi Henning,
On 23/05/2019 14.35, Henning Schild wrote:
> I was looking at the series and did not really find the place where the
> IMAGE_TRANSIENT_PACKAGES features get removed. Just that p2 does not
> append to the variable anymore.
>
> For the first patch i would suggest a split. Splittling refactoring
> changes from real functional ones ... I assume this patch does a
> combination of both and the feature is dropped in here.
That would mean I have to reimplement the TRANSIENT package feature for
the new system in this patch and then remove again in the next.
While this would be a clean approach, I just didn't want to spend the
time doing that.
Hmmm.. But if you insist, then I can switch it around, and remove it
first and then split it without that feature.
>
> On top i would suggest a bbwarn if IMAGE_TRANSIENT_PACKAGES is
> non-empty. That will give users of the feature a clue, maybe without
> having to read the API changelog.
Ok, I can do that in an additional patch. But I don't want to send a new
version of that patchset just for that.
Claudius
>
> Henning
>
> Am Mon, 20 May 2019 14:46:11 +0200
> schrieb "[ext] claudius.heine.ext@siemens.com"
> <claudius.heine.ext@siemens.com>:
>
>> From: Claudius Heine <ch@denx.de>
>>
>> This removes the isar-bootstrap-helper.bbclass and integrates its
>> functionality into the rootfs.bbclass.
>>
>> It introduces the rootfs_install task that runs through multiple
>> functions from command lists in order to allow easy integration of
>> custom functions.
>>
>> The introduced functions are, in general order of execution:
>>
>> rootfs_prepare: copy the isar-bootstrap rootfs base to workdir
>> rootfs_configure_*: basic configuration of the root file system,
>> before any custom packages where installed. (`/dev`, `/proc`, etc.
>> are not mounted). Functions are added to
>> `ROOTFS_CONFIGURE_COMMAND`. rootfs_do_mount: mount standard
>> directories and repositories into the rootfs
>> do_rootfs_install_*: downloading and installing packages. Functions
>> are added to `ROOTFS_INSTALL_COMMAND`.
>>
>> Signed-off-by: Claudius Heine <ch@denx.de>
>> ---
>> meta/classes/image.bbclass | 24 +--
>> meta/classes/isar-bootstrap-helper.bbclass | 143 ---------------
>> meta/classes/rootfs.bbclass | 172
>> +++++++++++++++++- .../isar-bootstrap/isar-bootstrap.inc |
>> 5 + .../buildchroot/buildchroot-host.bb | 16 +-
>> .../buildchroot/buildchroot-target.bb | 2 -
>> .../buildchroot/buildchroot.inc | 26 ++-
>> meta/recipes-devtools/sdkchroot/sdkchroot.bb | 48 ++---
>> 8 files changed, 228 insertions(+), 208 deletions(-)
>> delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass
>>
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 3eb261f..107b272 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -53,8 +53,10 @@ image_do_mounts() {
>> buildchroot_do_mounts
>> }
>>
>> -inherit isar-bootstrap-helper
>> -ROOTFS_FEATURES += "finalize-rootfs"
>> +ROOTFSDIR = "${IMAGE_ROOTFS}"
>> +ROOTFS_FEATURES += "copy-package-cache clean-package-cache
>> finalize-rootfs" +ROOTFS_PACKAGES += "${IMAGE_PREINSTALL}
>> ${IMAGE_INSTALL}" +
>> inherit rootfs
>> inherit image-sdk-extension
>> inherit image-cache-extension
>> @@ -121,8 +123,10 @@ python set_image_size () {
>> d.setVarFlag('ROOTFS_SIZE', 'export', '1')
>> }
>>
>> -do_image_gen_fstab() {
>> - cat > ${WORKDIR}/fstab << EOF
>> +ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
>> +image_configure_fstab[weight] = "2"
>> +image_configure_fstab() {
>> + sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
>> # Begin /etc/fstab
>> /dev/root / auto
>> defaults 0 0
>> proc /proc proc
>> nosuid,noexec,nodev 0 0 @@ -134,18 +138,6 @@
>> devtmpfs /dev devtmpfs
>> mode=0755,nosuid 0 0 # End /etc/fstab EOF }
>> -addtask image_gen_fstab before do_rootfs_install
>> -
>> -do_rootfs_install[depends] = "isar-apt:do_cache_config
>> isar-bootstrap-target:do_bootstrap" -do_rootfs_install[deptask] =
>> "do_deploy_deb" -do_rootfs_install[root_cleandirs] = "${IMAGE_ROOTFS}
>> \
>> - ${IMAGE_ROOTFS}/isar-apt"
>> -do_rootfs_install() {
>> - setup_root_file_system --clean --keep-apt-cache \
>> - --fstab "${WORKDIR}/fstab" \
>> - "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL}
>> -}
>> -addtask rootfs_install before do_build after do_unpack
>>
>> do_copy_boot_files[dirs] = "${DEPLOY_DIR_IMAGE}"
>> do_copy_boot_files() {
>> diff --git a/meta/classes/isar-bootstrap-helper.bbclass
>> b/meta/classes/isar-bootstrap-helper.bbclass deleted file mode 100644
>> index 8612be2..0000000
>> --- a/meta/classes/isar-bootstrap-helper.bbclass
>> +++ /dev/null
>> @@ -1,143 +0,0 @@
>> -# Helper functions for using isar-bootstrap
>> -#
>> -# This software is a part of ISAR.
>> -# Copyright (c) Siemens AG, 2018
>> -#
>> -# SPDX-License-Identifier: MIT
>> -
>> -IMAGE_TRANSIENT_PACKAGES ??= ""
>> -
>> -def reverse_bb_array(d, varname):
>> - array = d.getVar(varname, True)
>> - if array is None:
>> - return None
>> - array = reversed(array.split())
>> - return " ".join(i for i in array)
>> -
>> -
>> -update_etc_os_release() {
>> - OS_RELEASE_BUILD_ID=""
>> - OS_RELEASE_VARIANT=""
>> - while true; do
>> - case "$1" in
>> - --build-id) OS_RELEASE_BUILD_ID=$2; shift ;;
>> - --variant) OS_RELEASE_VARIANT=$2; shift ;;
>> - -*) bbfatal "$0: invalid option specified: $1" ;;
>> - *) break ;;
>> - esac
>> - shift
>> - done
>> - ROOTFSDIR="$1"
>> -
>> - if [ -n "${OS_RELEASE_BUILD_ID}" ]; then
>> - sudo sed -i '/^BUILD_ID=.*/d' ${ROOTFSDIR}/etc/os-release
>> - echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \
>> - sudo tee -a ${ROOTFSDIR}/etc/os-release
>> - fi
>> - if [ -n "${OS_RELEASE_VARIANT}" ]; then
>> - sudo sed -i '/^VARIANT=.*/d' ${ROOTFSDIR}/etc/os-release
>> - echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \
>> - sudo tee -a ${ROOTFSDIR}/etc/os-release
>> - fi
>> -}
>> -
>> -setup_root_file_system() {
>> - CLEAN=""
>> - COPYISARAPT=""
>> - FSTAB=""
>> - ROOTFS_ARCH="${DISTRO_ARCH}"
>> - ROOTFS_DISTRO="${DISTRO}"
>> - while true; do
>> - case "$1" in
>> - --clean) CLEAN=1 ;;
>> - --copyisarapt) COPYISARAPT=1 ;;
>> - --fstab) FSTAB=$2; shift ;;
>> - --host-arch) ROOTFS_ARCH="$2-$ROOTFS_ARCH"; shift ;;
>> - --host-distro) ROOTFS_DISTRO="$2"; shift ;;
>> - --keep-apt-cache) KEEP_APT_CACHE=1 ;;
>> - -*) bbfatal "$0: invalid option specified: $1" ;;
>> - *) break ;;
>> - esac
>> - shift
>> - done
>> - ROOTFSDIR="$1"
>> - shift
>> - PACKAGES="$@"
>> - APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
>> - CLEAN_FILES="${ROOTFSDIR}/etc/hostname
>> ${ROOTFSDIR}/etc/resolv.conf" -
>> - sudo cp -Trpfx \
>> - "${DEPLOY_DIR_BOOTSTRAP}/$ROOTFS_DISTRO-$ROOTFS_ARCH/" \
>> - "$ROOTFSDIR"
>> - [ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee
>> "$ROOTFSDIR/etc/fstab" -
>> - echo "deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main"
>> | \
>> - sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list"
>>> /dev/null -
>> - echo "Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority:
>> 1000" | \
>> - sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
>> -
>> - if [ ${COPYISARAPT} ]; then
>> - sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt
>> - else
>> - sudo mount --bind ${REPO_ISAR_DIR}/${DISTRO}
>> $ROOTFSDIR/isar-apt
>> - fi
>> -
>> - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then
>> - sudo mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt
>> - fi
>> -
>> - sudo mount --rbind /dev ${ROOTFSDIR}/dev
>> - sudo mount --make-rslave ${ROOTFSDIR}/dev
>> - sudo mount -t proc none $ROOTFSDIR/proc
>> - sudo mount --rbind /sys ${ROOTFSDIR}/sys
>> - sudo mount --make-rslave ${ROOTFSDIR}/sys
>> -
>> - # Refresh /etc/resolv.conf
>> - sudo cp -L /etc/resolv.conf ${ROOTFSDIR}/etc
>> -
>> - # Install packages:
>> - E="${@ bb.utils.export_proxies(d)}"
>> - export DEBIAN_FRONTEND=noninteractive
>> - # To avoid Perl locale warnings:
>> - export LANG=C
>> - export LANGUAGE=C
>> - export LC_ALL=C
>> - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \
>> - -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
>> - -o Dir::Etc::sourceparts="-" \
>> - -o APT::Get::List-Cleanup="0"
>> - # Add multiarch for cross-target
>> - if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then
>> - sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture
>> ${DISTRO_ARCH}
>> - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update
>> - fi
>> - sudo -E chroot "$ROOTFSDIR" \
>> - /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \
>> - ${IMAGE_TRANSIENT_PACKAGES}
>> - [ ${CLEAN} ] && sudo rm -f ${CLEAN_FILES}
>> - sudo -E chroot "$ROOTFSDIR" \
>> - /usr/bin/apt-get ${APT_ARGS} $PACKAGES
>> - for pkg in ${IMAGE_TRANSIENT_PACKAGES}; do
>> - sudo -E chroot "$ROOTFSDIR" \
>> - /usr/bin/apt-get ${APT_ARGS} $pkg
>> - done
>> - for pkg in ${@reverse_bb_array(d, "IMAGE_TRANSIENT_PACKAGES") or
>> ""}; do
>> - sudo -E chroot "$ROOTFSDIR" \
>> - /usr/bin/apt-get purge --yes $pkg
>> - done
>> - if [ ${CLEAN} ]; then
>> - if [ ${KEEP_APT_CACHE} -eq 1 ]; then
>> - mkdir -p ${WORKDIR}/apt_cache
>> - sudo find ${ROOTFSDIR}/var/cache/apt/archives \
>> - -maxdepth 1 -name '*.deb' -execdir /bin/mv -t
>> ${WORKDIR}/apt_cache '{}' '+'
>> - sudo chown -R $(whoami) ${WORKDIR}/apt_cache
>> - fi
>> - sudo -E chroot "$ROOTFSDIR" \
>> - /usr/bin/apt-get autoremove --purge --yes
>> - sudo -E chroot "$ROOTFSDIR" \
>> - /usr/bin/apt-get clean
>> - sudo "$ROOTFSDIR/chroot-setup.sh" "cleanup" "$ROOTFSDIR"
>> - sudo rm -rf "$ROOTFSDIR/chroot-setup.sh"
>> "$ROOTFSDIR/var/lib/apt/lists/"*
>> - fi
>> -}
>> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
>> index 45b0350..0fc8ba8 100644
>> --- a/meta/classes/rootfs.bbclass
>> +++ b/meta/classes/rootfs.bbclass
>> @@ -1,14 +1,178 @@
>> # This software is a part of ISAR.
>> # Copyright (c) Siemens AG, 2019
>>
>> +ROOTFS_ARCH ?= "${DISTRO_ARCH}"
>> +ROOTFS_DISTRO ?= "${DISTRO}"
>> +ROOTFS_PACKAGES ?= ""
>> +
>> # Features of the rootfs creation:
>> # available features are:
>> +# 'deploy-package-cache' - copy the package cache
>> ${WORKDIR}/apt_cache +# 'clean-package-cache' - delete package cache
>> from rootfs # 'finalize-rootfs' - delete files needed to chroot into
>> the rootfs ROOTFS_FEATURES ?= ""
>>
>> +ROOTFS_APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
>> +
>> +ROOTFS_CLEAN_FILES="/etc/hostname /etc/resolv.conf"
>> +
>> +# Useful environment variables:
>> +export E = "${@ bb.utils.export_proxies(d)}"
>> +export DEBIAN_FRONTEND = "noninteractive"
>> +# To avoid Perl locale warnings:
>> +export LANG = "C"
>> +export LANGUAGE = "C"
>> +export LC_ALL = "C"
>> +
>> +rootfs_do_mounts[weight] = "3"
>> +rootfs_do_mounts() {
>> + sudo -s <<'EOSUDO'
>> + mountpoint -q '${ROOTFSDIR}/dev' || \
>> + mount --rbind /dev '${ROOTFSDIR}/dev'
>> + mount --make-rslave '${ROOTFSDIR}/dev'
>> + mountpoint -q '${ROOTFSDIR}/proc' || \
>> + mount -t proc none '${ROOTFSDIR}/proc'
>> + mountpoint -q '${ROOTFSDIR}/sys' || \
>> + mount --rbind /sys '${ROOTFSDIR}/sys'
>> + mount --make-rslave '${ROOTFSDIR}/sys'
>> +
>> + # Mount isar-apt if the directory does not exist or if it is
>> empty
>> + # This prevents overwriting something that was copied there
>> + if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \
>> + [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1
>> | wc -l)" = "0" ]
>> + then
>> + mkdir -p '${ROOTFSDIR}/isar-apt'
>> + mountpoint -q '${ROOTFSDIR}/isar-apt' || \
>> + mount --bind '${REPO_ISAR_DIR}/${DISTRO}'
>> '${ROOTFSDIR}/isar-apt'
>> + fi
>> +
>> + # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set
>> + if
>> [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}"
>> = 'True' ]
>> + then
>> + mkdir -p '${ROOTFSDIR}/base-apt'
>> + mountpoint -q '${ROOTFSDIR}/base-apt' || \
>> + mount --bind '${REPO_BASE_DIR}'
>> '${ROOTFSDIR}/base-apt'
>> + fi
>> +
>> +EOSUDO
>> +}
>> +
>> +rootfs_do_qemu() {
>> + if [ '${@repr(d.getVar('ROOTFS_ARCH') ==
>> d.getVar('HOST_ARCH'))}' = 'False' ]
>> + then
>> + test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \
>> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static'
>> '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
>> + fi
>> +}
>> +
>> +BOOTSTRAP_SRC =
>> "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}-${DISTRO_ARCH}"
>> +BOOTSTRAP_SRC_${ROOTFS_ARCH} =
>> "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +
>> +rootfs_prepare[weight] = "25" +rootfs_prepare(){
>> + sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}'
>> +}
>> +
>> +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt"
>> +rootfs_configure_isar_apt[weight] = "2"
>> +rootfs_configure_isar_apt() {
>> + sudo -s <<'EOSUDO'
>> +
>> + mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
>> + echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main'
>>> \
>> + '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
>> +
>> + mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
>> + cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar'
>> +Package: *
>> +Pin: release n=${DEBDISTRONAME}
>> +Pin-Priority: 1000
>> +EOF
>> +EOSUDO
>> +}
>> +
>> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update"
>> +rootfs_install_pkgs_update[weight] = "5"
>> +rootfs_install_pkgs_update() {
>> + sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
>> + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
>> + -o Dir::Etc::sourceparts="-" \
>> + -o APT::Get::List-Cleanup="0"
>> +}
>> +ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf"
>> +rootfs_install_resolvconf[weight] = "1"
>> +rootfs_install_resolvconf() {
>> + sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
>> +}
>> +
>> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download"
>> +rootfs_install_pkgs_download[weight] = "600"
>> +rootfs_install_pkgs_download() {
>> + sudo -E chroot '${ROOTFSDIR}' \
>> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only
>> ${ROOTFS_PACKAGES} +}
>> +
>> +ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if
>> (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
>> +rootfs_install_clean_files[weight] = "2"
>> +rootfs_install_clean_files() {
>> + sudo -E chroot '${ROOTFSDIR}' \
>> + /bin/rm -f ${ROOTFS_CLEAN_FILES}
>> +}
>> +
>> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_install"
>> +rootfs_install_pkgs_install[weight] = "8000"
>> +rootfs_install_pkgs_install() {
>> + sudo -E chroot "${ROOTFSDIR}" \
>> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
>> +}
>> +
>> +do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
>> +do_rootfs_install[vardeps] = "${ROOTFS_CONFIGURE_COMMAND}
>> ${ROOTFS_INSTALL_COMMAND}" +do_rootfs_install[depends] =
>> "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') ==
>> d.getVar('DISTRO_ARCH') else 'host'}:do_build
>> isar-apt:do_cache_config" +do_rootfs_install[deptask] =
>> "do_deploy_deb" +python do_rootfs_install() {
>> + configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or
>> "").split()
>> + install_cmds = (d.getVar("ROOTFS_INSTALL_COMMAND", True) or
>> "").split() +
>> + # Mount after configure commands, so that they have time to copy
>> + # 'isar-apt' (sdkchroot):
>> + cmds = ['rootfs_prepare'] + configure_cmds +
>> ['rootfs_do_mounts'] + install_cmds +
>> + stage_weights = [int(d.getVarFlag(i, 'weight', True) or "20")
>> + for i in cmds]
>> +
>> + progress_reporter = bb.progress.MultiStageProgressReporter(d,
>> stage_weights) +
>> + for cmd in cmds:
>> + progress_reporter.next_stage()
>> + bb.build.exec_func(cmd, d)
>> + progress_reporter.finish()
>> +}
>> +addtask rootfs_install before do_rootfs_postprocess after do_unpack
>> +
>> +ROOTFS_POSTPROCESS_COMMAND +=
>> "${@bb.utils.contains('ROOTFS_FEATURES', 'copy-package-cache',
>> 'rootfs_postprocess_copy_package_cache', '', d)}"
>> +rootfs_postprocess_copy_package_cache() {
>> + mkdir -p '${WORKDIR}/apt_cache'
>> + sudo find '${ROOTFSDIR}/var/cache/apt/archives' \
>> + -maxdepth 1 -name '*.deb' -execdir /bin/mv -t
>> '${WORKDIR}/apt_cache' '{}' '+'
>> + me="$(id -u):$(id -g)"
>> + sudo chown -R "$me" '${WORKDIR}/apt_cache'
>> +}
>> +
>> +ROOTFS_POSTPROCESS_COMMAND +=
>> "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache',
>> 'rootfs_postprocess_clean_package_cache', '', d)}"
>> +rootfs_postprocess_clean_package_cache() {
>> + sudo -E chroot '${ROOTFSDIR}' \
>> + /usr/bin/apt-get clean
>> + sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
>> +}
>> +
>> ROOTFS_POSTPROCESS_COMMAND +=
>> "${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs',
>> 'rootfs_postprocess_finalize', '', d)}" rootfs_postprocess_finalize()
>> { sudo -s <<'EOSUDO'
>> + test -e "${ROOTFSDIR}/chroot-setup.sh" && \
>> + "${ROOTFSDIR}/chroot-setup.sh" "cleanup" "${ROOTFSDIR}"
>> + rm -f "${ROOTFSDIR}/chroot-setup.sh"
>> +
>> test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" && \
>> find "${ROOTFSDIR}/usr/bin" \
>> -maxdepth 1 -name 'qemu-*-static' -type f -delete
>> @@ -42,6 +206,12 @@ EOSUDO
>>
>> do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
>> python do_rootfs_postprocess() {
>> + # Take care that its correctly mounted:
>> + bb.build.exec_func('rootfs_do_mounts', d)
>> + # Take care that qemu-*-static is available, since it could have
>> been
>> + # removed on a previous execution of this task:
>> + bb.build.exec_func('rootfs_do_qemu', d)
>> +
>> cmds = d.getVar("ROOTFS_POSTPROCESS_COMMAND")
>> if cmds is None or not cmds.strip():
>> return
>> @@ -49,7 +219,7 @@ python do_rootfs_postprocess() {
>> for cmd in cmds:
>> bb.build.exec_func(cmd, d)
>> }
>> -addtask rootfs_postprocess before do_rootfs after do_rootfs_install
>> +addtask rootfs_postprocess before do_rootfs
>>
>> python do_rootfs() {
>> """Virtual task"""
>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
>> b740c57..f8741c2 100644 ---
>> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -327,6
>> +327,11 @@ isar_bootstrap() { mount --make-rslave ${ROOTFSDIR}/sys
>>
>> export DEBIAN_FRONTEND=noninteractive
>> +
>> + if [ ${IS_HOST} ]; then
>> + chroot "${ROOTFSDIR}" /usr/bin/dpkg
>> --add-architecture ${DISTRO_ARCH}
>> + fi
>> +
>> chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
>> chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
>> -o Debug::pkgProblemResolver=yes
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index
>> 73cd548..121992c 100644 ---
>> a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++
>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -7,6 +7,8
>> @@ DESCRIPTION = "Isar development filesystem for host" PF =
>> "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
>> require buildchroot.inc
>> +ROOTFS_ARCH = "${HOST_ARCH}"
>> +ROOTFS_DISTRO = "${HOST_DISTRO}"
>>
>> BUILDCHROOT_PREINSTALL ?= "make \
>> debhelper \
>> @@ -18,15 +20,5 @@ BUILDCHROOT_PREINSTALL ?= "make \
>> automake \
>> devscripts \
>> equivs \
>> - libc6:${DISTRO_ARCH}"
>> -
>> -# According to the wiki page:
>> -# https://wiki.debian.org/CrossToolchains
>> -BUILDCHROOT_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf
>> \
>> - crossbuild-essential-armhf"
>> -BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \
>> - crossbuild-essential-arm64"
>> -
>> -
>> -PARAMS = "--host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}'"
>> -do_build[depends] = "isar-apt:do_cache_config
>> isar-bootstrap-host:do_bootstrap"
>> + libc6:${DISTRO_ARCH} \
>> + crossbuild-essential-${DISTRO_ARCH}"
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb
>> b/meta/recipes-devtools/buildchroot/buildchroot-target.bb index
>> 20f4d23..5a01258 100644 ---
>> a/meta/recipes-devtools/buildchroot/buildchroot-target.bb +++
>> b/meta/recipes-devtools/buildchroot/buildchroot-target.bb @@ -21,5
>> +21,3 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ devscripts \
>> equivs \
>> adduser"
>> -
>> -do_build[depends] = "isar-apt:do_cache_config
>> isar-bootstrap-target:do_bootstrap" diff --git
>> a/meta/recipes-devtools/buildchroot/buildchroot.inc
>> b/meta/recipes-devtools/buildchroot/buildchroot.inc index
>> ca9ce51..40fc8b0 100644 ---
>> a/meta/recipes-devtools/buildchroot/buildchroot.inc +++
>> b/meta/recipes-devtools/buildchroot/buildchroot.inc @@ -13,18 +13,25
>> @@ SRC_URI = "file://configscript.sh \ file://deps.sh" PV = "1.0"
>>
>> -inherit isar-bootstrap-helper
>> +inherit rootfs
>>
>> BUILDCHROOT_DIR = "${WORKDIR}/rootfs"
>> +ROOTFSDIR = "${BUILDCHROOT_DIR}"
>> +ROOTFS_PACKAGES = "${BUILDCHROOT_PREINSTALL}"
>> +ROOTFS_CLEAN_FILES = ""
>> +
>> +rootfs_do_mounts_append() {
>> + sudo -s <<'EOSUDO'
>> + mkdir -p '${BUILDCHROOT_DIR}/downloads'
>> + mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \
>> + mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads'
>> +EOSUDO
>> +}
>>
>> -do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \
>> - ${BUILDCHROOT_DIR}/isar-apt \
>> - ${BUILDCHROOT_DIR}/downloads \
>> - ${BUILDCHROOT_DIR}/home/builder"
>> -
>> -do_build() {
>> - setup_root_file_system ${PARAMS} ${BUILDCHROOT_DIR}
>> ${BUILDCHROOT_PREINSTALL} -
>> +ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files"
>> +buildchroot_install_files() {
>> + sudo mkdir -p "${BUILDCHROOT_DIR}/home/builder"
>> + sudo mkdir -p "${BUILDCHROOT_DIR}/isar-apt"
>> # Install package builder script
>> sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder"
>> sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar
>> @@ -38,5 +45,4 @@ do_build() {
>> GROUP_ID=$(id -g)
>> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh $USER_ID
>> $GROUP_ID
>> - sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads
>> }
>> diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
>> b/meta/recipes-devtools/sdkchroot/sdkchroot.bb index 49aeb47..57d52ba
>> 100644 --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
>> +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
>> @@ -12,10 +12,25 @@ SRC_URI = " \
>> file://configscript.sh \
>> file://README.sdk"
>> PV = "0.1"
>> -
>> -inherit isar-bootstrap-helper
>> PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}"
>>
>> +TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
>> +TOOLCHAIN_${HOST_ARCH} = "build-essential"
>> +TOOLCHAIN_i386 = "build-essential"
>> +
>> +inherit rootfs
>> +ROOTFS_ARCH = "${HOST_ARCH}"
>> +ROOTFS_DISTRO = "${HOST_DISTRO}"
>> +ROOTFSDIR = "${S}"
>> +ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}"
>> +ROOTFS_FEATURES += "copy-package-cache"
>> +
>> +python() {
>> + if d.getVar("HOST_ARCH") not in ['i386', 'amd64']:
>> + raise bb.parse.SkipRecipe("SDK doesn't support {} as
>> host".format(
>> + d.getVar("ROOTFS_ARCH")))
>> +}
>> +
>> SDKCHROOT_PREINSTALL := "debhelper \
>> autotools-dev \
>> dpkg \
>> @@ -28,29 +43,14 @@ SDKCHROOT_PREINSTALL := "debhelper \
>>
>> S = "${WORKDIR}/rootfs"
>>
>> -do_build[dirs] = "${DEPLOY_DIR_IMAGE}"
>> -do_build[root_cleandirs] = "${S} \
>> - ${S}/isar-apt"
>> -
>> -do_build[depends] = "isar-apt:do_cache_config
>> isar-bootstrap-host:do_bootstrap" -
>> -do_build() {
>> -
>> - if [ ${HOST_DISTRO} != "debian-stretch" ]; then
>> - bbfatal "SDK doesn't support ${HOST_DISTRO}"
>> - fi
>> - if [ ${HOST_ARCH} != "i386" -a ${HOST_ARCH} != "amd64" ]; then
>> - bbfatal "SDK doesn't support ${HOST_ARCH} as host"
>> - fi
>> -
>> - if [ ${HOST_ARCH} = ${DISTRO_ARCH} -o ${DISTRO_ARCH} = "i386" ];
>> then
>> - packages="${SDKCHROOT_PREINSTALL} build-essential"
>> - else
>> - packages="${SDKCHROOT_PREINSTALL}
>> crossbuild-essential-${DISTRO_ARCH}"
>> - fi
>> -
>> - setup_root_file_system --copyisarapt --host-arch '${HOST_ARCH}'
>> --host-distro '${HOST_DISTRO}' "${S}" $packages
>> +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir"
>> +rootfs_configure_isar_apt_dir() {
>> + # Copy isar-apt instead of mounting:
>> + sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
>> +}
>>
>> +ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
>> +sdkchroot_install_files() {
>> # Configure root filesystem
>> sudo install -m 644 ${WORKDIR}/README.sdk ${S}
>> sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2019-05-23 12:46 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-20 12:46 [PATCH v3 0/8] pre-processing pipeline and transient package replacement claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline claudius.heine.ext
2019-05-23 12:35 ` Henning Schild
2019-05-23 12:46 ` Claudius Heine
2019-05-20 12:46 ` [PATCH v3 2/8] meta: remove transient package support claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 3/8] meta/classes: add image-locales-extension class claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 4/8] meta/classes: add image-account-extension class claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 5/8] doc: update description of image customization claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 6/8] doc: some fixes claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 7/8] meta-isar: local.conf.sample: update root password and isar user creation claudius.heine.ext
2019-05-20 12:46 ` [PATCH v3 8/8] RECIPE-API-CHANGELOG: update transient package removal + root password claudius.heine.ext
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox