From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6691586504498610176 X-Received: by 2002:a7b:c442:: with SMTP id l2mr2100517wmi.50.1558702196550; Fri, 24 May 2019 05:49:56 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:2d86:: with SMTP id t128ls1012425wmt.3.gmail; Fri, 24 May 2019 05:49:56 -0700 (PDT) X-Google-Smtp-Source: APXvYqyIjdAjYEmLzBO1iyPsoo0bwNpAa8FwRR6vsuLyDXVMwOnbfQf1XO3N4gI0nmd7Y+uRXd+q X-Received: by 2002:a7b:c317:: with SMTP id k23mr15316960wmj.124.1558702196072; Fri, 24 May 2019 05:49:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558702196; cv=none; d=google.com; s=arc-20160816; b=yXKpSrGLHnlQLL6VW3qTnthfwZOXjy4s6ewf9N8cdENXH7LCd/yKSSWc9uEaXTh9m6 kTmhaMVmaAisa4yXHRuNdfWH8DjZkNqGKDh2sxPT8+TptLO9ebl+A4Uv13V4Nsbb9Qjw J2rKaWkcxRQE2RfOO+jbV8N/t2RQPpoJBmUG0EvS90aZkRVnYgzXbNctM3bZ7CpN/Re0 wT1c9rKb/Oah4znS2dNE1r2KrbthoHErxGMT/UID2nm2ZZ7k75+kVxyH0XNmnQRr5BdT bctIBZ0GSZUqokL71Mx56Sh/oXuICUJgYYjh/WVr4srlFvsg+SkqhyQnxtTQTdiqBSrx nzPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:cc:to :subject; bh=36XIk0/cHBs8Mxa8N6k8f2hA+oyQTKxflxF8FIqOZrY=; b=uEI2oTq8yjmkkbup1eeEu6J/d7EWERAf4HyH1GpXEWG6MyccIh3ZX4N4Nmj++MXPSD xIK70xrC179v5AQBBzhgRG9XXt7+iyGJDI/LjxfO4YOfRY6EZjLmLQUrVQWho318PPGj ktHDxHWTJRsLdfSR3OBjBPXr9GUuIFKqvCFsVUcfOHzA7JjLsr2tstzjIWalF4FtdV6L 4lIpKJCpbLZXug+bH7MfOf9q1gu7e/OsQt4s6ZZFoWy5GtrHms2YKa2TwVOFtSPnHe/m X0N5TKTo4RrBxslh84iXnBWSbiJkpmMVKnNn3ufPSt9lvhCtaaU4CVILlGIpQpYBWlrA /6iA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id w5si58611wrs.5.2019.05.24.05.49.55 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 24 May 2019 05:49:55 -0700 (PDT) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from [192.168.1.3] (195.165-131-109.adsl-dyn.isp.belgacom.be [109.131.165.195] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x4OCnqM8014116 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 May 2019 14:49:52 +0200 Subject: Re: [PATCH v4 2/8] split up isar-bootstrap helper and implement pre-process pipeline To: claudius.heine.ext@siemens.com, isar-users@googlegroups.com Cc: Claudius Heine References: <20190523145521.23050-1-claudius.heine.ext@siemens.com> <20190523145521.23050-3-claudius.heine.ext@siemens.com> From: "Maxim Yu. Osipov" Organization: ilbers GmbH Message-ID: <1867c412-713b-5dfb-fca8-15f9ffc99499@ilbers.de> Date: Fri, 24 May 2019 14:49:45 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190523145521.23050-3-claudius.heine.ext@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 7wRtI1CQxIay Hi Claudius, One may agree that setup_root_file_system is too "fat" and should be split to logical parts. My concern is that one have to carefully keep track for every ROOTFS_CONFIGURE/ROOTFS_INSTALL command's 'weight' - order of execution - possible source of errors. Just a case: one my not define weight value (in such case default value is 20) as its is done for ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir" - so if we have several such default weights the order of commands could be wrong. Regards, Maxim. On 5/23/19 4:55 PM, claudius.heine.ext@siemens.com wrote: > From: Claudius Heine > > This removes the isar-bootstrap-helper.bbclass and integrates its > functionality into the rootfs.bbclass. > > It introduces the rootfs_install task that runs through multiple > functions from command lists in order to allow easy integration of > custom functions. > > The introduced functions are, in general order of execution: > > rootfs_prepare: copy the isar-bootstrap rootfs base to workdir > rootfs_configure_*: basic configuration of the root file system, > before any custom packages where installed. (`/dev`, `/proc`, etc. > are not mounted). Functions are added to `ROOTFS_CONFIGURE_COMMAND`. > rootfs_do_mount: mount standard directories and repositories into the > rootfs > do_rootfs_install_*: downloading and installing packages. Functions > are added to `ROOTFS_INSTALL_COMMAND`. > > Signed-off-by: Claudius Heine > --- > meta/classes/image.bbclass | 24 +-- > meta/classes/isar-bootstrap-helper.bbclass | 132 -------------- > meta/classes/rootfs.bbclass | 172 +++++++++++++++++- > .../isar-bootstrap/isar-bootstrap.inc | 5 + > .../buildchroot/buildchroot-host.bb | 16 +- > .../buildchroot/buildchroot-target.bb | 2 - > .../buildchroot/buildchroot.inc | 26 ++- > meta/recipes-devtools/sdkchroot/sdkchroot.bb | 48 ++--- > 8 files changed, 228 insertions(+), 197 deletions(-) > delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index d77626c..99eea92 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -51,8 +51,10 @@ image_do_mounts() { > buildchroot_do_mounts > } > > -inherit isar-bootstrap-helper > -ROOTFS_FEATURES += "finalize-rootfs" > +ROOTFSDIR = "${IMAGE_ROOTFS}" > +ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs" > +ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" > + > inherit rootfs > inherit image-sdk-extension > inherit image-cache-extension > @@ -119,8 +121,10 @@ python set_image_size () { > d.setVarFlag('ROOTFS_SIZE', 'export', '1') > } > > -do_image_gen_fstab() { > - cat > ${WORKDIR}/fstab << EOF > +ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab" > +image_configure_fstab[weight] = "2" > +image_configure_fstab() { > + sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF > # Begin /etc/fstab > /dev/root / auto defaults 0 0 > proc /proc proc nosuid,noexec,nodev 0 0 > @@ -132,18 +136,6 @@ devtmpfs /dev devtmpfs mode=0755,nosuid 0 0 > # End /etc/fstab > EOF > } > -addtask image_gen_fstab before do_rootfs_install > - > -do_rootfs_install[depends] = "isar-apt:do_cache_config isar-bootstrap-target:do_bootstrap" > -do_rootfs_install[deptask] = "do_deploy_deb" > -do_rootfs_install[root_cleandirs] = "${IMAGE_ROOTFS} \ > - ${IMAGE_ROOTFS}/isar-apt" > -do_rootfs_install() { > - setup_root_file_system --clean --keep-apt-cache \ > - --fstab "${WORKDIR}/fstab" \ > - "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL} > -} > -addtask rootfs_install before do_build after do_unpack > > do_copy_boot_files[dirs] = "${DEPLOY_DIR_IMAGE}" > do_copy_boot_files() { > diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass > deleted file mode 100644 > index bbc6322..0000000 > --- a/meta/classes/isar-bootstrap-helper.bbclass > +++ /dev/null > @@ -1,132 +0,0 @@ > -# Helper functions for using isar-bootstrap > -# > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > -# > -# SPDX-License-Identifier: MIT > - > -def reverse_bb_array(d, varname): > - array = d.getVar(varname, True) > - if array is None: > - return None > - array = reversed(array.split()) > - return " ".join(i for i in array) > - > - > -update_etc_os_release() { > - OS_RELEASE_BUILD_ID="" > - OS_RELEASE_VARIANT="" > - while true; do > - case "$1" in > - --build-id) OS_RELEASE_BUILD_ID=$2; shift ;; > - --variant) OS_RELEASE_VARIANT=$2; shift ;; > - -*) bbfatal "$0: invalid option specified: $1" ;; > - *) break ;; > - esac > - shift > - done > - ROOTFSDIR="$1" > - > - if [ -n "${OS_RELEASE_BUILD_ID}" ]; then > - sudo sed -i '/^BUILD_ID=.*/d' ${ROOTFSDIR}/etc/os-release > - echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \ > - sudo tee -a ${ROOTFSDIR}/etc/os-release > - fi > - if [ -n "${OS_RELEASE_VARIANT}" ]; then > - sudo sed -i '/^VARIANT=.*/d' ${ROOTFSDIR}/etc/os-release > - echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \ > - sudo tee -a ${ROOTFSDIR}/etc/os-release > - fi > -} > - > -setup_root_file_system() { > - CLEAN="" > - COPYISARAPT="" > - FSTAB="" > - ROOTFS_ARCH="${DISTRO_ARCH}" > - ROOTFS_DISTRO="${DISTRO}" > - while true; do > - case "$1" in > - --clean) CLEAN=1 ;; > - --copyisarapt) COPYISARAPT=1 ;; > - --fstab) FSTAB=$2; shift ;; > - --host-arch) ROOTFS_ARCH="$2-$ROOTFS_ARCH"; shift ;; > - --host-distro) ROOTFS_DISTRO="$2"; shift ;; > - --keep-apt-cache) KEEP_APT_CACHE=1 ;; > - -*) bbfatal "$0: invalid option specified: $1" ;; > - *) break ;; > - esac > - shift > - done > - ROOTFSDIR="$1" > - shift > - PACKAGES="$@" > - APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes" > - CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf" > - > - sudo cp -Trpfx \ > - "${DEPLOY_DIR_BOOTSTRAP}/$ROOTFS_DISTRO-$ROOTFS_ARCH/" \ > - "$ROOTFSDIR" > - [ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee "$ROOTFSDIR/etc/fstab" > - > - echo "deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main" | \ > - sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list" >/dev/null > - > - echo "Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000" | \ > - sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null > - > - if [ ${COPYISARAPT} ]; then > - sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt > - else > - sudo mount --bind ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt > - fi > - > - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then > - sudo mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt > - fi > - > - sudo mount --rbind /dev ${ROOTFSDIR}/dev > - sudo mount --make-rslave ${ROOTFSDIR}/dev > - sudo mount -t proc none $ROOTFSDIR/proc > - sudo mount --rbind /sys ${ROOTFSDIR}/sys > - sudo mount --make-rslave ${ROOTFSDIR}/sys > - > - # Refresh /etc/resolv.conf > - sudo cp -L /etc/resolv.conf ${ROOTFSDIR}/etc > - > - # Install packages: > - E="${@ bb.utils.export_proxies(d)}" > - export DEBIAN_FRONTEND=noninteractive > - # To avoid Perl locale warnings: > - export LANG=C > - export LANGUAGE=C > - export LC_ALL=C > - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \ > - -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ > - -o Dir::Etc::sourceparts="-" \ > - -o APT::Get::List-Cleanup="0" > - # Add multiarch for cross-target > - if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then > - sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} > - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update > - fi > - sudo -E chroot "$ROOTFSDIR" \ > - /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES > - [ ${CLEAN} ] && sudo rm -f ${CLEAN_FILES} > - sudo -E chroot "$ROOTFSDIR" \ > - /usr/bin/apt-get ${APT_ARGS} $PACKAGES > - if [ ${CLEAN} ]; then > - if [ ${KEEP_APT_CACHE} -eq 1 ]; then > - mkdir -p ${WORKDIR}/apt_cache > - sudo find ${ROOTFSDIR}/var/cache/apt/archives \ > - -maxdepth 1 -name '*.deb' -execdir /bin/mv -t ${WORKDIR}/apt_cache '{}' '+' > - sudo chown -R $(whoami) ${WORKDIR}/apt_cache > - fi > - sudo -E chroot "$ROOTFSDIR" \ > - /usr/bin/apt-get autoremove --purge --yes > - sudo -E chroot "$ROOTFSDIR" \ > - /usr/bin/apt-get clean > - sudo "$ROOTFSDIR/chroot-setup.sh" "cleanup" "$ROOTFSDIR" > - sudo rm -rf "$ROOTFSDIR/chroot-setup.sh" "$ROOTFSDIR/var/lib/apt/lists/"* > - fi > -} > diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass > index 45b0350..0fc8ba8 100644 > --- a/meta/classes/rootfs.bbclass > +++ b/meta/classes/rootfs.bbclass > @@ -1,14 +1,178 @@ > # This software is a part of ISAR. > # Copyright (c) Siemens AG, 2019 > > +ROOTFS_ARCH ?= "${DISTRO_ARCH}" > +ROOTFS_DISTRO ?= "${DISTRO}" > +ROOTFS_PACKAGES ?= "" > + > # Features of the rootfs creation: > # available features are: > +# 'deploy-package-cache' - copy the package cache ${WORKDIR}/apt_cache > +# 'clean-package-cache' - delete package cache from rootfs > # 'finalize-rootfs' - delete files needed to chroot into the rootfs > ROOTFS_FEATURES ?= "" > > +ROOTFS_APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes" > + > +ROOTFS_CLEAN_FILES="/etc/hostname /etc/resolv.conf" > + > +# Useful environment variables: > +export E = "${@ bb.utils.export_proxies(d)}" > +export DEBIAN_FRONTEND = "noninteractive" > +# To avoid Perl locale warnings: > +export LANG = "C" > +export LANGUAGE = "C" > +export LC_ALL = "C" > + > +rootfs_do_mounts[weight] = "3" > +rootfs_do_mounts() { > + sudo -s <<'EOSUDO' > + mountpoint -q '${ROOTFSDIR}/dev' || \ > + mount --rbind /dev '${ROOTFSDIR}/dev' > + mount --make-rslave '${ROOTFSDIR}/dev' > + mountpoint -q '${ROOTFSDIR}/proc' || \ > + mount -t proc none '${ROOTFSDIR}/proc' > + mountpoint -q '${ROOTFSDIR}/sys' || \ > + mount --rbind /sys '${ROOTFSDIR}/sys' > + mount --make-rslave '${ROOTFSDIR}/sys' > + > + # Mount isar-apt if the directory does not exist or if it is empty > + # This prevents overwriting something that was copied there > + if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \ > + [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1 | wc -l)" = "0" ] > + then > + mkdir -p '${ROOTFSDIR}/isar-apt' > + mountpoint -q '${ROOTFSDIR}/isar-apt' || \ > + mount --bind '${REPO_ISAR_DIR}/${DISTRO}' '${ROOTFSDIR}/isar-apt' > + fi > + > + # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set > + if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ] > + then > + mkdir -p '${ROOTFSDIR}/base-apt' > + mountpoint -q '${ROOTFSDIR}/base-apt' || \ > + mount --bind '${REPO_BASE_DIR}' '${ROOTFSDIR}/base-apt' > + fi > + > +EOSUDO > +} > + > +rootfs_do_qemu() { > + if [ '${@repr(d.getVar('ROOTFS_ARCH') == d.getVar('HOST_ARCH'))}' = 'False' ] > + then > + test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \ > + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' > + fi > +} > + > +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}-${DISTRO_ARCH}" > +BOOTSTRAP_SRC_${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" > + > +rootfs_prepare[weight] = "25" > +rootfs_prepare(){ > + sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' > +} > + > +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" > +rootfs_configure_isar_apt[weight] = "2" > +rootfs_configure_isar_apt() { > + sudo -s <<'EOSUDO' > + > + mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d' > + echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \ > + '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list' > + > + mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d' > + cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar' > +Package: * > +Pin: release n=${DEBDISTRONAME} > +Pin-Priority: 1000 > +EOF > +EOSUDO > +} > + > +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update" > +rootfs_install_pkgs_update[weight] = "5" > +rootfs_install_pkgs_update() { > + sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \ > + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ > + -o Dir::Etc::sourceparts="-" \ > + -o APT::Get::List-Cleanup="0" > +} > +ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf" > +rootfs_install_resolvconf[weight] = "1" > +rootfs_install_resolvconf() { > + sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc' > +} > + > +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" > +rootfs_install_pkgs_download[weight] = "600" > +rootfs_install_pkgs_download() { > + sudo -E chroot '${ROOTFSDIR}' \ > + /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES} > +} > + > +ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}" > +rootfs_install_clean_files[weight] = "2" > +rootfs_install_clean_files() { > + sudo -E chroot '${ROOTFSDIR}' \ > + /bin/rm -f ${ROOTFS_CLEAN_FILES} > +} > + > +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_install" > +rootfs_install_pkgs_install[weight] = "8000" > +rootfs_install_pkgs_install() { > + sudo -E chroot "${ROOTFSDIR}" \ > + /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES} > +} Honestly I don't see benefits in such multiple configure_cmds/rootfs_install_cmds - it doesn't increase code readability as one have to additionally keep track for every command 'weight' value and it's easy to miss the correct order of commands execution. > +do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" > +do_rootfs_install[vardeps] = "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}" > +do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build isar-apt:do_cache_config" > +do_rootfs_install[deptask] = "do_deploy_deb" > +python do_rootfs_install() { > + configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or "").split() > + install_cmds = (d.getVar("ROOTFS_INSTALL_COMMAND", True) or "").split() > + > + # Mount after configure commands, so that they have time to copy > + # 'isar-apt' (sdkchroot): > + cmds = ['rootfs_prepare'] + configure_cmds + ['rootfs_do_mounts'] + install_cmds > + > + stage_weights = [int(d.getVarFlag(i, 'weight', True) or "20") > + for i in cmds] > + > + progress_reporter = bb.progress.MultiStageProgressReporter(d, stage_weights) > + > + for cmd in cmds: > + progress_reporter.next_stage() > + bb.build.exec_func(cmd, d) > + progress_reporter.finish() > +} > +addtask rootfs_install before do_rootfs_postprocess after do_unpack > + > +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'copy-package-cache', 'rootfs_postprocess_copy_package_cache', '', d)}" > +rootfs_postprocess_copy_package_cache() { > + mkdir -p '${WORKDIR}/apt_cache' > + sudo find '${ROOTFSDIR}/var/cache/apt/archives' \ > + -maxdepth 1 -name '*.deb' -execdir /bin/mv -t '${WORKDIR}/apt_cache' '{}' '+' > + me="$(id -u):$(id -g)" > + sudo chown -R "$me" '${WORKDIR}/apt_cache' > +} > + > +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}" > +rootfs_postprocess_clean_package_cache() { > + sudo -E chroot '${ROOTFSDIR}' \ > + /usr/bin/apt-get clean > + sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* > +} > + > ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs', 'rootfs_postprocess_finalize', '', d)}" > rootfs_postprocess_finalize() { > sudo -s <<'EOSUDO' > + test -e "${ROOTFSDIR}/chroot-setup.sh" && \ > + "${ROOTFSDIR}/chroot-setup.sh" "cleanup" "${ROOTFSDIR}" > + rm -f "${ROOTFSDIR}/chroot-setup.sh" > + > test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" && \ > find "${ROOTFSDIR}/usr/bin" \ > -maxdepth 1 -name 'qemu-*-static' -type f -delete > @@ -42,6 +206,12 @@ EOSUDO > > do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" > python do_rootfs_postprocess() { > + # Take care that its correctly mounted: > + bb.build.exec_func('rootfs_do_mounts', d) > + # Take care that qemu-*-static is available, since it could have been > + # removed on a previous execution of this task: > + bb.build.exec_func('rootfs_do_qemu', d) > + > cmds = d.getVar("ROOTFS_POSTPROCESS_COMMAND") > if cmds is None or not cmds.strip(): > return > @@ -49,7 +219,7 @@ python do_rootfs_postprocess() { > for cmd in cmds: > bb.build.exec_func(cmd, d) > } > -addtask rootfs_postprocess before do_rootfs after do_rootfs_install > +addtask rootfs_postprocess before do_rootfs > > python do_rootfs() { > """Virtual task""" > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index b740c57..f8741c2 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -327,6 +327,11 @@ isar_bootstrap() { > mount --make-rslave ${ROOTFSDIR}/sys > > export DEBIAN_FRONTEND=noninteractive > + > + if [ ${IS_HOST} ]; then > + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} > + fi > + > chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ > -o Debug::pkgProblemResolver=yes > diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb > index 73cd548..121992c 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb > @@ -7,6 +7,8 @@ DESCRIPTION = "Isar development filesystem for host" > PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}" > > require buildchroot.inc > +ROOTFS_ARCH = "${HOST_ARCH}" > +ROOTFS_DISTRO = "${HOST_DISTRO}" > > BUILDCHROOT_PREINSTALL ?= "make \ > debhelper \ > @@ -18,15 +20,5 @@ BUILDCHROOT_PREINSTALL ?= "make \ > automake \ > devscripts \ > equivs \ > - libc6:${DISTRO_ARCH}" > - > -# According to the wiki page: > -# https://wiki.debian.org/CrossToolchains > -BUILDCHROOT_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf \ > - crossbuild-essential-armhf" > -BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \ > - crossbuild-essential-arm64" > - > - > -PARAMS = "--host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}'" > -do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap" > + libc6:${DISTRO_ARCH} \ > + crossbuild-essential-${DISTRO_ARCH}" > diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb b/meta/recipes-devtools/buildchroot/buildchroot-target.bb > index 20f4d23..5a01258 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot-target.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot-target.bb > @@ -21,5 +21,3 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ > devscripts \ > equivs \ > adduser" > - > -do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-target:do_bootstrap" > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc > index ca9ce51..40fc8b0 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot.inc > +++ b/meta/recipes-devtools/buildchroot/buildchroot.inc > @@ -13,18 +13,25 @@ SRC_URI = "file://configscript.sh \ > file://deps.sh" > PV = "1.0" > > -inherit isar-bootstrap-helper > +inherit rootfs > > BUILDCHROOT_DIR = "${WORKDIR}/rootfs" > +ROOTFSDIR = "${BUILDCHROOT_DIR}" > +ROOTFS_PACKAGES = "${BUILDCHROOT_PREINSTALL}" > +ROOTFS_CLEAN_FILES = "" > + > +rootfs_do_mounts_append() { > + sudo -s <<'EOSUDO' > + mkdir -p '${BUILDCHROOT_DIR}/downloads' > + mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \ > + mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads' > +EOSUDO > +} > > -do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \ > - ${BUILDCHROOT_DIR}/isar-apt \ > - ${BUILDCHROOT_DIR}/downloads \ > - ${BUILDCHROOT_DIR}/home/builder" > - > -do_build() { > - setup_root_file_system ${PARAMS} ${BUILDCHROOT_DIR} ${BUILDCHROOT_PREINSTALL} > - > +ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files" > +buildchroot_install_files() { > + sudo mkdir -p "${BUILDCHROOT_DIR}/home/builder" > + sudo mkdir -p "${BUILDCHROOT_DIR}/isar-apt" > # Install package builder script > sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder" > sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar > @@ -38,5 +45,4 @@ do_build() { > GROUP_ID=$(id -g) > sudo chroot ${BUILDCHROOT_DIR} /configscript.sh $USER_ID $GROUP_ID > > - sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads > } > diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb > index 49aeb47..57d52ba 100644 > --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb > +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb > @@ -12,10 +12,25 @@ SRC_URI = " \ > file://configscript.sh \ > file://README.sdk" > PV = "0.1" > - > -inherit isar-bootstrap-helper > PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}" > > +TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}" > +TOOLCHAIN_${HOST_ARCH} = "build-essential" > +TOOLCHAIN_i386 = "build-essential" > + > +inherit rootfs > +ROOTFS_ARCH = "${HOST_ARCH}" > +ROOTFS_DISTRO = "${HOST_DISTRO}" > +ROOTFSDIR = "${S}" > +ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}" > +ROOTFS_FEATURES += "copy-package-cache" > + > +python() { > + if d.getVar("HOST_ARCH") not in ['i386', 'amd64']: > + raise bb.parse.SkipRecipe("SDK doesn't support {} as host".format( > + d.getVar("ROOTFS_ARCH"))) > +} > + > SDKCHROOT_PREINSTALL := "debhelper \ > autotools-dev \ > dpkg \ > @@ -28,29 +43,14 @@ SDKCHROOT_PREINSTALL := "debhelper \ > > S = "${WORKDIR}/rootfs" > > -do_build[dirs] = "${DEPLOY_DIR_IMAGE}" > -do_build[root_cleandirs] = "${S} \ > - ${S}/isar-apt" > - > -do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap" > - > -do_build() { > - > - if [ ${HOST_DISTRO} != "debian-stretch" ]; then > - bbfatal "SDK doesn't support ${HOST_DISTRO}" > - fi > - if [ ${HOST_ARCH} != "i386" -a ${HOST_ARCH} != "amd64" ]; then > - bbfatal "SDK doesn't support ${HOST_ARCH} as host" > - fi > - > - if [ ${HOST_ARCH} = ${DISTRO_ARCH} -o ${DISTRO_ARCH} = "i386" ]; then > - packages="${SDKCHROOT_PREINSTALL} build-essential" > - else > - packages="${SDKCHROOT_PREINSTALL} crossbuild-essential-${DISTRO_ARCH}" > - fi > - > - setup_root_file_system --copyisarapt --host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}' "${S}" $packages > +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir" > +rootfs_configure_isar_apt_dir() { > + # Copy isar-apt instead of mounting: > + sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt > +} > > +ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files" > +sdkchroot_install_files() { > # Configure root filesystem > sudo install -m 644 ${WORKDIR}/README.sdk ${S} > sudo install -m 755 ${WORKDIR}/configscript.sh ${S} > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov