From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6691586504498610176 X-Received: by 2002:a17:906:4d57:: with SMTP id b23mr75780241ejv.254.1558615578216; Thu, 23 May 2019 05:46:18 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:7807:: with SMTP id u7ls1344870ejm.13.gmail; Thu, 23 May 2019 05:46:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqyd0tIjTn+ZdscaqhQ7T47V5GLnMS6WDzFLiyzyA9qODrTInrEauW9BRPXCcVbSIIDKKBi7 X-Received: by 2002:a17:906:24d8:: with SMTP id f24mr55968067ejb.1.1558615577776; Thu, 23 May 2019 05:46:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558615577; cv=none; d=google.com; s=arc-20160816; b=IZbeiKj5piN5fOW4yEq0ws/qHTqM9i22/Z+mxMVkPcBxEJX1aRsGBkqF3Olu8R3n7F E+J2C79tqTNZf1r2LvrL2unyLpg8Rv3OtZaza10XvTty5wCLyxhliFVhCH6nR9Vn1wWG ARr2PqgGBODgLkx8uKPDxN44KJJbWxs44yu68vz1aoYc/4MkCpwduNB8kduEET6nOVAu QC9sW06UzKURTbjXI9rIuK1pp67ig4Pq5Iae36eDnJVOBLCq11BP46GG4UMey2YeRb0K 3LJ6EgIUdTLBkw/ojDtw5cM1wNjy/XnmDXVu2ccg/vek7MHVDZ+kt+6iZaB3UMM1dx76 7fZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=yJSpPHM8QudRMzc7EPBJzQ9ek/IPGxiHO2AcCQelXRU=; b=Zs+cbtVrdgHYwRPO4FV5/NKUQtHKD4GWOaS7r3mXV8/F3WjCOsspinKU9khd4XsKrJ jfXwzNoBq4xFhy0n+CfC+pziSsUhTn4ebQfaqciSuursmdfAr8GfZclz/F9ETFJO6S2Z dS8Jg8wAxWOmsXJiTv0k6gT6OzoBrP53DeW7cawjASUKAuQXqpaNGyMxUz5UO/rraZXs xYQAMOS0Eje9D4Z8mgq/4o/K8QckBH9XXU98Kp22cKzBtcdbsQuS621mL/8+H4EaWHh3 icMizQwlFCDjASTDPbu5Y2kyd7VJc5DUFhpP4L2s6QCIv6wesxphH0NU16sniROcSI5Z gaFA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id z45si2535422edc.3.2019.05.23.05.46.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 May 2019 05:46:17 -0700 (PDT) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id x4NCkFl3011666 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 23 May 2019 14:46:15 +0200 Received: from [139.25.69.232] (linux-ses-ext02.ppmd.siemens.net [139.25.69.232]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x4NCkFou003395; Thu, 23 May 2019 14:46:15 +0200 Subject: Re: [PATCH v3 1/8] split up isar-bootstrap helper and implement pre-process pipeline To: Henning Schild Cc: isar-users@googlegroups.com, Claudius Heine References: <20190520124618.16598-1-claudius.heine.ext@siemens.com> <20190520124618.16598-2-claudius.heine.ext@siemens.com> <20190523143534.20a8ad34@md1za8fc.ad001.siemens.net> From: Claudius Heine Message-ID: Date: Thu, 23 May 2019 14:46:15 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190523143534.20a8ad34@md1za8fc.ad001.siemens.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: Bll/Ri2DuNjx Hi Henning, On 23/05/2019 14.35, Henning Schild wrote: > I was looking at the series and did not really find the place where the > IMAGE_TRANSIENT_PACKAGES features get removed. Just that p2 does not > append to the variable anymore. > > For the first patch i would suggest a split. Splittling refactoring > changes from real functional ones ... I assume this patch does a > combination of both and the feature is dropped in here. That would mean I have to reimplement the TRANSIENT package feature for the new system in this patch and then remove again in the next. While this would be a clean approach, I just didn't want to spend the time doing that. Hmmm.. But if you insist, then I can switch it around, and remove it first and then split it without that feature. > > On top i would suggest a bbwarn if IMAGE_TRANSIENT_PACKAGES is > non-empty. That will give users of the feature a clue, maybe without > having to read the API changelog. Ok, I can do that in an additional patch. But I don't want to send a new version of that patchset just for that. Claudius > > Henning > > Am Mon, 20 May 2019 14:46:11 +0200 > schrieb "[ext] claudius.heine.ext@siemens.com" > : > >> From: Claudius Heine >> >> This removes the isar-bootstrap-helper.bbclass and integrates its >> functionality into the rootfs.bbclass. >> >> It introduces the rootfs_install task that runs through multiple >> functions from command lists in order to allow easy integration of >> custom functions. >> >> The introduced functions are, in general order of execution: >> >> rootfs_prepare: copy the isar-bootstrap rootfs base to workdir >> rootfs_configure_*: basic configuration of the root file system, >> before any custom packages where installed. (`/dev`, `/proc`, etc. >> are not mounted). Functions are added to >> `ROOTFS_CONFIGURE_COMMAND`. rootfs_do_mount: mount standard >> directories and repositories into the rootfs >> do_rootfs_install_*: downloading and installing packages. Functions >> are added to `ROOTFS_INSTALL_COMMAND`. >> >> Signed-off-by: Claudius Heine >> --- >> meta/classes/image.bbclass | 24 +-- >> meta/classes/isar-bootstrap-helper.bbclass | 143 --------------- >> meta/classes/rootfs.bbclass | 172 >> +++++++++++++++++- .../isar-bootstrap/isar-bootstrap.inc | >> 5 + .../buildchroot/buildchroot-host.bb | 16 +- >> .../buildchroot/buildchroot-target.bb | 2 - >> .../buildchroot/buildchroot.inc | 26 ++- >> meta/recipes-devtools/sdkchroot/sdkchroot.bb | 48 ++--- >> 8 files changed, 228 insertions(+), 208 deletions(-) >> delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass >> >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass >> index 3eb261f..107b272 100644 >> --- a/meta/classes/image.bbclass >> +++ b/meta/classes/image.bbclass >> @@ -53,8 +53,10 @@ image_do_mounts() { >> buildchroot_do_mounts >> } >> >> -inherit isar-bootstrap-helper >> -ROOTFS_FEATURES += "finalize-rootfs" >> +ROOTFSDIR = "${IMAGE_ROOTFS}" >> +ROOTFS_FEATURES += "copy-package-cache clean-package-cache >> finalize-rootfs" +ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} >> ${IMAGE_INSTALL}" + >> inherit rootfs >> inherit image-sdk-extension >> inherit image-cache-extension >> @@ -121,8 +123,10 @@ python set_image_size () { >> d.setVarFlag('ROOTFS_SIZE', 'export', '1') >> } >> >> -do_image_gen_fstab() { >> - cat > ${WORKDIR}/fstab << EOF >> +ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab" >> +image_configure_fstab[weight] = "2" >> +image_configure_fstab() { >> + sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF >> # Begin /etc/fstab >> /dev/root / auto >> defaults 0 0 >> proc /proc proc >> nosuid,noexec,nodev 0 0 @@ -134,18 +138,6 @@ >> devtmpfs /dev devtmpfs >> mode=0755,nosuid 0 0 # End /etc/fstab EOF } >> -addtask image_gen_fstab before do_rootfs_install >> - >> -do_rootfs_install[depends] = "isar-apt:do_cache_config >> isar-bootstrap-target:do_bootstrap" -do_rootfs_install[deptask] = >> "do_deploy_deb" -do_rootfs_install[root_cleandirs] = "${IMAGE_ROOTFS} >> \ >> - ${IMAGE_ROOTFS}/isar-apt" >> -do_rootfs_install() { >> - setup_root_file_system --clean --keep-apt-cache \ >> - --fstab "${WORKDIR}/fstab" \ >> - "${IMAGE_ROOTFS}" ${IMAGE_PREINSTALL} ${IMAGE_INSTALL} >> -} >> -addtask rootfs_install before do_build after do_unpack >> >> do_copy_boot_files[dirs] = "${DEPLOY_DIR_IMAGE}" >> do_copy_boot_files() { >> diff --git a/meta/classes/isar-bootstrap-helper.bbclass >> b/meta/classes/isar-bootstrap-helper.bbclass deleted file mode 100644 >> index 8612be2..0000000 >> --- a/meta/classes/isar-bootstrap-helper.bbclass >> +++ /dev/null >> @@ -1,143 +0,0 @@ >> -# Helper functions for using isar-bootstrap >> -# >> -# This software is a part of ISAR. >> -# Copyright (c) Siemens AG, 2018 >> -# >> -# SPDX-License-Identifier: MIT >> - >> -IMAGE_TRANSIENT_PACKAGES ??= "" >> - >> -def reverse_bb_array(d, varname): >> - array = d.getVar(varname, True) >> - if array is None: >> - return None >> - array = reversed(array.split()) >> - return " ".join(i for i in array) >> - >> - >> -update_etc_os_release() { >> - OS_RELEASE_BUILD_ID="" >> - OS_RELEASE_VARIANT="" >> - while true; do >> - case "$1" in >> - --build-id) OS_RELEASE_BUILD_ID=$2; shift ;; >> - --variant) OS_RELEASE_VARIANT=$2; shift ;; >> - -*) bbfatal "$0: invalid option specified: $1" ;; >> - *) break ;; >> - esac >> - shift >> - done >> - ROOTFSDIR="$1" >> - >> - if [ -n "${OS_RELEASE_BUILD_ID}" ]; then >> - sudo sed -i '/^BUILD_ID=.*/d' ${ROOTFSDIR}/etc/os-release >> - echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \ >> - sudo tee -a ${ROOTFSDIR}/etc/os-release >> - fi >> - if [ -n "${OS_RELEASE_VARIANT}" ]; then >> - sudo sed -i '/^VARIANT=.*/d' ${ROOTFSDIR}/etc/os-release >> - echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \ >> - sudo tee -a ${ROOTFSDIR}/etc/os-release >> - fi >> -} >> - >> -setup_root_file_system() { >> - CLEAN="" >> - COPYISARAPT="" >> - FSTAB="" >> - ROOTFS_ARCH="${DISTRO_ARCH}" >> - ROOTFS_DISTRO="${DISTRO}" >> - while true; do >> - case "$1" in >> - --clean) CLEAN=1 ;; >> - --copyisarapt) COPYISARAPT=1 ;; >> - --fstab) FSTAB=$2; shift ;; >> - --host-arch) ROOTFS_ARCH="$2-$ROOTFS_ARCH"; shift ;; >> - --host-distro) ROOTFS_DISTRO="$2"; shift ;; >> - --keep-apt-cache) KEEP_APT_CACHE=1 ;; >> - -*) bbfatal "$0: invalid option specified: $1" ;; >> - *) break ;; >> - esac >> - shift >> - done >> - ROOTFSDIR="$1" >> - shift >> - PACKAGES="$@" >> - APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes" >> - CLEAN_FILES="${ROOTFSDIR}/etc/hostname >> ${ROOTFSDIR}/etc/resolv.conf" - >> - sudo cp -Trpfx \ >> - "${DEPLOY_DIR_BOOTSTRAP}/$ROOTFS_DISTRO-$ROOTFS_ARCH/" \ >> - "$ROOTFSDIR" >> - [ -n "${FSTAB}" ] && cat ${FSTAB} | sudo tee >> "$ROOTFSDIR/etc/fstab" - >> - echo "deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main" >> | \ >> - sudo tee "$ROOTFSDIR/etc/apt/sources.list.d/isar-apt.list" >>> /dev/null - >> - echo "Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: >> 1000" | \ >> - sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null >> - >> - if [ ${COPYISARAPT} ]; then >> - sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} $ROOTFSDIR/isar-apt >> - else >> - sudo mount --bind ${REPO_ISAR_DIR}/${DISTRO} >> $ROOTFSDIR/isar-apt >> - fi >> - >> - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then >> - sudo mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt >> - fi >> - >> - sudo mount --rbind /dev ${ROOTFSDIR}/dev >> - sudo mount --make-rslave ${ROOTFSDIR}/dev >> - sudo mount -t proc none $ROOTFSDIR/proc >> - sudo mount --rbind /sys ${ROOTFSDIR}/sys >> - sudo mount --make-rslave ${ROOTFSDIR}/sys >> - >> - # Refresh /etc/resolv.conf >> - sudo cp -L /etc/resolv.conf ${ROOTFSDIR}/etc >> - >> - # Install packages: >> - E="${@ bb.utils.export_proxies(d)}" >> - export DEBIAN_FRONTEND=noninteractive >> - # To avoid Perl locale warnings: >> - export LANG=C >> - export LANGUAGE=C >> - export LC_ALL=C >> - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \ >> - -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ >> - -o Dir::Etc::sourceparts="-" \ >> - -o APT::Get::List-Cleanup="0" >> - # Add multiarch for cross-target >> - if [ "${ROOTFS_ARCH}" != "${DISTRO_ARCH}" ]; then >> - sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg --add-architecture >> ${DISTRO_ARCH} >> - sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update >> - fi >> - sudo -E chroot "$ROOTFSDIR" \ >> - /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \ >> - ${IMAGE_TRANSIENT_PACKAGES} >> - [ ${CLEAN} ] && sudo rm -f ${CLEAN_FILES} >> - sudo -E chroot "$ROOTFSDIR" \ >> - /usr/bin/apt-get ${APT_ARGS} $PACKAGES >> - for pkg in ${IMAGE_TRANSIENT_PACKAGES}; do >> - sudo -E chroot "$ROOTFSDIR" \ >> - /usr/bin/apt-get ${APT_ARGS} $pkg >> - done >> - for pkg in ${@reverse_bb_array(d, "IMAGE_TRANSIENT_PACKAGES") or >> ""}; do >> - sudo -E chroot "$ROOTFSDIR" \ >> - /usr/bin/apt-get purge --yes $pkg >> - done >> - if [ ${CLEAN} ]; then >> - if [ ${KEEP_APT_CACHE} -eq 1 ]; then >> - mkdir -p ${WORKDIR}/apt_cache >> - sudo find ${ROOTFSDIR}/var/cache/apt/archives \ >> - -maxdepth 1 -name '*.deb' -execdir /bin/mv -t >> ${WORKDIR}/apt_cache '{}' '+' >> - sudo chown -R $(whoami) ${WORKDIR}/apt_cache >> - fi >> - sudo -E chroot "$ROOTFSDIR" \ >> - /usr/bin/apt-get autoremove --purge --yes >> - sudo -E chroot "$ROOTFSDIR" \ >> - /usr/bin/apt-get clean >> - sudo "$ROOTFSDIR/chroot-setup.sh" "cleanup" "$ROOTFSDIR" >> - sudo rm -rf "$ROOTFSDIR/chroot-setup.sh" >> "$ROOTFSDIR/var/lib/apt/lists/"* >> - fi >> -} >> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass >> index 45b0350..0fc8ba8 100644 >> --- a/meta/classes/rootfs.bbclass >> +++ b/meta/classes/rootfs.bbclass >> @@ -1,14 +1,178 @@ >> # This software is a part of ISAR. >> # Copyright (c) Siemens AG, 2019 >> >> +ROOTFS_ARCH ?= "${DISTRO_ARCH}" >> +ROOTFS_DISTRO ?= "${DISTRO}" >> +ROOTFS_PACKAGES ?= "" >> + >> # Features of the rootfs creation: >> # available features are: >> +# 'deploy-package-cache' - copy the package cache >> ${WORKDIR}/apt_cache +# 'clean-package-cache' - delete package cache >> from rootfs # 'finalize-rootfs' - delete files needed to chroot into >> the rootfs ROOTFS_FEATURES ?= "" >> >> +ROOTFS_APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes" >> + >> +ROOTFS_CLEAN_FILES="/etc/hostname /etc/resolv.conf" >> + >> +# Useful environment variables: >> +export E = "${@ bb.utils.export_proxies(d)}" >> +export DEBIAN_FRONTEND = "noninteractive" >> +# To avoid Perl locale warnings: >> +export LANG = "C" >> +export LANGUAGE = "C" >> +export LC_ALL = "C" >> + >> +rootfs_do_mounts[weight] = "3" >> +rootfs_do_mounts() { >> + sudo -s <<'EOSUDO' >> + mountpoint -q '${ROOTFSDIR}/dev' || \ >> + mount --rbind /dev '${ROOTFSDIR}/dev' >> + mount --make-rslave '${ROOTFSDIR}/dev' >> + mountpoint -q '${ROOTFSDIR}/proc' || \ >> + mount -t proc none '${ROOTFSDIR}/proc' >> + mountpoint -q '${ROOTFSDIR}/sys' || \ >> + mount --rbind /sys '${ROOTFSDIR}/sys' >> + mount --make-rslave '${ROOTFSDIR}/sys' >> + >> + # Mount isar-apt if the directory does not exist or if it is >> empty >> + # This prevents overwriting something that was copied there >> + if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \ >> + [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1 >> | wc -l)" = "0" ] >> + then >> + mkdir -p '${ROOTFSDIR}/isar-apt' >> + mountpoint -q '${ROOTFSDIR}/isar-apt' || \ >> + mount --bind '${REPO_ISAR_DIR}/${DISTRO}' >> '${ROOTFSDIR}/isar-apt' >> + fi >> + >> + # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set >> + if >> [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" >> = 'True' ] >> + then >> + mkdir -p '${ROOTFSDIR}/base-apt' >> + mountpoint -q '${ROOTFSDIR}/base-apt' || \ >> + mount --bind '${REPO_BASE_DIR}' >> '${ROOTFSDIR}/base-apt' >> + fi >> + >> +EOSUDO >> +} >> + >> +rootfs_do_qemu() { >> + if [ '${@repr(d.getVar('ROOTFS_ARCH') == >> d.getVar('HOST_ARCH'))}' = 'False' ] >> + then >> + test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \ >> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >> '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' >> + fi >> +} >> + >> +BOOTSTRAP_SRC = >> "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}-${DISTRO_ARCH}" >> +BOOTSTRAP_SRC_${ROOTFS_ARCH} = >> "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" + >> +rootfs_prepare[weight] = "25" +rootfs_prepare(){ >> + sudo cp -Trpfx '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' >> +} >> + >> +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" >> +rootfs_configure_isar_apt[weight] = "2" >> +rootfs_configure_isar_apt() { >> + sudo -s <<'EOSUDO' >> + >> + mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d' >> + echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' >>> \ >> + '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list' >> + >> + mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d' >> + cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar' >> +Package: * >> +Pin: release n=${DEBDISTRONAME} >> +Pin-Priority: 1000 >> +EOF >> +EOSUDO >> +} >> + >> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update" >> +rootfs_install_pkgs_update[weight] = "5" >> +rootfs_install_pkgs_update() { >> + sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \ >> + -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ >> + -o Dir::Etc::sourceparts="-" \ >> + -o APT::Get::List-Cleanup="0" >> +} >> +ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf" >> +rootfs_install_resolvconf[weight] = "1" >> +rootfs_install_resolvconf() { >> + sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc' >> +} >> + >> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" >> +rootfs_install_pkgs_download[weight] = "600" >> +rootfs_install_pkgs_download() { >> + sudo -E chroot '${ROOTFSDIR}' \ >> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only >> ${ROOTFS_PACKAGES} +} >> + >> +ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if >> (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}" >> +rootfs_install_clean_files[weight] = "2" >> +rootfs_install_clean_files() { >> + sudo -E chroot '${ROOTFSDIR}' \ >> + /bin/rm -f ${ROOTFS_CLEAN_FILES} >> +} >> + >> +ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_install" >> +rootfs_install_pkgs_install[weight] = "8000" >> +rootfs_install_pkgs_install() { >> + sudo -E chroot "${ROOTFSDIR}" \ >> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES} >> +} >> + >> +do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" >> +do_rootfs_install[vardeps] = "${ROOTFS_CONFIGURE_COMMAND} >> ${ROOTFS_INSTALL_COMMAND}" +do_rootfs_install[depends] = >> "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == >> d.getVar('DISTRO_ARCH') else 'host'}:do_build >> isar-apt:do_cache_config" +do_rootfs_install[deptask] = >> "do_deploy_deb" +python do_rootfs_install() { >> + configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or >> "").split() >> + install_cmds = (d.getVar("ROOTFS_INSTALL_COMMAND", True) or >> "").split() + >> + # Mount after configure commands, so that they have time to copy >> + # 'isar-apt' (sdkchroot): >> + cmds = ['rootfs_prepare'] + configure_cmds + >> ['rootfs_do_mounts'] + install_cmds + >> + stage_weights = [int(d.getVarFlag(i, 'weight', True) or "20") >> + for i in cmds] >> + >> + progress_reporter = bb.progress.MultiStageProgressReporter(d, >> stage_weights) + >> + for cmd in cmds: >> + progress_reporter.next_stage() >> + bb.build.exec_func(cmd, d) >> + progress_reporter.finish() >> +} >> +addtask rootfs_install before do_rootfs_postprocess after do_unpack >> + >> +ROOTFS_POSTPROCESS_COMMAND += >> "${@bb.utils.contains('ROOTFS_FEATURES', 'copy-package-cache', >> 'rootfs_postprocess_copy_package_cache', '', d)}" >> +rootfs_postprocess_copy_package_cache() { >> + mkdir -p '${WORKDIR}/apt_cache' >> + sudo find '${ROOTFSDIR}/var/cache/apt/archives' \ >> + -maxdepth 1 -name '*.deb' -execdir /bin/mv -t >> '${WORKDIR}/apt_cache' '{}' '+' >> + me="$(id -u):$(id -g)" >> + sudo chown -R "$me" '${WORKDIR}/apt_cache' >> +} >> + >> +ROOTFS_POSTPROCESS_COMMAND += >> "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', >> 'rootfs_postprocess_clean_package_cache', '', d)}" >> +rootfs_postprocess_clean_package_cache() { >> + sudo -E chroot '${ROOTFSDIR}' \ >> + /usr/bin/apt-get clean >> + sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* >> +} >> + >> ROOTFS_POSTPROCESS_COMMAND += >> "${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs', >> 'rootfs_postprocess_finalize', '', d)}" rootfs_postprocess_finalize() >> { sudo -s <<'EOSUDO' >> + test -e "${ROOTFSDIR}/chroot-setup.sh" && \ >> + "${ROOTFSDIR}/chroot-setup.sh" "cleanup" "${ROOTFSDIR}" >> + rm -f "${ROOTFSDIR}/chroot-setup.sh" >> + >> test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" && \ >> find "${ROOTFSDIR}/usr/bin" \ >> -maxdepth 1 -name 'qemu-*-static' -type f -delete >> @@ -42,6 +206,12 @@ EOSUDO >> >> do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" >> python do_rootfs_postprocess() { >> + # Take care that its correctly mounted: >> + bb.build.exec_func('rootfs_do_mounts', d) >> + # Take care that qemu-*-static is available, since it could have >> been >> + # removed on a previous execution of this task: >> + bb.build.exec_func('rootfs_do_qemu', d) >> + >> cmds = d.getVar("ROOTFS_POSTPROCESS_COMMAND") >> if cmds is None or not cmds.strip(): >> return >> @@ -49,7 +219,7 @@ python do_rootfs_postprocess() { >> for cmd in cmds: >> bb.build.exec_func(cmd, d) >> } >> -addtask rootfs_postprocess before do_rootfs after do_rootfs_install >> +addtask rootfs_postprocess before do_rootfs >> >> python do_rootfs() { >> """Virtual task""" >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index >> b740c57..f8741c2 100644 --- >> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -327,6 >> +327,11 @@ isar_bootstrap() { mount --make-rslave ${ROOTFSDIR}/sys >> >> export DEBIAN_FRONTEND=noninteractive >> + >> + if [ ${IS_HOST} ]; then >> + chroot "${ROOTFSDIR}" /usr/bin/dpkg >> --add-architecture ${DISTRO_ARCH} >> + fi >> + >> chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y >> chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ >> -o Debug::pkgProblemResolver=yes >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index >> 73cd548..121992c 100644 --- >> a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++ >> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -7,6 +7,8 >> @@ DESCRIPTION = "Isar development filesystem for host" PF = >> "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}" >> require buildchroot.inc >> +ROOTFS_ARCH = "${HOST_ARCH}" >> +ROOTFS_DISTRO = "${HOST_DISTRO}" >> >> BUILDCHROOT_PREINSTALL ?= "make \ >> debhelper \ >> @@ -18,15 +20,5 @@ BUILDCHROOT_PREINSTALL ?= "make \ >> automake \ >> devscripts \ >> equivs \ >> - libc6:${DISTRO_ARCH}" >> - >> -# According to the wiki page: >> -# https://wiki.debian.org/CrossToolchains >> -BUILDCHROOT_PREINSTALL_append_armhf += "binutils-arm-linux-gnueabihf >> \ >> - crossbuild-essential-armhf" >> -BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \ >> - crossbuild-essential-arm64" >> - >> - >> -PARAMS = "--host-arch '${HOST_ARCH}' --host-distro '${HOST_DISTRO}'" >> -do_build[depends] = "isar-apt:do_cache_config >> isar-bootstrap-host:do_bootstrap" >> + libc6:${DISTRO_ARCH} \ >> + crossbuild-essential-${DISTRO_ARCH}" >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb >> b/meta/recipes-devtools/buildchroot/buildchroot-target.bb index >> 20f4d23..5a01258 100644 --- >> a/meta/recipes-devtools/buildchroot/buildchroot-target.bb +++ >> b/meta/recipes-devtools/buildchroot/buildchroot-target.bb @@ -21,5 >> +21,3 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ devscripts \ >> equivs \ >> adduser" >> - >> -do_build[depends] = "isar-apt:do_cache_config >> isar-bootstrap-target:do_bootstrap" diff --git >> a/meta/recipes-devtools/buildchroot/buildchroot.inc >> b/meta/recipes-devtools/buildchroot/buildchroot.inc index >> ca9ce51..40fc8b0 100644 --- >> a/meta/recipes-devtools/buildchroot/buildchroot.inc +++ >> b/meta/recipes-devtools/buildchroot/buildchroot.inc @@ -13,18 +13,25 >> @@ SRC_URI = "file://configscript.sh \ file://deps.sh" PV = "1.0" >> >> -inherit isar-bootstrap-helper >> +inherit rootfs >> >> BUILDCHROOT_DIR = "${WORKDIR}/rootfs" >> +ROOTFSDIR = "${BUILDCHROOT_DIR}" >> +ROOTFS_PACKAGES = "${BUILDCHROOT_PREINSTALL}" >> +ROOTFS_CLEAN_FILES = "" >> + >> +rootfs_do_mounts_append() { >> + sudo -s <<'EOSUDO' >> + mkdir -p '${BUILDCHROOT_DIR}/downloads' >> + mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \ >> + mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads' >> +EOSUDO >> +} >> >> -do_build[root_cleandirs] = "${BUILDCHROOT_DIR} \ >> - ${BUILDCHROOT_DIR}/isar-apt \ >> - ${BUILDCHROOT_DIR}/downloads \ >> - ${BUILDCHROOT_DIR}/home/builder" >> - >> -do_build() { >> - setup_root_file_system ${PARAMS} ${BUILDCHROOT_DIR} >> ${BUILDCHROOT_PREINSTALL} - >> +ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files" >> +buildchroot_install_files() { >> + sudo mkdir -p "${BUILDCHROOT_DIR}/home/builder" >> + sudo mkdir -p "${BUILDCHROOT_DIR}/isar-apt" >> # Install package builder script >> sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder" >> sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar >> @@ -38,5 +45,4 @@ do_build() { >> GROUP_ID=$(id -g) >> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh $USER_ID >> $GROUP_ID >> - sudo mount --bind ${DL_DIR} ${BUILDCHROOT_DIR}/downloads >> } >> diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb >> b/meta/recipes-devtools/sdkchroot/sdkchroot.bb index 49aeb47..57d52ba >> 100644 --- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb >> +++ b/meta/recipes-devtools/sdkchroot/sdkchroot.bb >> @@ -12,10 +12,25 @@ SRC_URI = " \ >> file://configscript.sh \ >> file://README.sdk" >> PV = "0.1" >> - >> -inherit isar-bootstrap-helper >> PF = "${PN}-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}" >> >> +TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}" >> +TOOLCHAIN_${HOST_ARCH} = "build-essential" >> +TOOLCHAIN_i386 = "build-essential" >> + >> +inherit rootfs >> +ROOTFS_ARCH = "${HOST_ARCH}" >> +ROOTFS_DISTRO = "${HOST_DISTRO}" >> +ROOTFSDIR = "${S}" >> +ROOTFS_PACKAGES = "${SDKCHROOT_PREINSTALL} ${TOOLCHAIN}" >> +ROOTFS_FEATURES += "copy-package-cache" >> + >> +python() { >> + if d.getVar("HOST_ARCH") not in ['i386', 'amd64']: >> + raise bb.parse.SkipRecipe("SDK doesn't support {} as >> host".format( >> + d.getVar("ROOTFS_ARCH"))) >> +} >> + >> SDKCHROOT_PREINSTALL := "debhelper \ >> autotools-dev \ >> dpkg \ >> @@ -28,29 +43,14 @@ SDKCHROOT_PREINSTALL := "debhelper \ >> >> S = "${WORKDIR}/rootfs" >> >> -do_build[dirs] = "${DEPLOY_DIR_IMAGE}" >> -do_build[root_cleandirs] = "${S} \ >> - ${S}/isar-apt" >> - >> -do_build[depends] = "isar-apt:do_cache_config >> isar-bootstrap-host:do_bootstrap" - >> -do_build() { >> - >> - if [ ${HOST_DISTRO} != "debian-stretch" ]; then >> - bbfatal "SDK doesn't support ${HOST_DISTRO}" >> - fi >> - if [ ${HOST_ARCH} != "i386" -a ${HOST_ARCH} != "amd64" ]; then >> - bbfatal "SDK doesn't support ${HOST_ARCH} as host" >> - fi >> - >> - if [ ${HOST_ARCH} = ${DISTRO_ARCH} -o ${DISTRO_ARCH} = "i386" ]; >> then >> - packages="${SDKCHROOT_PREINSTALL} build-essential" >> - else >> - packages="${SDKCHROOT_PREINSTALL} >> crossbuild-essential-${DISTRO_ARCH}" >> - fi >> - >> - setup_root_file_system --copyisarapt --host-arch '${HOST_ARCH}' >> --host-distro '${HOST_DISTRO}' "${S}" $packages >> +ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt_dir" >> +rootfs_configure_isar_apt_dir() { >> + # Copy isar-apt instead of mounting: >> + sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt >> +} >> >> +ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files" >> +sdkchroot_install_files() { >> # Configure root filesystem >> sudo install -m 644 ${WORKDIR}/README.sdk ${S} >> sudo install -m 755 ${WORKDIR}/configscript.sh ${S} > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de