From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7018164937051602944 X-Received: by 2002:a05:6512:3341:: with SMTP id y1mr40814786lfd.496.1634121729709; Wed, 13 Oct 2021 03:42:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:982:: with SMTP id b2ls331132ljq.7.gmail; Wed, 13 Oct 2021 03:42:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwZvVqXQw372peYjjXmuTWm9RI/0LkHmFkITA9YBt8WTBaCERLnatO1ZpGWJ9HCl8UIfh7b X-Received: by 2002:a2e:bd82:: with SMTP id o2mr37049608ljq.289.1634121728418; Wed, 13 Oct 2021 03:42:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634121728; cv=none; d=google.com; s=arc-20160816; b=NMrMeHUIKqCQay2k9uAH+AHmSrwAUZQMLQlXwaZYjX6Kzq6ZX2jAfZlXCaS6R3eNCw Hz7Ak98PHHvJt3Uy5uMd7z8Ri4y/nF9PcqtfiMaQwP+nEc3+LxA5wNAgeeLjcRChu45K DDkbX6u/ZcZD0quCAksLmscv24/dydGBJ7bbcgD5SZC4NGOu3UqIFQrf+tNy65nltuiw g9Jc5tYA/XBh1VJMuBuYKRlSA8jGIjCgqdjdFy69SfoZaVP7Nq89zT2yHI+qrnrl9kpr BeBttZEC+pjB3wlzBO7AUBuk6AQyCvodlRX2Cy/weLpNfwHXL6HhXRX+e/KGRe4NbN2Z NYHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=KP4BiM7mF8rx1L+y8j0GRKRhGr9NZkwEy9ygM0x0g+E=; b=Df5vj8NzSSjEn6lcRRcf4Jj0fV5fD/NDhEQUZAfFl6NsRIICu75qR6DxJarI9ReIV/ oDJG8Iy9gbtVyXkSg2NHpXdeo3ATHMCPwWjBH3/lpbnvZ117aQ69iM9+cEwHvkhviJbt TyCSiBv/MUtwf5GtnhrQJp7LHBd0JZXHNTd8AnkqSFscA62aQfcTVHxG+eznByIJs6lJ 9/0oTmbXEJQp+nSyQlGb4QczyrkfeuB7jvf33d7dH88qJ4tD4Eui2iofCb70z2WP3YnB +DZbqZjZzpqEoRZEXaGQTVGrk12Ni97EaQw9xld0ZwBOz43mr0vw3ZnXvFjhw1+Z8Q6z ttSQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id t12si768608ljh.0.2021.10.13.03.42.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Oct 2021 03:42:08 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 19DAg70Z029587 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 13 Oct 2021 12:42:07 +0200 Received: from [167.87.72.172] ([167.87.72.172]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 19DAg7Mb021073; Wed, 13 Oct 2021 12:42:07 +0200 Subject: Re: [RFC PATCH 5/5] meta: refactor to use the new mounting mechanism To: Adriaan Schmidt , isar-users@googlegroups.com References: <20211012130413.1719424-1-adriaan.schmidt@siemens.com> <20211012130413.1719424-6-adriaan.schmidt@siemens.com> From: Jan Kiszka Message-ID: Date: Wed, 13 Oct 2021 12:42:06 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <20211012130413.1719424-6-adriaan.schmidt@siemens.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 2MvC75HD9zsp -ENOCOMMITMSG On 12.10.21 15:04, Adriaan Schmidt wrote: > Signed-off-by: Adriaan Schmidt > --- > meta/classes/buildchroot.bbclass | 52 ++++++++----------- > meta/classes/cpiogz-img.bbclass | 3 +- > meta/classes/dpkg-base.bbclass | 44 +++------------- > meta/classes/dpkg-gbp.bbclass | 2 - > meta/classes/dpkg.bbclass | 4 +- > meta/classes/ext4-img.bbclass | 3 +- > meta/classes/fit-img.bbclass | 4 +- > meta/classes/image-tools-extension.bbclass | 4 +- > meta/classes/image.bbclass | 30 +++-------- > meta/classes/initramfs.bbclass | 2 +- > meta/classes/isar-events.bbclass | 4 +- > meta/classes/rootfs.bbclass | 50 ++++++------------ > meta/classes/ubi-img.bbclass | 3 +- > meta/classes/ubifs-img.bbclass | 3 +- > meta/classes/vm-img.bbclass | 7 +-- > meta/classes/wic-img.bbclass | 31 ++++------- > .../isar-bootstrap/isar-bootstrap.inc | 43 +++++++++------ > .../buildchroot/buildchroot.inc | 8 +-- > 18 files changed, 104 insertions(+), 193 deletions(-) > > diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass > index e9eb9af..7c34834 100644 > --- a/meta/classes/buildchroot.bbclass > +++ b/meta/classes/buildchroot.bbclass > @@ -13,50 +13,42 @@ python __anonymous() { > (d.getVar('HOST_DISTRO') == "debian-stretch" and distro_arch == "i386"): > dep = "buildchroot-target:do_build" > rootfs = d.getVar('BUILDCHROOT_TARGET_DIR', True) > + mount_ctx = "buildchroot-target" > else: > dep = "buildchroot-host:do_build" > rootfs = d.getVar('BUILDCHROOT_HOST_DIR', True) > + mount_ctx = "buildchroot-host" > > d.setVar('BUILDCHROOT_DEP', dep) > d.setVar('BUILDCHROOT_DIR', rootfs) > + d.setVar('MOUNTS_CONTEXT', mount_ctx + "-" + d.getVar('DISTRO') + "-" + d.getVar('DISTRO_ARCH')) > } > > -MOUNT_LOCKFILE = "${BUILDCHROOT_DIR}.lock" > +# mount settings > +BUILDCHROOT_MOUNTS = " \ > + bind:${REPO_ISAR_DIR}/${DISTRO}:${BUILDCHROOT_DIR}/isar-apt \ > + bind:${DL_DIR}:${BUILDCHROOT_DIR}/downloads \ > + rbind:/dev:${BUILDCHROOT_DIR}/dev \ > + proc::${BUILDCHROOT_DIR}/proc \ > + rbind:/sys:${BUILDCHROOT_DIR}/sys \ > + ${@oe.utils.vartrue("ISAR_USE_CACHED_BASE_REPO", "bind:${REPO_BASE_DIR}:${BUILDCHROOT_DIR}/base-apt", "", d)} \ > + " > + > +python () { > + # find all tasks that want to use buildchroot > + for task in [t for t in d.keys() if d.getVarFlag(t, 'task') and d.getVarFlag(t, 'buildchroot') == '1']: > + d.prependVarFlag(task, 'prefuncs', "buildchroot_task_prefunc ") > +} > > -buildchroot_do_mounts() { > +buildchroot_task_prefunc() { > sudo -s <<'EOSUDO' > - ( flock 9 > set -e > - > - mountpoint -q '${BUILDCHROOT_DIR}/isar-apt' || > - mount --bind '${REPO_ISAR_DIR}/${DISTRO}' '${BUILDCHROOT_DIR}/isar-apt' > - mountpoint -q '${BUILDCHROOT_DIR}/downloads' || > - mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads' > - mountpoint -q '${BUILDCHROOT_DIR}/dev' || > - mount --rbind /dev '${BUILDCHROOT_DIR}/dev' > - mount --make-rslave '${BUILDCHROOT_DIR}/dev' > - mountpoint -q '${BUILDCHROOT_DIR}/proc' || > - mount -t proc none '${BUILDCHROOT_DIR}/proc' > - mountpoint -q '${BUILDCHROOT_DIR}/sys' || > - mount --rbind /sys '${BUILDCHROOT_DIR}/sys' > - mount --make-rslave '${BUILDCHROOT_DIR}/sys' > - > - # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set > - if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ] > - then > - mkdir -p '${BUILDCHROOT_DIR}/base-apt' > - mountpoint -q '${BUILDCHROOT_DIR}/base-apt' || \ > - mount --bind '${REPO_BASE_DIR}' '${BUILDCHROOT_DIR}/base-apt' > - fi > - > - # Refresh or remove /etc/resolv.conf at this chance > + # Refresh or remove /etc/resolv.conf > if [ "${@repr(bb.utils.to_boolean(d.getVar('BB_NO_NETWORK')))}" = 'True' ] > then > - rm -rf '${BUILDCHROOT_DIR}/etc/resolv.conf' > - else > + rm -f '${BUILDCHROOT_DIR}/etc/resolv.conf' > + elif [ -d '${BUILDCHROOT_DIR}/etc' ]; then > cp -L /etc/resolv.conf '${BUILDCHROOT_DIR}/etc' > fi > - > - ) 9>'${MOUNT_LOCKFILE}' Which lock is protecting the remaining bits? If none, we need a word why that is fine. > EOSUDO > } > diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass > index 940e2fb..095e133 100644 > --- a/meta/classes/cpiogz-img.bbclass > +++ b/meta/classes/cpiogz-img.bbclass > @@ -10,12 +10,11 @@ CPIO_IMAGE_FORMAT ?= "newc" > > do_cpiogz_image() { > sudo rm -f ${CPIOGZ_IMAGE_FILE} > - image_do_mounts > sudo chroot ${BUILDCHROOT_DIR} \ > sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \ > /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o | /usr/bin/gzip -9 > \ > ${PP_DEPLOY}/${CPIOGZ_FNAME}" > sudo chown $(id -u):$(id -g) ${CPIOGZ_IMAGE_FILE} > } > - > +do_cpiogz_image[mounts] = "${IMAGE_MOUNTS}" > addtask cpiogz_image before do_image after do_image_tools > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index 8a39a6d..71c1acd 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -96,7 +96,6 @@ python() { > } > > do_apt_fetch() { > - dpkg_do_mounts > E="${@ isar_export_proxies(d)}" > sudo -E chroot ${BUILDCHROOT_DIR} /usr/bin/apt-get update \ > -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > @@ -107,10 +106,9 @@ do_apt_fetch() { > sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}" > done > - > - dpkg_undo_mounts > } > - > +do_apt_fetch[mounts] = "${DPKG_MOUNTS}" > +do_apt_fetch[buildchroot] = "1" > addtask apt_fetch after do_unpack before do_apt_unpack > do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > > @@ -119,7 +117,6 @@ do_apt_fetch[depends] = "${BUILDCHROOT_DEP}" > > do_apt_unpack() { > rm -rf ${S} > - dpkg_do_mounts > E="${@ isar_export_proxies(d)}" > > for uri in "${SRC_APT}"; do > @@ -132,10 +129,9 @@ do_apt_unpack() { > dpkg-source -x "${dscfile}" "${PPS}"' \ > my_script "${DISTRO}" "${uri}" > done > - > - dpkg_undo_mounts > } > - > +do_apt_unpack[mounts] = "${DPKG_MOUNTS}" > +do_apt_unpack[buildchroot] = "1" > addtask apt_unpack after do_apt_fetch before do_patch > > addtask cleanall_apt before do_cleanall > @@ -174,27 +170,7 @@ do_prepare_build[deptask] = "do_deploy_deb" > > BUILDROOT = "${BUILDCHROOT_DIR}/${PP}" > > -dpkg_do_mounts() { > - mkdir -p ${BUILDROOT} > - sudo mount --bind ${WORKDIR} ${BUILDROOT} > - > - buildchroot_do_mounts > -} > - > -dpkg_undo_mounts() { > - i=0 > - while ! sudo umount ${BUILDROOT}; do > - sleep 0.1 > - if [ `expr $i % 100` -eq 0 ] ; then > - bbwarn "${BUILDROOT}: Couldn't unmount ($i), retrying..." > - fi > - if [ $i -ge 10000 ]; then > - bbfatal "${BUILDROOT}: Couldn't unmount after timeout" > - fi > - i=`expr $i + 1` > - done > - sudo rmdir ${BUILDROOT} > -} > +DPKG_MOUNTS = "bind:${WORKDIR}:${BUILDROOT} ${BUILDCHROOT_MOUNTS}" > > # Placeholder for actual dpkg_runbuild() implementation > dpkg_runbuild() { > @@ -204,14 +180,13 @@ dpkg_runbuild() { > python do_dpkg_build() { > lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock", > shared=True) > - bb.build.exec_func("dpkg_do_mounts", d) > try: > bb.build.exec_func("dpkg_runbuild", d) > finally: > - bb.build.exec_func("dpkg_undo_mounts", d) > bb.utils.unlockfile(lock) > } > > +do_dpkg_build[mounts] = "${DPKG_MOUNTS}" > addtask dpkg_build before do_build > > KEEP_INSTALLED_ON_CLEAN ?= "0" > @@ -248,18 +223,15 @@ do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" > do_deploy_deb[dirs] = "${S}" > > python do_devshell() { > - bb.build.exec_func('dpkg_do_mounts', d) > - > isar_export_proxies(d) > > buildchroot = d.getVar('BUILDCHROOT_DIR') > pp_pps = os.path.join(d.getVar('PP'), d.getVar('PPS')) > termcmd = "sudo -E chroot {0} sh -c 'cd {1}; $SHELL -i'" > oe_terminal(termcmd.format(buildchroot, pp_pps), "Isar devshell", d) > - > - bb.build.exec_func('dpkg_undo_mounts', d) > } > - > +do_devshell[mounts] = "${DPKG_MOUNTS}" > +do_devshell[buildchroot] = "1" > addtask devshell after do_prepare_build > DEVSHELL_STARTDIR ?= "${S}" > do_devshell[dirs] = "${DEVSHELL_STARTDIR}" > diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass > index d956e8c..e3bf305 100644 > --- a/meta/classes/dpkg-gbp.bbclass > +++ b/meta/classes/dpkg-gbp.bbclass > @@ -13,7 +13,6 @@ GBP_DEPENDS ?= "git-buildpackage pristine-tar" > GBP_EXTRA_OPTIONS ?= "--git-pristine-tar" > > do_install_builddeps_append() { > - dpkg_do_mounts > distro="${DISTRO}" > if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then > distro="${HOST_DISTRO}" > @@ -26,7 +25,6 @@ do_install_builddeps_append() { > sudo -E chroot ${BUILDCHROOT_DIR} \ > apt-get install -y -o Debug::pkgProblemResolver=yes \ > --no-install-recommends ${GBP_DEPENDS} > - dpkg_undo_mounts > } > > dpkg_runbuild_prepend() { > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > index 4e7c2f7..96aefba 100644 > --- a/meta/classes/dpkg.bbclass > +++ b/meta/classes/dpkg.bbclass > @@ -7,7 +7,6 @@ PACKAGE_ARCH ?= "${DISTRO_ARCH}" > > # Install build dependencies for package > do_install_builddeps() { > - dpkg_do_mounts > E="${@ isar_export_proxies(d)}" > distro="${DISTRO}" > if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then > @@ -19,12 +18,13 @@ do_install_builddeps() { > deb_dl_dir_export "${BUILDCHROOT_DIR}" "${distro}" > sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \ > ${PP}/${PPS} ${PACKAGE_ARCH} > - dpkg_undo_mounts > } > > addtask install_builddeps after do_prepare_build before do_dpkg_build > # apt and reprepro may not run in parallel, acquire the Isar lock > do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > +do_install_builddeps[mounts] = "${DPKG_MOUNTS}" > +do_install_builddeps[buildchroot] = "1" > > addtask devshell after do_install_builddeps > > diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass > index 334dc64..84dd4f2 100644 > --- a/meta/classes/ext4-img.bbclass > +++ b/meta/classes/ext4-img.bbclass > @@ -13,11 +13,10 @@ do_ext4_image() { > > truncate -s ${ROOTFS_SIZE}K '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}' > > - image_do_mounts > - > sudo chroot ${BUILDCHROOT_DIR} /sbin/mke2fs ${MKE2FS_ARGS} \ > -F -d '${PP_ROOTFS}' '${PP_DEPLOY}/${EXT4_IMAGE_FILE}' > } > > addtask ext4_image before do_image after do_image_tools > do_ext4_image[prefuncs] = 'set_image_size' > +do_ext4_image[mounts] = "${IMAGE_MOUNTS}" > diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass > index 82b96d8..a34517a 100644 > --- a/meta/classes/fit-img.bbclass > +++ b/meta/classes/fit-img.bbclass > @@ -18,11 +18,11 @@ do_fit_image() { > > rm -f '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}' > > - image_do_mounts > - > # Create fit image using buildchroot tools > sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \ > -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${PP_DEPLOY}/${FIT_IMAGE_FILE}' > sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}' > } > + > addtask fit_image before do_image after do_image_tools do_transform_template > +do_fit_image[mounts] = "${IMAGE_MOUNTS}" > diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass > index 9f28800..265fb0c 100644 > --- a/meta/classes/image-tools-extension.bbclass > +++ b/meta/classes/image-tools-extension.bbclass > @@ -17,13 +17,13 @@ DEPENDS += "${IMAGER_BUILD_DEPS}" > do_install_imager_deps[depends] = "${BUILDCHROOT_DEP}" > do_install_imager_deps[deptask] = "do_deploy_deb" > do_install_imager_deps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > +do_install_imager_deps[mounts] = "${BUILDCHROOT_MOUNTS}" > +do_install_imager_deps[buildchroot] = "1" > do_install_imager_deps() { > if [ -z "${@d.getVar("IMAGER_INSTALL", True).strip()}" ]; then > exit > fi > > - buildchroot_do_mounts > - > E="${@ isar_export_proxies(d)}" > deb_dl_dir_import ${BUILDCHROOT_DIR} ${DISTRO} > sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' \ > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index ec93cab..dd934a7 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -52,15 +52,12 @@ DEPENDS += "${IMAGE_INSTALL}" > ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_core} describe --tags --dirty --match 'v[0-9].[0-9]*'" > ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}" > > -image_do_mounts() { > - sudo flock ${MOUNT_LOCKFILE} -c ' \ > - mkdir -p "${BUILDROOT_DEPLOY}" "${BUILDROOT_ROOTFS}" "${BUILDROOT_WORK}" > - mount --bind "${DEPLOY_DIR_IMAGE}" "${BUILDROOT_DEPLOY}" > - mount --bind "${IMAGE_ROOTFS}" "${BUILDROOT_ROOTFS}" > - mount --bind "${WORKDIR}" "${BUILDROOT_WORK}" > - ' > - buildchroot_do_mounts > -} > +IMAGE_MOUNTS = " \ > + bind:${DEPLOY_DIR_IMAGE}:${BUILDROOT_DEPLOY} \ > + bind:${IMAGE_ROOTFS}:${BUILDROOT_ROOTFS} \ > + bind:${WORKDIR}:${BUILDROOT_WORK} \ > + ${BUILDCHROOT_MOUNTS} \ > + " > > ROOTFSDIR = "${IMAGE_ROOTFS}" > ROOTFS_FEATURES += "clean-package-cache generate-manifest export-dpkg-status" > @@ -190,21 +187,6 @@ do_rootfs_finalize() { > find "${ROOTFSDIR}/usr/bin" \ > -maxdepth 1 -name 'qemu-*-static' -type f -delete > > - mountpoint -q '${ROOTFSDIR}/isar-apt' && \ > - umount -l ${ROOTFSDIR}/isar-apt > - rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-apt > - > - mountpoint -q '${ROOTFSDIR}/base-apt' && \ > - umount -l ${ROOTFSDIR}/base-apt > - rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/base-apt Who is deleting the empty isar-apt base-apt dirs? > - > - mountpoint -q '${ROOTFSDIR}/dev' && \ > - umount -l ${ROOTFSDIR}/dev > - mountpoint -q '${ROOTFSDIR}/sys' && \ > - umount -l ${ROOTFSDIR}/proc > - mountpoint -q '${ROOTFSDIR}/sys' && \ > - umount -l ${ROOTFSDIR}/sys > - > rm -f "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" > > rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list" > diff --git a/meta/classes/initramfs.bbclass b/meta/classes/initramfs.bbclass > index 10a642b..b5aba91 100644 > --- a/meta/classes/initramfs.bbclass > +++ b/meta/classes/initramfs.bbclass > @@ -25,8 +25,8 @@ ROOTFS_PACKAGES = "initramfs-tools ${INITRAMFS_PREINSTALL} ${INITRAMFS_INSTALL}" > inherit rootfs > > do_generate_initramfs[dirs] = "${DEPLOY_DIR_IMAGE}" > +do_generate_initramfs[mounts] = "${ROOTFS_MOUNTS}" > do_generate_initramfs() { > - rootfs_do_mounts > rootfs_do_qemu > > sudo -E chroot "${INITRAMFS_ROOTFS}" \ > diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass > index 92aff20..73419b4 100644 > --- a/meta/classes/isar-events.bbclass > +++ b/meta/classes/isar-events.bbclass > @@ -8,8 +8,6 @@ addhandler build_started > > python build_started() { > bb.utils.remove(d.getVar('TMPDIR') + "/work/*/*/*/temp/once.*") > - bb.utils.remove(d.getVar('TMPDIR') + "/work/*/*/*/rootfs.mount") > - bb.utils.remove(d.getVar('TMPDIR') + "/deploy/buildchroot-*/*.mount") > } > build_started[eventmask] = "bb.event.BuildStarted" > > @@ -54,7 +52,7 @@ python build_completed() { > with open('/proc/mounts') as f: > for line in f.readlines(): > if basepath in line: > - bb.debug(1, '%s left mounted, unmounting...' % line.split()[1]) > + bb.warn('%s left mounted, unmounting...' % line.split()[1]) Ah, you are brave and make this a real warning again. Means you've never seen it? > subprocess.call( > ["sudo", "umount", "-l", line.split()[1]], > stdout=subprocess.DEVNULL, > diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass > index f9151c5..7b9fbb1 100644 > --- a/meta/classes/rootfs.bbclass > +++ b/meta/classes/rootfs.bbclass > @@ -26,38 +26,17 @@ export LANG = "C" > export LANGUAGE = "C" > export LC_ALL = "C" > > -rootfs_do_mounts[weight] = "3" > -rootfs_do_mounts() { > - sudo -s <<'EOSUDO' > - mountpoint -q '${ROOTFSDIR}/dev' || \ > - mount --rbind /dev '${ROOTFSDIR}/dev' > - mount --make-rslave '${ROOTFSDIR}/dev' > - mountpoint -q '${ROOTFSDIR}/proc' || \ > - mount -t proc none '${ROOTFSDIR}/proc' > - mountpoint -q '${ROOTFSDIR}/sys' || \ > - mount --rbind /sys '${ROOTFSDIR}/sys' > - mount --make-rslave '${ROOTFSDIR}/sys' > - > - # Mount isar-apt if the directory does not exist or if it is empty > - # This prevents overwriting something that was copied there > - if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \ > - [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1 | wc -l)" = "0" ] > - then > - mkdir -p '${ROOTFSDIR}/isar-apt' > - mountpoint -q '${ROOTFSDIR}/isar-apt' || \ > - mount --bind '${REPO_ISAR_DIR}/${DISTRO}' '${ROOTFSDIR}/isar-apt' > - fi > - > - # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set > - if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ] > - then > - mkdir -p '${ROOTFSDIR}/base-apt' > - mountpoint -q '${ROOTFSDIR}/base-apt' || \ > - mount --bind '${REPO_BASE_DIR}' '${ROOTFSDIR}/base-apt' > - fi > - > -EOSUDO > -} > +MOUNTS_CONTEXT = "${PN}-${DISTRO}-${DISTRO_ARCH}" > +ROOTFS_MOUNTS = " \ > + rbind:/dev:${ROOTFSDIR}/dev \ > + proc::${ROOTFSDIR}/proc \ > + rbind:/sys:${ROOTFSDIR}/sys \ > + bind:${REPO_ISAR_DIR}/${DISTRO}:${ROOTFSDIR}/isar-apt \ > + ${@oe.utils.vartrue("ISAR_USE_CACHED_BASE_REPO", "bind:${REPO_BASE_DIR}:${ROOTFSDIR}/base-apt", "", d)} \ > + " > + > +mounts_task_prefunc[weight] = "3" > +mounts_task_postfunc[weight] = "3" > > rootfs_do_qemu() { > if [ '${@repr(d.getVar('ROOTFS_ARCH') == d.getVar('HOST_ARCH'))}' = 'False' ] > @@ -153,13 +132,15 @@ do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}" > do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}" > do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build" > do_rootfs_install[deptask] = "do_deploy_deb" > +do_rootfs_install[mounts] = "${ROOTFS_MOUNTS}" > +do_rootfs_install[mounts-noauto] = "1" Why "noauto"? How is it done otherwise? > python do_rootfs_install() { > configure_cmds = (d.getVar("ROOTFS_CONFIGURE_COMMAND", True) or "").split() > install_cmds = (d.getVar("ROOTFS_INSTALL_COMMAND", True) or "").split() > > # Mount after configure commands, so that they have time to copy > # 'isar-apt' (sdkchroot): > - cmds = ['rootfs_prepare'] + configure_cmds + ['rootfs_do_mounts'] + install_cmds > + cmds = ['rootfs_prepare'] + configure_cmds + ['mounts_task_prefunc'] + install_cmds + ['mounts_task_postfunc'] > > # NOTE: The weights specify how long each task takes in seconds and are used > # by the MultiStageProgressReporter to render a progress bar for this task. > @@ -230,9 +211,8 @@ rootfs_export_dpkg_status() { > } > > do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" > +do_rootfs_postprocess[mounts] = "${ROOTFS_MOUNTS}" > python do_rootfs_postprocess() { > - # Take care that its correctly mounted: > - bb.build.exec_func('rootfs_do_mounts', d) > # Take care that qemu-*-static is available, since it could have been > # removed on a previous execution of this task: > bb.build.exec_func('rootfs_do_qemu', d) > diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass > index c69ac4d..87f0187 100644 > --- a/meta/classes/ubi-img.bbclass > +++ b/meta/classes/ubi-img.bbclass > @@ -21,11 +21,10 @@ do_ubi_image() { > > rm -f '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}' > > - image_do_mounts > - > # Create ubi image using buildchroot tools > sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \ > -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}' '${PP_WORK}/${UBINIZE_CFG}' > sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}' > } > addtask ubi_image before do_image after do_image_tools do_transform_template > +do_ubi_image[mounts] = "${IMAGE_MOUNTS}" > diff --git a/meta/classes/ubifs-img.bbclass b/meta/classes/ubifs-img.bbclass > index 5d48c1d..c6775d6 100644 > --- a/meta/classes/ubifs-img.bbclass > +++ b/meta/classes/ubifs-img.bbclass > @@ -20,12 +20,11 @@ ISAR_CROSS_COMPILE_armhf = "1" > do_ubifs_image() { > rm -f '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}' > > - image_do_mounts > - > # Create ubifs image using buildchroot tools > sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \ > -r '${PP_ROOTFS}' '${PP_DEPLOY}/${UBIFS_IMAGE_FILE}' > sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}' > } > > +do_ubifs_image[mounts] = "${IMAGE_MOUNTS}" > addtask ubifs_image before do_image after do_image_tools > diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass > index b230af2..c6cfbf9 100644 > --- a/meta/classes/vm-img.bbclass > +++ b/meta/classes/vm-img.bbclass > @@ -33,13 +33,12 @@ CONVERSION_OPTIONS = "${@set_convert_options(d)}" > > do_convert_wic() { > rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}' > - image_do_mounts > bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${WIC_IMAGE_FILE}" > sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \ > '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}' > } > - > +do_convert_wic[mounts] = "${IMAGE_MOUNTS}" > addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template > > # User settings for OVA > @@ -92,8 +91,6 @@ do_create_ova() { > export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK}) > export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ") > > - image_do_mounts > - > sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO' > export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \ > | gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}') > @@ -112,5 +109,5 @@ do_create_ova() { > tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE} > EOSUDO > } > - > +do_create_ova[mounts] = "${IMAGE_MOUNTS}" > addtask do_create_ova after do_convert_wic before do_deploy > diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass > index d849ad9..f8f9c0d 100644 > --- a/meta/classes/wic-img.bbclass > +++ b/meta/classes/wic-img.bbclass > @@ -137,6 +137,17 @@ python check_for_wic_warnings() { > } > > do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}" > +do_wic_image[mounts] = "${BUILDCHROOT_MOUNTS}" > +python() { > + buildchroot = d.getVar('BUILDCHROOT_DIR') > + dirs = ((d.getVar('BBLAYERS') or '').split() + > + (d.getVar('STAGING_DIR') or '').split() + > + (d.getVar('SCRIPTSDIR') or '').split() + > + (d.getVar('BITBAKEDIR') or '').split()) > + for dir in dirs: > + d.appendVarFlag('do_wic_image', 'mounts', f" pbind:{dir}:{buildchroot}{dir}") > +} > + > python do_wic_image() { > bb.build.exec_func("generate_wic_image", d) > bb.build.exec_func("check_for_wic_warnings", d) > @@ -144,17 +155,6 @@ python do_wic_image() { > addtask wic_image before do_image after do_image_tools > > generate_wic_image() { > - buildchroot_do_mounts > - sudo -s <<'EOSUDO' > - ( flock 9 > - for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}; do > - mkdir -p ${BUILDCHROOT_DIR}/$dir > - if ! mountpoint ${BUILDCHROOT_DIR}/$dir >/dev/null 2>&1; then > - mount --bind --make-private $dir ${BUILDCHROOT_DIR}/$dir > - fi > - done > - ) 9>${MOUNT_LOCKFILE} > -EOSUDO > export FAKEROOTCMD=${FAKEROOTCMD} > export BUILDDIR=${BUILDDIR} > export MTOOLS_SKIP_CHECK=1 > @@ -200,13 +200,4 @@ EOSUDO > done > rm -rf ${BUILDCHROOT_DIR}/${WICTMP} > rm -rf ${IMAGE_ROOTFS}/../pseudo > - sudo -s <<'EOSUDO' > - ( flock 9 > - for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}; do > - if mountpoint -q ${BUILDCHROOT_DIR}/$dir; then > - umount ${BUILDCHROOT_DIR}/$dir > - fi > - done > - ) 9>${MOUNT_LOCKFILE} > -EOSUDO > } > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index b8af676..cc6c073 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -117,7 +117,6 @@ def get_apt_source_mirror(d, aptsources_entry_list): > mirror_list = [entry.split() > for entry in premirrors.split('\\n') > if any(entry)] > - > for regex, replace in mirror_list: > match = re.search(regex, aptsources_entry_list[2]) > > @@ -318,9 +317,6 @@ do_bootstrap() { > fi > echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" > echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" > - > - mkdir -p ${ROOTFSDIR}/base-apt > - mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt > else > install -v -m644 "${APTSRCS}" \ > "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" > @@ -364,13 +360,27 @@ do_bootstrap() { > install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" > "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" > > - # update APT > - mount --rbind /dev ${ROOTFSDIR}/dev > - mount --make-rslave ${ROOTFSDIR}/dev > - mount -t proc none ${ROOTFSDIR}/proc > - mount --rbind /sys ${ROOTFSDIR}/sys > - mount --make-rslave ${ROOTFSDIR}/sys > +EOSUDO > + deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_DISTRO}" > +} > > +addtask bootstrap before do_build after do_generate_keyrings > +do_bootstrap[vardeps] += " \ > + DISTRO_APT_PREMIRRORS \ > + ISAR_ENABLE_COMPAT_ARCH \ > + ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ > + " > +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" > +do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" > + > + > +do_bootstrap_finalize() { > + E="${@ isar_export_proxies(d)}" > + export BOOTSTRAP_FOR_HOST E > + > + deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_DISTRO}" > + > + sudo -E -s <<'EOSUDO' > export DEBIAN_FRONTEND=noninteractive > > if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then > @@ -386,18 +396,19 @@ do_bootstrap() { > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ > -o Debug::pkgProblemResolver=yes > > - umount -l "${ROOTFSDIR}/dev" > - umount -l "${ROOTFSDIR}/proc" > - umount -l "${ROOTFSDIR}/sys" > - umount -l "${ROOTFSDIR}/base-apt" || true > - > # Finalize debootstrap by setting the link in deploy > ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" > EOSUDO > deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_DISTRO}" > } > > -addtask bootstrap before do_build after do_generate_keyrings > +addtask bootstrap_finalize after do_bootstrap before do_build > +do_bootstrap_finalize[mounts] = " \ > + rbind:/dev:${ROOTFSDIR}/dev \ > + proc::${ROOTFSDIR}/proc \ > + rbind:/sys:${ROOTFSDIR}/sys \ > + ${@oe.utils.vartrue("ISAR_USE_CACHED_BASE_REPO", "bind:${REPO_BASE_DIR}:${ROOTFSDIR}/base-apt", "", d)} \ > + " > > CLEANFUNCS = "clean_deploy" > clean_deploy() { > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc > index 31524a1..ea4a3ba 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot.inc > +++ b/meta/recipes-devtools/buildchroot/buildchroot.inc > @@ -41,13 +41,7 @@ BUILDCHROOT_PREINSTALL_COMMON = " \ > equivs \ > adduser" > > -rootfs_do_mounts_append() { > - sudo -s <<'EOSUDO' > - mkdir -p '${BUILDCHROOT_DIR}/downloads' > - mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \ > - mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads' > -EOSUDO > -} > +ROOTFS_MOUNTS += "bind:${DL_DIR}:${BUILDCHROOT_DIR}/downloads" > > ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files" > buildchroot_install_files() { > Maybe split this up into multiple conversions of the same type? Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux