From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519532471426482176 X-Received: by 10.46.77.193 with SMTP id c62mr159561ljd.32.1518180015244; Fri, 09 Feb 2018 04:40:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.80.30 with SMTP id e30ls754501lfb.10.gmail; Fri, 09 Feb 2018 04:40:14 -0800 (PST) X-Google-Smtp-Source: AH8x226j1sZ7zEePKicb5Y/Y67FSormwgpw4WmLP/uY0vGq6MloZBh0NsWfZbkNWU74h8zWwuyIl X-Received: by 10.25.32.194 with SMTP id g185mr189676lfg.13.1518180014663; Fri, 09 Feb 2018 04:40:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518180014; cv=none; d=google.com; s=arc-20160816; b=O5nk7HjjjExii2Nu4v+A91HrzJua3s1/Uu+GdYgX8DFvRRdPGyCo9K/yVE3gPZ/JBI DU9hGPRGhR21GY4vTwf3h/yeOAu7BV5UtWfr0/LnS6NxT7jXApWHJlpeRucILTjhWSro beOBJhjqUA27GKgWBIgANQG7eqwTvpeK51X3KrhaER9uCWCGkuQJgoWgqruUZeMjgQWf zNzp3gKc5ohR3FSfL2vYI+V96LOUcKrNnSQ6BLFIItaFCXcLdjKSq2/TgEEtOoVX6RXo lJNN8ZhljIDPb4LrIaK87PMWJ3DqBJVETm68bKpZry/2Txsqtbc2UoWIK0nZloZSRRwC 3RLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=8zWN/zeDaqdn9NLrmR7ns++QY2ztnCLFnC0Y32Up2Kc=; b=FaO5h6z8HVZgwVAeAM8Lmz6dGsMle0JQVNSdlp50IwMNYDYYn37icYwd2sEVXenqtL GAWExpDFwYfKDhBBHfvVqR/lwOOVgBn76K8HkNzYUcUspod3RRKY4AxBQvgt65JKWXAs lZ4zPC7sEyFgzLE8rieVo6FlAMv+bcfhUs2QvSZI+0AGIqvhttcFsrjB3pdhv1Ic4MRZ tXWZQ0ZqmGnpirzV4yP6hqKzNUaQPlZbV+tgyN6KlY3ytks7LceTS0CNpQ3Yn9KTIDRe vGS8Rpw0nVlr/d3wB72twz8/WrHdUG/ZO1N9czOiA2vpYJH31GlSzSv6PL5KXo4dGq0D fDhg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id w29si150882lfc.5.2018.02.09.04.40.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Feb 2018 04:40:14 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w19CeDpo012052 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 9 Feb 2018 13:40:13 +0100 Received: from mmd1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w19CeDLE017266; Fri, 9 Feb 2018 13:40:13 +0100 Date: Fri, 9 Feb 2018 13:40:13 +0100 From: Henning Schild To: Jan Kiszka Cc: Alexander Smirnov , Subject: Re: [PATCH] isar: Clean mount point on bitbake exit Message-ID: <20180209134013.022008e2@mmd1pvb1c.ad001.siemens.net> In-Reply-To: <0fe2f7a9-4a02-9abd-7a97-44605f4f865b@siemens.com> References: <20180206195516.32153-1-asmirnov@ilbers.de> <20180209133340.681c00b5@mmd1pvb1c.ad001.siemens.net> <0fe2f7a9-4a02-9abd-7a97-44605f4f865b@siemens.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: O+fkrpLGDF1F Am Fri, 9 Feb 2018 13:35:15 +0100 schrieb Jan Kiszka : > On 2018-02-09 13:33, [ext] Henning Schild wrote: > > Hi, > > > > this patch is causing problems when building in a docker container, > > because sysfs can only be mounted ro. (Subject: current next bash in > > buildchroot problem) > > Now we could discuss whether we should relax the security of our > > containers even more, or whether Isar should care about that > > use-case. > > > > But this patch actually does several things at a time, it changes > > the way we mount and adds three new mounts. I would suggest to > > split it up so we can discuss the issues with dev and sys while > > already merging the rest. > > I think (didn't check if there was an update of next this morning) it > works for me - in Docker. How are you starting the container? docker run -e USER_ID=$(id -u) --rm -t -i --cap-add=SYS_ADMIN --cap-add=MKNOD --device $(/sbin/losetup -f) -e ... proxy stuff ... inside my sysfs is ro, a bind-mount of sysfs is ro and a "mount -t sysfs ..." will be ro. Maybe i could add a "-o rw" to the mount but for now i just reverted the two patches that deal with mounting. Might also be a difference in our host systems. Henning > Jan > > > > > Henning > > > > Am Tue, 6 Feb 2018 22:55:16 +0300 > > schrieb Alexander Smirnov : > > > >> 8<-- > >> > >> That's it! Branch 'asmirnov/devel', please test and enjoy :-) > >> > >> 8<-- > >> > >> Now each multiconfig has registered handler for BuildCompleted > >> event (see class 'isar-event.bbclass'). Moreover, the > >> '/proc/mounts' file contains all the active mounts. In addition, > >> from event handler we could derive all the variables like > >> ${TMPDIR}, ${DISTRO} etc. So it's possible to find all the active > >> mounts for current multiconfig and clean them. > >> > >> NOTE: if build is interrupted by double ^C, some mount points could > >> stay uncleaned. This is caused by remaining processes started by > >> bitbake, for example: > >> - 'chroot build.sh ...' > >> - 'multistrap ...' > >> > >> So please be careful when interrupting build. > >> > >> Signed-off-by: Alexander Smirnov > >> --- > >> meta-isar/recipes-core/images/isar-image-base.bb | 11 ++++------ > >> meta/classes/dpkg-base.bbclass | 12 > >> ++++------- meta/classes/isar-events.bbclass | 15 > >> +++++++++++--- meta/recipes-devtools/buildchroot/buildchroot.bb | > >> 24 > >> +++++++++------------- .../buildchroot/files/configscript.sh > >> | 4 ---- .../buildchroot/files/download_dev-random | 13 > >> ------------ 6 files changed, 30 insertions(+), 49 deletions(-) > >> delete mode 100644 > >> meta/recipes-devtools/buildchroot/files/download_dev-random > >> > >> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb > >> b/meta-isar/recipes-core/images/isar-image-base.bb index > >> e359ac3..8ddbabb 100644 --- > >> a/meta-isar/recipes-core/images/isar-image-base.bb +++ > >> b/meta-isar/recipes-core/images/isar-image-base.bb @@ -55,14 +55,10 > >> @@ do_rootfs() { -e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \ > >> "${WORKDIR}/multistrap.conf.in" > > >> "${WORKDIR}/multistrap.conf" > >> + # Do not use bitbake flag [dirs] here because this folder > >> should have > >> + # specific ownership. > >> [ ! -d ${IMAGE_ROOTFS}/proc ] && sudo install -d -o 0 -g 0 -m > >> 555 ${IMAGE_ROOTFS}/proc sudo mount -t proc none > >> ${IMAGE_ROOTFS}/proc > >> - _do_rootfs_cleanup() { > >> - ret=$? > >> - sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true > >> - (exit $ret) || bb_exit_handler > >> - } > >> - trap '_do_rootfs_cleanup' EXIT > >> > >> # Create root filesystem. We must use sudo -E here to preserve > >> the environment # because of proxy settings > >> @@ -72,5 +68,6 @@ do_rootfs() { > >> sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} > >> ${MACHINE_SERIAL} ${BAUDRATE_TTY} \ ${ROOTFS_DEV} > >> sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}" > >> - _do_rootfs_cleanup > >> + > >> + sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true > >> } > >> diff --git a/meta/classes/dpkg-base.bbclass > >> b/meta/classes/dpkg-base.bbclass index 5d5a924..a34c21f 100644 > >> --- a/meta/classes/dpkg-base.bbclass > >> +++ b/meta/classes/dpkg-base.bbclass > >> @@ -20,15 +20,11 @@ dpkg_runbuild() { > >> do_build() { > >> mkdir -p ${BUILDROOT} > >> sudo mount --bind ${WORKDIR} ${BUILDROOT} > >> - _do_build_cleanup() { > >> - ret=$? > >> - sudo umount ${BUILDROOT} 2>/dev/null || true > >> - sudo rmdir ${BUILDROOT} 2>/dev/null || true > >> - (exit $ret) || bb_exit_handler > >> - } > >> - trap '_do_build_cleanup' EXIT > >> + > >> dpkg_runbuild > >> - _do_build_cleanup > >> + > >> + sudo umount ${BUILDROOT} 2>/dev/null || true > >> + sudo rmdir ${BUILDROOT} 2>/dev/null || true > >> } > >> > >> # Install package to Isar-apt > >> diff --git a/meta/classes/isar-events.bbclass > >> b/meta/classes/isar-events.bbclass index 55fc106..ae0f791 100644 > >> --- a/meta/classes/isar-events.bbclass > >> +++ b/meta/classes/isar-events.bbclass > >> @@ -11,10 +11,19 @@ python isar_handler () { > >> devnull = open(os.devnull, 'w') > >> > >> if isinstance(e, bb.event.BuildCompleted): > >> - bchroot = d.getVar('BUILDCHROOT_DIR', True) > >> + tmpdir = d.getVar('TMPDIR', True) > >> + distro = d.getVar('DISTRO', True) > >> + arch = d.getVar('DISTRO_ARCH', True) > >> > >> - # Clean up buildchroot > >> - subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + > >> '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, > >> shell=True) > >> + w = tmpdir + '/work/' + distro + '-' + arch > >> + > >> + # '/proc/mounts' contains all the active mounts, so > >> knowing 'w' we > >> + # could get the list of mounts for the specific > >> multiconfig and > >> + # clean them. > >> + with open('/proc/mounts', 'rU') as f: > >> + for line in f: > >> + if w in line: > >> + subprocess.call('sudo umount -f ' + > >> line.split()[1], stdout=devnull, stderr=devnull, shell=True) > >> devnull.close() > >> } > >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb > >> b/meta/recipes-devtools/buildchroot/buildchroot.bb index > >> 304c67e..df9df19 100644 --- > >> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ > >> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -12,7 +12,6 > >> @@ FILESPATH =. > >> "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" SRC_URI = > >> "file://multistrap.conf.in \ file://configscript.sh \ > >> file://setup.sh \ > >> - file://download_dev-random \ > >> file://build.sh" > >> PV = "1.0" > >> > >> @@ -32,8 +31,10 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ > >> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > >> > >> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> -do_build[dirs] = "${WORKDIR}/hooks_multistrap \ > >> - ${BUILDCHROOT_DIR}/isar-apt" > >> +do_build[dirs] = "${BUILDCHROOT_DIR}/isar-apt \ > >> + ${BUILDCHROOT_DIR}/dev \ > >> + ${BUILDCHROOT_DIR}/proc \ > >> + ${BUILDCHROOT_DIR}/sys" > >> do_build[depends] = "isar-apt:do_cache_config" > >> > >> do_build() { > >> @@ -41,7 +42,6 @@ do_build() { > >> > >> chmod +x "${WORKDIR}/setup.sh" > >> chmod +x "${WORKDIR}/configscript.sh" > >> - install -m 755 "${WORKDIR}/download_dev-random" > >> "${WORKDIR}/hooks_multistrap/" > >> # Multistrap accepts only relative path in configuration > >> files, so get it: cd ${TOPDIR} > >> @@ -60,15 +60,6 @@ do_build() { > >> -e > >> 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ > >> "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" > >> - [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 > >> ${BUILDCHROOT_DIR}/proc > >> - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > >> - _do_build_cleanup() { > >> - ret=$? > >> - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > >> - (exit $ret) || bb_exit_handler > >> - } > >> - trap '_do_build_cleanup' EXIT > >> - > >> do_setup_mounts > >> > >> # Create root filesystem > >> @@ -79,7 +70,6 @@ do_build() { > >> > >> # Configure root filesystem > >> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > >> - _do_build_cleanup > >> > >> do_cleanup_mounts > >> } > >> @@ -96,10 +86,16 @@ do_setup_mounts[stamp-extra-info] = > >> "${DISTRO}-${DISTRO_ARCH}" > >> do_setup_mounts() { > >> sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} > >> ${BUILDCHROOT_DIR}/isar-apt > >> + sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev > >> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > >> + sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys > >> } > >> > >> addtask setup_mounts after do_build > >> > >> do_cleanup_mounts() { > >> sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true > >> + sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true > >> + sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > >> + sudo umount ${BUILDCHROOT_DIR}/sys 2>/dev/null || true > >> } > >> diff --git > >> a/meta/recipes-devtools/buildchroot/files/configscript.sh > >> b/meta/recipes-devtools/buildchroot/files/configscript.sh index > >> 9813c9a..524e50c 100644 --- > >> a/meta/recipes-devtools/buildchroot/files/configscript.sh +++ > >> b/meta/recipes-devtools/buildchroot/files/configscript.sh @@ > >> -39,10 +39,6 @@ export LC_ALL=C LANGUAGE=C LANG=C #run pre > >> installation script /var/lib/dpkg/info/dash.preinst install > >> -# apt-get http method, gpg require /dev/null > >> -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev > >> - > >> #configuring packages > >> dpkg --configure -a > >> apt-get update > >> -umount /dev > >> diff --git > >> a/meta/recipes-devtools/buildchroot/files/download_dev-random > >> b/meta/recipes-devtools/buildchroot/files/download_dev-random > >> deleted file mode 100644 index 5b5b96b..0000000 --- > >> a/meta/recipes-devtools/buildchroot/files/download_dev-random > >> +++ /dev/null @@ -1,13 +0,0 @@ > >> -#!/bin/sh > >> - > >> -set -e > >> - > >> -readonly ROOTFS="$1" > >> - > >> -mknod "${ROOTFS}/dev/random" c 1 8 > >> -chmod 640 "${ROOTFS}/dev/random" > >> -chown 0:0 "${ROOTFS}/dev/random" > >> - > >> -mknod "${ROOTFS}/dev/urandom" c 1 9 > >> -chmod 640 "${ROOTFS}/dev/urandom" > >> -chown 0:0 "${ROOTFS}/dev/urandom" > > >