From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519532471426482176 X-Received: by 10.80.230.18 with SMTP id y18mr744976edm.2.1518179621977; Fri, 09 Feb 2018 04:33:41 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.171.6 with SMTP id s6ls3474186edc.2.gmail; Fri, 09 Feb 2018 04:33:41 -0800 (PST) X-Google-Smtp-Source: AH8x226i+z8jtiZZfmWTzJYtBK4f8oa78I+kpjsi9l8JGTF+Xvpl12YUC7B9d46iZ179kRCMmnKd X-Received: by 10.80.201.74 with SMTP id p10mr740573edh.7.1518179621294; Fri, 09 Feb 2018 04:33:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518179621; cv=none; d=google.com; s=arc-20160816; b=W9KFDcYF9gmpHWEMT4nhfcVoekuUlqB3CdziZ0HlOvfL5qqM0RaXo0PqB/QCUNRGAb sG6e1/01ErADu8kJ0kn87MfJZOQtvtVZ6kek6puoreDKR35S0yYIEfR4pZqMD2ugobSV DQkYVJf3JpMFsKBQJ+a0QrDtjSiCghhNnEmMIDJpgimjuSfblJopcXovdE/SRlV+2VQ2 RJusuznYgYEI+G0KZ7cBkE7npuk24OC597vl2co9tUqOKKdqjIfbWJHyoxLufTUYvH4a dLwFXYOMbraMoXocai+guPbDG+PKMPTO5djX4PknF52aQ+ywnEdzNf8gAgEsGo01IE/2 cVtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=d5kBX+AoKwsZg/ON0WTyQMUfp3feFXkBsV+aNS0EcTI=; b=tyCm17bW0gxm/rBrvtMSGCaOtq4UgMQyCVsh9bAYTzNob7vW0YsMYgWX3INI8U8hCq VlH1IVarOE5Xg/037/jb/W/R6XpbElO36IehGSGcV5hlEP4UQUPih20gqmDBly2Gz1a7 xPUJN8P9jUNalgvxieQA4vVgDINJL5y6lDXOwq9SHAy2GyKXTK2vAgNkAQWJQQuDPdfD TJMDz/YvUhe+bx60jjvUl/9mZ5QyT92+zuxbIQSH5NvIq0EL8aQcBVW2um7eCsyvwC9P gDDy6xWVNFP0ASSRtC7y55Nvmd6pIar0GMltW+kFwGgScHccgC6wDEER9S1xGwF2Yt3H zLLA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id t4si136373edt.2.2018.02.09.04.33.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Feb 2018 04:33:41 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w19CXeDj009915 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 9 Feb 2018 13:33:40 +0100 Received: from mmd1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w19CXeKe016984; Fri, 9 Feb 2018 13:33:40 +0100 Date: Fri, 9 Feb 2018 13:33:40 +0100 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH] isar: Clean mount point on bitbake exit Message-ID: <20180209133340.681c00b5@mmd1pvb1c.ad001.siemens.net> In-Reply-To: <20180206195516.32153-1-asmirnov@ilbers.de> References: <20180206195516.32153-1-asmirnov@ilbers.de> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: fuHEJRDlO5dz Hi, this patch is causing problems when building in a docker container, because sysfs can only be mounted ro. (Subject: current next bash in buildchroot problem) Now we could discuss whether we should relax the security of our containers even more, or whether Isar should care about that use-case. But this patch actually does several things at a time, it changes the way we mount and adds three new mounts. I would suggest to split it up so we can discuss the issues with dev and sys while already merging the rest. Henning Am Tue, 6 Feb 2018 22:55:16 +0300 schrieb Alexander Smirnov : > 8<-- > > That's it! Branch 'asmirnov/devel', please test and enjoy :-) > > 8<-- > > Now each multiconfig has registered handler for BuildCompleted event > (see class 'isar-event.bbclass'). Moreover, the '/proc/mounts' file > contains all the active mounts. In addition, from event handler we > could derive all the variables like ${TMPDIR}, ${DISTRO} etc. So it's > possible to find all the active mounts for current multiconfig and > clean them. > > NOTE: if build is interrupted by double ^C, some mount points could > stay uncleaned. This is caused by remaining processes started by > bitbake, for example: > - 'chroot build.sh ...' > - 'multistrap ...' > > So please be careful when interrupting build. > > Signed-off-by: Alexander Smirnov > --- > meta-isar/recipes-core/images/isar-image-base.bb | 11 ++++------ > meta/classes/dpkg-base.bbclass | 12 ++++------- > meta/classes/isar-events.bbclass | 15 > +++++++++++--- meta/recipes-devtools/buildchroot/buildchroot.bb | > 24 > +++++++++------------- .../buildchroot/files/configscript.sh > | 4 ---- .../buildchroot/files/download_dev-random | 13 > ------------ 6 files changed, 30 insertions(+), 49 deletions(-) > delete mode 100644 > meta/recipes-devtools/buildchroot/files/download_dev-random > > diff --git a/meta-isar/recipes-core/images/isar-image-base.bb > b/meta-isar/recipes-core/images/isar-image-base.bb index > e359ac3..8ddbabb 100644 --- > a/meta-isar/recipes-core/images/isar-image-base.bb +++ > b/meta-isar/recipes-core/images/isar-image-base.bb @@ -55,14 +55,10 > @@ do_rootfs() { -e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \ > "${WORKDIR}/multistrap.conf.in" > > "${WORKDIR}/multistrap.conf" > + # Do not use bitbake flag [dirs] here because this folder should > have > + # specific ownership. > [ ! -d ${IMAGE_ROOTFS}/proc ] && sudo install -d -o 0 -g 0 -m > 555 ${IMAGE_ROOTFS}/proc sudo mount -t proc none ${IMAGE_ROOTFS}/proc > - _do_rootfs_cleanup() { > - ret=$? > - sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true > - (exit $ret) || bb_exit_handler > - } > - trap '_do_rootfs_cleanup' EXIT > > # Create root filesystem. We must use sudo -E here to preserve > the environment # because of proxy settings > @@ -72,5 +68,6 @@ do_rootfs() { > sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} > ${MACHINE_SERIAL} ${BAUDRATE_TTY} \ ${ROOTFS_DEV} > sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}" > - _do_rootfs_cleanup > + > + sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true > } > diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass index 5d5a924..a34c21f 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -20,15 +20,11 @@ dpkg_runbuild() { > do_build() { > mkdir -p ${BUILDROOT} > sudo mount --bind ${WORKDIR} ${BUILDROOT} > - _do_build_cleanup() { > - ret=$? > - sudo umount ${BUILDROOT} 2>/dev/null || true > - sudo rmdir ${BUILDROOT} 2>/dev/null || true > - (exit $ret) || bb_exit_handler > - } > - trap '_do_build_cleanup' EXIT > + > dpkg_runbuild > - _do_build_cleanup > + > + sudo umount ${BUILDROOT} 2>/dev/null || true > + sudo rmdir ${BUILDROOT} 2>/dev/null || true > } > > # Install package to Isar-apt > diff --git a/meta/classes/isar-events.bbclass > b/meta/classes/isar-events.bbclass index 55fc106..ae0f791 100644 > --- a/meta/classes/isar-events.bbclass > +++ b/meta/classes/isar-events.bbclass > @@ -11,10 +11,19 @@ python isar_handler () { > devnull = open(os.devnull, 'w') > > if isinstance(e, bb.event.BuildCompleted): > - bchroot = d.getVar('BUILDCHROOT_DIR', True) > + tmpdir = d.getVar('TMPDIR', True) > + distro = d.getVar('DISTRO', True) > + arch = d.getVar('DISTRO_ARCH', True) > > - # Clean up buildchroot > - subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + > '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) > + w = tmpdir + '/work/' + distro + '-' + arch > + > + # '/proc/mounts' contains all the active mounts, so knowing > 'w' we > + # could get the list of mounts for the specific multiconfig > and > + # clean them. > + with open('/proc/mounts', 'rU') as f: > + for line in f: > + if w in line: > + subprocess.call('sudo umount -f ' + > line.split()[1], stdout=devnull, stderr=devnull, shell=True) > devnull.close() > } > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb > b/meta/recipes-devtools/buildchroot/buildchroot.bb index > 304c67e..df9df19 100644 --- > a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ > b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -12,7 +12,6 @@ > FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" > SRC_URI = "file://multistrap.conf.in \ file://configscript.sh \ > file://setup.sh \ > - file://download_dev-random \ > file://build.sh" > PV = "1.0" > > @@ -32,8 +31,10 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > -do_build[dirs] = "${WORKDIR}/hooks_multistrap \ > - ${BUILDCHROOT_DIR}/isar-apt" > +do_build[dirs] = "${BUILDCHROOT_DIR}/isar-apt \ > + ${BUILDCHROOT_DIR}/dev \ > + ${BUILDCHROOT_DIR}/proc \ > + ${BUILDCHROOT_DIR}/sys" > do_build[depends] = "isar-apt:do_cache_config" > > do_build() { > @@ -41,7 +42,6 @@ do_build() { > > chmod +x "${WORKDIR}/setup.sh" > chmod +x "${WORKDIR}/configscript.sh" > - install -m 755 "${WORKDIR}/download_dev-random" > "${WORKDIR}/hooks_multistrap/" > # Multistrap accepts only relative path in configuration files, > so get it: cd ${TOPDIR} > @@ -60,15 +60,6 @@ do_build() { > -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ > "${WORKDIR}/multistrap.conf.in" > > "${WORKDIR}/multistrap.conf" > - [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 > ${BUILDCHROOT_DIR}/proc > - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > - _do_build_cleanup() { > - ret=$? > - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > - (exit $ret) || bb_exit_handler > - } > - trap '_do_build_cleanup' EXIT > - > do_setup_mounts > > # Create root filesystem > @@ -79,7 +70,6 @@ do_build() { > > # Configure root filesystem > sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > - _do_build_cleanup > > do_cleanup_mounts > } > @@ -96,10 +86,16 @@ do_setup_mounts[stamp-extra-info] = > "${DISTRO}-${DISTRO_ARCH}" > do_setup_mounts() { > sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} > ${BUILDCHROOT_DIR}/isar-apt > + sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev > + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > + sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys > } > > addtask setup_mounts after do_build > > do_cleanup_mounts() { > sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true > + sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true > + sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > + sudo umount ${BUILDCHROOT_DIR}/sys 2>/dev/null || true > } > diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh > b/meta/recipes-devtools/buildchroot/files/configscript.sh index > 9813c9a..524e50c 100644 --- > a/meta/recipes-devtools/buildchroot/files/configscript.sh +++ > b/meta/recipes-devtools/buildchroot/files/configscript.sh @@ -39,10 > +39,6 @@ export LC_ALL=C LANGUAGE=C LANG=C #run pre installation > script /var/lib/dpkg/info/dash.preinst install > > -# apt-get http method, gpg require /dev/null > -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev > - > #configuring packages > dpkg --configure -a > apt-get update > -umount /dev > diff --git > a/meta/recipes-devtools/buildchroot/files/download_dev-random > b/meta/recipes-devtools/buildchroot/files/download_dev-random deleted > file mode 100644 index 5b5b96b..0000000 --- > a/meta/recipes-devtools/buildchroot/files/download_dev-random > +++ /dev/null @@ -1,13 +0,0 @@ > -#!/bin/sh > - > -set -e > - > -readonly ROOTFS="$1" > - > -mknod "${ROOTFS}/dev/random" c 1 8 > -chmod 640 "${ROOTFS}/dev/random" > -chown 0:0 "${ROOTFS}/dev/random" > - > -mknod "${ROOTFS}/dev/urandom" c 1 9 > -chmod 640 "${ROOTFS}/dev/urandom" > -chown 0:0 "${ROOTFS}/dev/urandom"