From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519532471426482176 X-Received: by 10.28.178.207 with SMTP id b198mr262372wmf.0.1518179718336; Fri, 09 Feb 2018 04:35:18 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.165.85 with SMTP id j21ls2177554wrb.12.gmail; Fri, 09 Feb 2018 04:35:17 -0800 (PST) X-Google-Smtp-Source: AH8x227pt+X4nrpBU/jqsbbFAQOkqmCIEXv1Uxud8nd5UBudb82zZQMc5R5+KcRGjQ+7fX9emLZe X-Received: by 10.223.201.4 with SMTP id m4mr301230wrh.5.1518179717718; Fri, 09 Feb 2018 04:35:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518179717; cv=none; d=google.com; s=arc-20160816; b=tqf/VvZU1rvHhyL5mcftYt5QQs5cetYymgoDGl57PdLcjzPkkZx0kgkxeKBq3mA+Um EjIE9e753pxxDDaYbo1LJIr15Wn6QLBx+U80wzM4ZGkqM2FTvSHXpl5EL7PNI8BcvMFq fTzY3wMroZhq696ygOOiRn+QYb8YnyrFKSxR0CqVFNAeOB69qJ2ifBXIDXT3OwfACWjA vw1W6xWDdWbh6n3zO28EX2lDpBaH27dtfgmKayOM/ewPfcj5ZjB0K4JlBFXkMUtPhMTg jYhTulUtlhRZ5pKNNrDukzMtlAHklnXhRhSlQQdEDAmz6x7c5xBYahIc5Ab9bgz0Nz8G uXJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=XbQyOJRzmBa+T4CyEIMysuoupVVBI//Qr1WzYzcwfu0=; b=B3obyxVbszWpmy66RL5x0CzPiCUgJneT6S7tNKAXluNYCdPi4doA0lafnvMNdIFHst nQInDggI+vVbbss4Trpbwf4Q4+CeWYluIOLNmPBABF+SmXNutEmpvfXnh7ydzFelwa0K 27N0hejDygl6oskJxhnacTyj+3HqNCSLM4WQbodlWF0XH12Tf3RaZZTkLxUNkMmsUMI6 4ZlI8+UxNhucz1DaaAtVCxJWmp7UJLIA1tbAx89rmfmMgd6yTRmpThnVZLplnAJUu7Aj bTcJ/X4CAuCGbX8orFYwO+c10P72EWEVJSMw7JhqLZktAPtanbIVPIMT2VQ43Cnn7RU5 Q3FQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id 200si161598wml.2.2018.02.09.04.35.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Feb 2018 04:35:17 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w19CZHLu013045 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 9 Feb 2018 13:35:17 +0100 Received: from [139.25.246.30] ([139.25.246.30]) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w19CZGXS025775; Fri, 9 Feb 2018 13:35:16 +0100 Subject: Re: [PATCH] isar: Clean mount point on bitbake exit To: "[ext] Henning Schild" , Alexander Smirnov Cc: isar-users@googlegroups.com References: <20180206195516.32153-1-asmirnov@ilbers.de> <20180209133340.681c00b5@mmd1pvb1c.ad001.siemens.net> From: Jan Kiszka Message-ID: <0fe2f7a9-4a02-9abd-7a97-44605f4f865b@siemens.com> Date: Fri, 9 Feb 2018 13:35:15 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <20180209133340.681c00b5@mmd1pvb1c.ad001.siemens.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: fXgLuD7RrF+8 On 2018-02-09 13:33, [ext] Henning Schild wrote: > Hi, > > this patch is causing problems when building in a docker container, > because sysfs can only be mounted ro. (Subject: current next bash in > buildchroot problem) > Now we could discuss whether we should relax the security of our > containers even more, or whether Isar should care about that use-case. > > But this patch actually does several things at a time, it changes the > way we mount and adds three new mounts. I would suggest to split it up > so we can discuss the issues with dev and sys while already merging the > rest. I think (didn't check if there was an update of next this morning) it works for me - in Docker. How are you starting the container? Jan > > Henning > > Am Tue, 6 Feb 2018 22:55:16 +0300 > schrieb Alexander Smirnov : > >> 8<-- >> >> That's it! Branch 'asmirnov/devel', please test and enjoy :-) >> >> 8<-- >> >> Now each multiconfig has registered handler for BuildCompleted event >> (see class 'isar-event.bbclass'). Moreover, the '/proc/mounts' file >> contains all the active mounts. In addition, from event handler we >> could derive all the variables like ${TMPDIR}, ${DISTRO} etc. So it's >> possible to find all the active mounts for current multiconfig and >> clean them. >> >> NOTE: if build is interrupted by double ^C, some mount points could >> stay uncleaned. This is caused by remaining processes started by >> bitbake, for example: >> - 'chroot build.sh ...' >> - 'multistrap ...' >> >> So please be careful when interrupting build. >> >> Signed-off-by: Alexander Smirnov >> --- >> meta-isar/recipes-core/images/isar-image-base.bb | 11 ++++------ >> meta/classes/dpkg-base.bbclass | 12 ++++------- >> meta/classes/isar-events.bbclass | 15 >> +++++++++++--- meta/recipes-devtools/buildchroot/buildchroot.bb | >> 24 >> +++++++++------------- .../buildchroot/files/configscript.sh >> | 4 ---- .../buildchroot/files/download_dev-random | 13 >> ------------ 6 files changed, 30 insertions(+), 49 deletions(-) >> delete mode 100644 >> meta/recipes-devtools/buildchroot/files/download_dev-random >> >> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb >> b/meta-isar/recipes-core/images/isar-image-base.bb index >> e359ac3..8ddbabb 100644 --- >> a/meta-isar/recipes-core/images/isar-image-base.bb +++ >> b/meta-isar/recipes-core/images/isar-image-base.bb @@ -55,14 +55,10 >> @@ do_rootfs() { -e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \ >> "${WORKDIR}/multistrap.conf.in" > >> "${WORKDIR}/multistrap.conf" >> + # Do not use bitbake flag [dirs] here because this folder should >> have >> + # specific ownership. >> [ ! -d ${IMAGE_ROOTFS}/proc ] && sudo install -d -o 0 -g 0 -m >> 555 ${IMAGE_ROOTFS}/proc sudo mount -t proc none ${IMAGE_ROOTFS}/proc >> - _do_rootfs_cleanup() { >> - ret=$? >> - sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true >> - (exit $ret) || bb_exit_handler >> - } >> - trap '_do_rootfs_cleanup' EXIT >> >> # Create root filesystem. We must use sudo -E here to preserve >> the environment # because of proxy settings >> @@ -72,5 +68,6 @@ do_rootfs() { >> sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} >> ${MACHINE_SERIAL} ${BAUDRATE_TTY} \ ${ROOTFS_DEV} >> sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}" >> - _do_rootfs_cleanup >> + >> + sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true >> } >> diff --git a/meta/classes/dpkg-base.bbclass >> b/meta/classes/dpkg-base.bbclass index 5d5a924..a34c21f 100644 >> --- a/meta/classes/dpkg-base.bbclass >> +++ b/meta/classes/dpkg-base.bbclass >> @@ -20,15 +20,11 @@ dpkg_runbuild() { >> do_build() { >> mkdir -p ${BUILDROOT} >> sudo mount --bind ${WORKDIR} ${BUILDROOT} >> - _do_build_cleanup() { >> - ret=$? >> - sudo umount ${BUILDROOT} 2>/dev/null || true >> - sudo rmdir ${BUILDROOT} 2>/dev/null || true >> - (exit $ret) || bb_exit_handler >> - } >> - trap '_do_build_cleanup' EXIT >> + >> dpkg_runbuild >> - _do_build_cleanup >> + >> + sudo umount ${BUILDROOT} 2>/dev/null || true >> + sudo rmdir ${BUILDROOT} 2>/dev/null || true >> } >> >> # Install package to Isar-apt >> diff --git a/meta/classes/isar-events.bbclass >> b/meta/classes/isar-events.bbclass index 55fc106..ae0f791 100644 >> --- a/meta/classes/isar-events.bbclass >> +++ b/meta/classes/isar-events.bbclass >> @@ -11,10 +11,19 @@ python isar_handler () { >> devnull = open(os.devnull, 'w') >> >> if isinstance(e, bb.event.BuildCompleted): >> - bchroot = d.getVar('BUILDCHROOT_DIR', True) >> + tmpdir = d.getVar('TMPDIR', True) >> + distro = d.getVar('DISTRO', True) >> + arch = d.getVar('DISTRO_ARCH', True) >> >> - # Clean up buildchroot >> - subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + >> '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) >> + w = tmpdir + '/work/' + distro + '-' + arch >> + >> + # '/proc/mounts' contains all the active mounts, so knowing >> 'w' we >> + # could get the list of mounts for the specific multiconfig >> and >> + # clean them. >> + with open('/proc/mounts', 'rU') as f: >> + for line in f: >> + if w in line: >> + subprocess.call('sudo umount -f ' + >> line.split()[1], stdout=devnull, stderr=devnull, shell=True) >> devnull.close() >> } >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb >> b/meta/recipes-devtools/buildchroot/buildchroot.bb index >> 304c67e..df9df19 100644 --- >> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ >> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -12,7 +12,6 @@ >> FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" >> SRC_URI = "file://multistrap.conf.in \ file://configscript.sh \ >> file://setup.sh \ >> - file://download_dev-random \ >> file://build.sh" >> PV = "1.0" >> >> @@ -32,8 +31,10 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ >> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >> >> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >> -do_build[dirs] = "${WORKDIR}/hooks_multistrap \ >> - ${BUILDCHROOT_DIR}/isar-apt" >> +do_build[dirs] = "${BUILDCHROOT_DIR}/isar-apt \ >> + ${BUILDCHROOT_DIR}/dev \ >> + ${BUILDCHROOT_DIR}/proc \ >> + ${BUILDCHROOT_DIR}/sys" >> do_build[depends] = "isar-apt:do_cache_config" >> >> do_build() { >> @@ -41,7 +42,6 @@ do_build() { >> >> chmod +x "${WORKDIR}/setup.sh" >> chmod +x "${WORKDIR}/configscript.sh" >> - install -m 755 "${WORKDIR}/download_dev-random" >> "${WORKDIR}/hooks_multistrap/" >> # Multistrap accepts only relative path in configuration files, >> so get it: cd ${TOPDIR} >> @@ -60,15 +60,6 @@ do_build() { >> -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ >> "${WORKDIR}/multistrap.conf.in" > >> "${WORKDIR}/multistrap.conf" >> - [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 >> ${BUILDCHROOT_DIR}/proc >> - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc >> - _do_build_cleanup() { >> - ret=$? >> - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true >> - (exit $ret) || bb_exit_handler >> - } >> - trap '_do_build_cleanup' EXIT >> - >> do_setup_mounts >> >> # Create root filesystem >> @@ -79,7 +70,6 @@ do_build() { >> >> # Configure root filesystem >> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh >> - _do_build_cleanup >> >> do_cleanup_mounts >> } >> @@ -96,10 +86,16 @@ do_setup_mounts[stamp-extra-info] = >> "${DISTRO}-${DISTRO_ARCH}" >> do_setup_mounts() { >> sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} >> ${BUILDCHROOT_DIR}/isar-apt >> + sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev >> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc >> + sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys >> } >> >> addtask setup_mounts after do_build >> >> do_cleanup_mounts() { >> sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true >> + sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true >> + sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true >> + sudo umount ${BUILDCHROOT_DIR}/sys 2>/dev/null || true >> } >> diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh >> b/meta/recipes-devtools/buildchroot/files/configscript.sh index >> 9813c9a..524e50c 100644 --- >> a/meta/recipes-devtools/buildchroot/files/configscript.sh +++ >> b/meta/recipes-devtools/buildchroot/files/configscript.sh @@ -39,10 >> +39,6 @@ export LC_ALL=C LANGUAGE=C LANG=C #run pre installation >> script /var/lib/dpkg/info/dash.preinst install >> >> -# apt-get http method, gpg require /dev/null >> -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev >> - >> #configuring packages >> dpkg --configure -a >> apt-get update >> -umount /dev >> diff --git >> a/meta/recipes-devtools/buildchroot/files/download_dev-random >> b/meta/recipes-devtools/buildchroot/files/download_dev-random deleted >> file mode 100644 index 5b5b96b..0000000 --- >> a/meta/recipes-devtools/buildchroot/files/download_dev-random >> +++ /dev/null @@ -1,13 +0,0 @@ >> -#!/bin/sh >> - >> -set -e >> - >> -readonly ROOTFS="$1" >> - >> -mknod "${ROOTFS}/dev/random" c 1 8 >> -chmod 640 "${ROOTFS}/dev/random" >> -chown 0:0 "${ROOTFS}/dev/random" >> - >> -mknod "${ROOTFS}/dev/urandom" c 1 9 >> -chmod 640 "${ROOTFS}/dev/urandom" >> -chown 0:0 "${ROOTFS}/dev/urandom" >