From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519532471426482176 X-Received: by 10.80.144.177 with SMTP id c46mr610143eda.8.1518170180842; Fri, 09 Feb 2018 01:56:20 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.185.67 with SMTP id m61ls3320283ede.4.gmail; Fri, 09 Feb 2018 01:56:20 -0800 (PST) X-Google-Smtp-Source: AH8x227qwemYWgO/jX1VtkriB9jx93gEVQQy5Qcsb6qloa9PX/k2bHDT2Dp1HhMz46U2JEArnY9W X-Received: by 10.80.171.15 with SMTP id s15mr610411edc.6.1518170180221; Fri, 09 Feb 2018 01:56:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518170180; cv=none; d=google.com; s=arc-20160816; b=OQFpZNLQBUK2A1a1dGZiv0Nk2WF2UaZyZqq61Pa5W6qUTyPCbuPcLxlLxZd1SwV5sf ams6Gs4lbTZDaPBB3cQDkmo11xRzixdtqtPgi/BnJMGrmFwnfz3vgLqC7sIXUO0I0kJ5 Vb83Q6HhoRcBAsLjNBnH1/a7y3YOMCdHpayNS3aEgTiBjxRIDL+6W63dCSPBJVuMLx/b M48wI1jJ5eSeKckIaEXaNdBFtG8h9l6snjH8mRgJIatkG/Md5cWbEPIQ9aiUcASq91RI IeKHEc1cQPUEfQbWiSJbi+0KQVsM33QBmrehLRVRqV/EgoX7hs7s9hJQ/3Oe5rEAxPei TbiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=RyOjBIBYd+lSmPAcbhs1AFYqA8Zz8L6Uwhb/dZzOd64=; b=hfcOv6z8ceU3dSulNz13JM92V5ghKGDnj9vCkyBjkGQ0Ja0IwIDlUWzQVR3aJ7qXbM B0nUp9TDKudBMSiSrEUciqagcz/wPakJ48RuXQeBSgIqO9AQcZ/N8IqS1nM9JcwAzr2b lAlQm9P8PySyScSaO4knEG/b7SjBMMVelVaPbtbQzQeaUu7/9yKYzjIEiXb09lruXfAn 69gbI/fRg534WvLuAZIfpyxDf09RCSGZ2Ma2fXvFapdFhyXP0bxXtFCL5HrztcJlPT4+ DFWGsuqpU+KIT2pA2/X4NmTQo0OauPUuo3GBUGL/Qt4Ax4eM+dIS/dksvXJlW1mHkcAZ tZ3A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id z17si144591edz.1.2018.02.09.01.56.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Feb 2018 01:56:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] (client.yota.ru [94.25.228.179] (may be forged)) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w199uG9Q012404 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Fri, 9 Feb 2018 10:56:18 +0100 Subject: Re: [PATCH] isar: Clean mount point on bitbake exit To: isar-users References: <20180206195516.32153-1-asmirnov@ilbers.de> From: Alexander Smirnov Message-ID: Date: Fri, 9 Feb 2018 12:56:11 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20180206195516.32153-1-asmirnov@ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: Qb36uuR3WZE0 On 02/06/2018 10:55 PM, Alexander Smirnov wrote: > 8<-- > > That's it! Branch 'asmirnov/devel', please test and enjoy :-) > > 8<-- > > Now each multiconfig has registered handler for BuildCompleted event (see > class 'isar-event.bbclass'). Moreover, the '/proc/mounts' file contains > all the active mounts. In addition, from event handler we could derive > all the variables like ${TMPDIR}, ${DISTRO} etc. So it's possible to > find all the active mounts for current multiconfig and clean them. > > NOTE: if build is interrupted by double ^C, some mount points could stay > uncleaned. This is caused by remaining processes started by bitbake, for > example: > - 'chroot build.sh ...' > - 'multistrap ...' > > So please be careful when interrupting build. > > Signed-off-by: Alexander Smirnov > --- > meta-isar/recipes-core/images/isar-image-base.bb | 11 ++++------ > meta/classes/dpkg-base.bbclass | 12 ++++------- > meta/classes/isar-events.bbclass | 15 +++++++++++--- > meta/recipes-devtools/buildchroot/buildchroot.bb | 24 +++++++++------------- > .../buildchroot/files/configscript.sh | 4 ---- > .../buildchroot/files/download_dev-random | 13 ------------ > 6 files changed, 30 insertions(+), 49 deletions(-) > delete mode 100644 meta/recipes-devtools/buildchroot/files/download_dev-random > > diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb > index e359ac3..8ddbabb 100644 > --- a/meta-isar/recipes-core/images/isar-image-base.bb > +++ b/meta-isar/recipes-core/images/isar-image-base.bb > @@ -55,14 +55,10 @@ do_rootfs() { > -e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \ > "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" > > + # Do not use bitbake flag [dirs] here because this folder should have > + # specific ownership. > [ ! -d ${IMAGE_ROOTFS}/proc ] && sudo install -d -o 0 -g 0 -m 555 ${IMAGE_ROOTFS}/proc > sudo mount -t proc none ${IMAGE_ROOTFS}/proc > - _do_rootfs_cleanup() { > - ret=$? > - sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true > - (exit $ret) || bb_exit_handler > - } > - trap '_do_rootfs_cleanup' EXIT > > # Create root filesystem. We must use sudo -E here to preserve the environment > # because of proxy settings > @@ -72,5 +68,6 @@ do_rootfs() { > sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} ${MACHINE_SERIAL} ${BAUDRATE_TTY} \ > ${ROOTFS_DEV} > sudo rm "${IMAGE_ROOTFS}/${DISTRO_CONFIG_SCRIPT}" > - _do_rootfs_cleanup > + > + sudo umount ${IMAGE_ROOTFS}/proc 2>/dev/null || true > } > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index 5d5a924..a34c21f 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -20,15 +20,11 @@ dpkg_runbuild() { > do_build() { > mkdir -p ${BUILDROOT} > sudo mount --bind ${WORKDIR} ${BUILDROOT} > - _do_build_cleanup() { > - ret=$? > - sudo umount ${BUILDROOT} 2>/dev/null || true > - sudo rmdir ${BUILDROOT} 2>/dev/null || true > - (exit $ret) || bb_exit_handler > - } > - trap '_do_build_cleanup' EXIT > + > dpkg_runbuild > - _do_build_cleanup > + > + sudo umount ${BUILDROOT} 2>/dev/null || true > + sudo rmdir ${BUILDROOT} 2>/dev/null || true > } > > # Install package to Isar-apt > diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass > index 55fc106..ae0f791 100644 > --- a/meta/classes/isar-events.bbclass > +++ b/meta/classes/isar-events.bbclass > @@ -11,10 +11,19 @@ python isar_handler () { > devnull = open(os.devnull, 'w') > > if isinstance(e, bb.event.BuildCompleted): > - bchroot = d.getVar('BUILDCHROOT_DIR', True) > + tmpdir = d.getVar('TMPDIR', True) > + distro = d.getVar('DISTRO', True) > + arch = d.getVar('DISTRO_ARCH', True) > > - # Clean up buildchroot > - subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) > + w = tmpdir + '/work/' + distro + '-' + arch > + > + # '/proc/mounts' contains all the active mounts, so knowing 'w' we > + # could get the list of mounts for the specific multiconfig and > + # clean them. > + with open('/proc/mounts', 'rU') as f: > + for line in f: > + if w in line: > + subprocess.call('sudo umount -f ' + line.split()[1], stdout=devnull, stderr=devnull, shell=True) > > devnull.close() > } > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb > index 304c67e..df9df19 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb > @@ -12,7 +12,6 @@ FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" > SRC_URI = "file://multistrap.conf.in \ > file://configscript.sh \ > file://setup.sh \ > - file://download_dev-random \ > file://build.sh" > PV = "1.0" > > @@ -32,8 +31,10 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > -do_build[dirs] = "${WORKDIR}/hooks_multistrap \ > - ${BUILDCHROOT_DIR}/isar-apt" > +do_build[dirs] = "${BUILDCHROOT_DIR}/isar-apt \ > + ${BUILDCHROOT_DIR}/dev \ > + ${BUILDCHROOT_DIR}/proc \ > + ${BUILDCHROOT_DIR}/sys" > do_build[depends] = "isar-apt:do_cache_config" > > do_build() { > @@ -41,7 +42,6 @@ do_build() { > > chmod +x "${WORKDIR}/setup.sh" > chmod +x "${WORKDIR}/configscript.sh" > - install -m 755 "${WORKDIR}/download_dev-random" "${WORKDIR}/hooks_multistrap/" > > # Multistrap accepts only relative path in configuration files, so get it: > cd ${TOPDIR} > @@ -60,15 +60,6 @@ do_build() { > -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ > "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" > > - [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 ${BUILDCHROOT_DIR}/proc > - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > - _do_build_cleanup() { > - ret=$? > - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > - (exit $ret) || bb_exit_handler > - } > - trap '_do_build_cleanup' EXIT > - > do_setup_mounts > > # Create root filesystem > @@ -79,7 +70,6 @@ do_build() { > > # Configure root filesystem > sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > - _do_build_cleanup > > do_cleanup_mounts > } > @@ -96,10 +86,16 @@ do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > do_setup_mounts() { > sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt > + sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev Replaced this line by: (like it was before in configurescript.sh): 8<-- sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev 8<-- Otherwise debian-wheezy-arm:buildchroot fails on CI server with strange error (while it works on local machine): 8<-- Setting up initscripts (2.88dsf-41+deb7u1) ... /bin/rm: cannot remove `/dev/shm': Device or resource busy dpkg: error processing initscripts (--configure): 8<-- After brief debugging of initscripts/postinst, seems it incorrectly identifies guest environment in chroot and tries to create new /dev/shm entry. Alex > + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > + sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys > } > > addtask setup_mounts after do_build > > do_cleanup_mounts() { > sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true > + sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true > + sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > + sudo umount ${BUILDCHROOT_DIR}/sys 2>/dev/null || true > } > diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh > index 9813c9a..524e50c 100644 > --- a/meta/recipes-devtools/buildchroot/files/configscript.sh > +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh > @@ -39,10 +39,6 @@ export LC_ALL=C LANGUAGE=C LANG=C > #run pre installation script > /var/lib/dpkg/info/dash.preinst install > > -# apt-get http method, gpg require /dev/null > -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev > - > #configuring packages > dpkg --configure -a > apt-get update > -umount /dev > diff --git a/meta/recipes-devtools/buildchroot/files/download_dev-random b/meta/recipes-devtools/buildchroot/files/download_dev-random > deleted file mode 100644 > index 5b5b96b..0000000 > --- a/meta/recipes-devtools/buildchroot/files/download_dev-random > +++ /dev/null > @@ -1,13 +0,0 @@ > -#!/bin/sh > - > -set -e > - > -readonly ROOTFS="$1" > - > -mknod "${ROOTFS}/dev/random" c 1 8 > -chmod 640 "${ROOTFS}/dev/random" > -chown 0:0 "${ROOTFS}/dev/random" > - > -mknod "${ROOTFS}/dev/urandom" c 1 9 > -chmod 640 "${ROOTFS}/dev/urandom" > -chown 0:0 "${ROOTFS}/dev/urandom" > -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov