From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519112020741062656 X-Received: by 10.223.196.13 with SMTP id v13mr2076011wrf.22.1517853843378; Mon, 05 Feb 2018 10:04:03 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.47.194 with SMTP id v185ls1678655wmv.4.canary-gmail; Mon, 05 Feb 2018 10:04:02 -0800 (PST) X-Google-Smtp-Source: AH8x227wRaoW7fY6Wks2mawixURVaNQQbYrxvaIec8WCeiFK2cd9NrMMOdqKnY27lr8dN2F7G1Gy X-Received: by 10.28.228.138 with SMTP id b132mr10117wmh.19.1517853842844; Mon, 05 Feb 2018 10:04:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517853842; cv=none; d=google.com; s=arc-20160816; b=S2/JVWzy17qZaKmQT/MDgFerRjKaD/2TIRbG6ukr1tQLgEJxtbaC4LEXWR58AlabwY kZ8+h3XW/GgV1XKbKAY4A+ludBMjhXqv93b1dJCLSQFWvXIgMno56f7gG7BLA9rhd+Kx ep0exfAqkQDdBIJ/bLipR9QqXmOIfrRF41FnahxBiXbahGMJ/4sCASOsXGPwc/UE25/+ C3cf+W9lYTEAD0Vrnb1UuP0remsWdEP1ntZlEV6NXRtr2XB0/nevh2LrycI8jZevfq6k 4uXj2qLuI/6/V44OAUAqBuvAEVW/dvBzkIfPh6DSYGLra64Vmh2lly5s//U16LNbEHKp k8AA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=7zHFGmgyrBo8HwOdoJ/VqXpNnV8TFq5rYfD13O0PpvQ=; b=sKIQFAwb3ryxsSfSl7P97Jb5k/3/KjYpsFqApdnI0JDy2V+Lgb0nj+qk0VsJQM0GwZ 8jZxPNiGPzvCDh/VpXHiYOcso8muxJzt8lp1OEA8hPkw8JPluVjp57UgtlqkFglb5wRO s9R/gcB3AWeGaMHE3Kkk4NY/uymD0bq6OaQ2JfZ55RJ1tzteP8G9AjIgIPUg3w3h/67j AkiM11Ez8g0UqD9L1/PIUrIzJFKg2GEb02via6QDkeBszOi+utHKA6zExscH1VVkNN+z WBYBWQ2XvoymkfeuemqkP4gzpP79KwpPn/K3e5uZtYhCXqzTvzdI3FfSxqiY75FjTB1l zrcw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id k21si559635wrd.1.2018.02.05.10.04.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Feb 2018 10:04:02 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w15I3xvj025709 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 5 Feb 2018 19:04:01 +0100 Subject: Re: [PATCH 2/2] buildchroot: Permanently mount '/dev', '/sys' and '/proc' To: Jan Kiszka , isar-users@googlegroups.com References: <20180205164347.28375-1-asmirnov@ilbers.de> <20180205164347.28375-3-asmirnov@ilbers.de> <604637ca-63f4-524a-0484-34f38f706919@siemens.com> From: Alexander Smirnov Message-ID: Date: Mon, 5 Feb 2018 21:03:54 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: pzpggKFHv86Z On 02/05/2018 08:50 PM, Jan Kiszka wrote: > On 2018-02-05 18:32, Jan Kiszka wrote: >> On 2018-02-05 18:30, [ext] Jan Kiszka wrote: >>> On 2018-02-05 18:16, Alexander Smirnov wrote: >>>> On 02/05/2018 07:53 PM, Jan Kiszka wrote: >>>>> On 2018-02-05 17:43, Alexander Smirnov wrote: >>>>>> Mount these folders permanently before creating buildchroot. Mounts are >>>>>> performed in dedicated task do_prepare, so it will also work if you run >>>>>> build with already existing buildchroot. >>>>>> >>>>>> BuildCompleted event is used for clean up, so we should not take care >>>>>> about this manually. >>>>>> >>>>>> Signed-off-by: Alexander Smirnov >>>>>> --- >>>>>>   meta/classes/isar-events.bbclass                    |  3 +++ >>>>>>   meta/recipes-devtools/buildchroot/buildchroot.bb    | 21 >>>>>> +++++++-------------- >>>>>>   .../buildchroot/files/configscript.sh               |  4 ---- >>>>>>   .../buildchroot/files/download_dev-random           | 13 ------------- >>>>>>   4 files changed, 10 insertions(+), 31 deletions(-) >>>>>>   delete mode 100644 >>>>>> meta/recipes-devtools/buildchroot/files/download_dev-random >>>>>> >>>>>> diff --git a/meta/classes/isar-events.bbclass >>>>>> b/meta/classes/isar-events.bbclass >>>>>> index 55fc106..9f581a6 100644 >>>>>> --- a/meta/classes/isar-events.bbclass >>>>>> +++ b/meta/classes/isar-events.bbclass >>>>>> @@ -15,6 +15,9 @@ python isar_handler () { >>>>>>             # Clean up buildchroot >>>>>>           subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + >>>>>> '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) >>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + >>>>>> '/dev || /bin/true', stdout=devnull, stderr=devnull, shell=True) >>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + >>>>>> '/proc || /bin/true', stdout=devnull, stderr=devnull, shell=True) >>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + >>>>>> '/sys || /bin/true', stdout=devnull, stderr=devnull, shell=True) >>>>>>         devnull.close() >>>>>>   } >>>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb >>>>>> b/meta/recipes-devtools/buildchroot/buildchroot.bb >>>>>> index 0044a71..b8aaf28 100644 >>>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb >>>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb >>>>>> @@ -12,7 +12,6 @@ FILESPATH =. >>>>>> "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" >>>>>>   SRC_URI = "file://multistrap.conf.in \ >>>>>>              file://configscript.sh \ >>>>>>              file://setup.sh \ >>>>>> -           file://download_dev-random \ >>>>>>              file://build.sh" >>>>>>   PV = "1.0" >>>>>>   @@ -32,16 +31,21 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ >>>>>>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >>>>>>     do_prepare[nostamp] = "1" >>>>>> -do_prepare[dirs] = "${BUILDCHROOT_DIR}/isar-apt" >>>>>> +do_prepare[dirs] = "${BUILDCHROOT_DIR}/isar-apt \ >>>>>> +                    ${BUILDCHROOT_DIR}/dev \ >>>>>> +                    ${BUILDCHROOT_DIR}/proc \ >>>>>> +                    ${BUILDCHROOT_DIR}/sys" >>>>>>     do_prepare() { >>>>>>       sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} >>>>>> ${BUILDCHROOT_DIR}/isar-apt >>>>>> +    sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev >>>>>> +    sudo mount -t proc none ${BUILDCHROOT_DIR}/proc >>>>>> +    sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys >>>>>>   } >>>>>>     addtask prepare before do_build after do_unpack >>>>>>     do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>>>>> -do_build[dirs] = "${WORKDIR}/hooks_multistrap" >>>>>>   do_build[depends] = "isar-apt:do_cache_config" >>>>>>     do_build() { >>>>>> @@ -49,7 +53,6 @@ do_build() { >>>>>>         chmod +x "${WORKDIR}/setup.sh" >>>>>>       chmod +x "${WORKDIR}/configscript.sh" >>>>>> -    install -m 755 "${WORKDIR}/download_dev-random" >>>>>> "${WORKDIR}/hooks_multistrap/" >>>>>>         # Multistrap accepts only relative path in configuration >>>>>> files, so get it: >>>>>>       cd ${TOPDIR} >>>>>> @@ -68,15 +71,6 @@ do_build() { >>>>>>           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ >>>>>>              "${WORKDIR}/multistrap.conf.in" > >>>>>> "${WORKDIR}/multistrap.conf" >>>>>>   -    [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 >>>>>> ${BUILDCHROOT_DIR}/proc >>>>>> -    sudo mount -t proc none ${BUILDCHROOT_DIR}/proc >>>>>> -    _do_build_cleanup() { >>>>>> -        ret=$? >>>>>> -        sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true >>>>>> -        (exit $ret) || bb_exit_handler >>>>>> -    } >>>>>> -    trap '_do_build_cleanup' EXIT >>>>>> - >>>>>>       # Create root filesystem >>>>>>       sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f >>>>>> "${WORKDIR}/multistrap.conf" >>>>>>   @@ -85,5 +79,4 @@ do_build() { >>>>>>         # Configure root filesystem >>>>>>       sudo chroot ${BUILDCHROOT_DIR} /configscript.sh >>>>>> -    _do_build_cleanup >>>>>>   } >>>>>> diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh >>>>>> b/meta/recipes-devtools/buildchroot/files/configscript.sh >>>>>> index 458c94b..a5942b5 100644 >>>>>> --- a/meta/recipes-devtools/buildchroot/files/configscript.sh >>>>>> +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh >>>>>> @@ -39,9 +39,5 @@ export LC_ALL=C LANGUAGE=C LANG=C >>>>>>   #run pre installation script >>>>>>   /var/lib/dpkg/info/dash.preinst install >>>>>>   -# apt-get http method, gpg require /dev/null >>>>>> -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev >>>>>> - >>>>>>   #configuring packages >>>>>>   dpkg --configure -a >>>>>> -umount /dev >>>>>> diff --git >>>>>> a/meta/recipes-devtools/buildchroot/files/download_dev-random >>>>>> b/meta/recipes-devtools/buildchroot/files/download_dev-random >>>>>> deleted file mode 100644 >>>>>> index 5b5b96b..0000000 >>>>>> --- a/meta/recipes-devtools/buildchroot/files/download_dev-random >>>>>> +++ /dev/null >>>>>> @@ -1,13 +0,0 @@ >>>>>> -#!/bin/sh >>>>>> - >>>>>> -set -e >>>>>> - >>>>>> -readonly ROOTFS="$1" >>>>>> - >>>>>> -mknod "${ROOTFS}/dev/random" c 1 8 >>>>>> -chmod 640 "${ROOTFS}/dev/random" >>>>>> -chown 0:0 "${ROOTFS}/dev/random" >>>>>> - >>>>>> -mknod "${ROOTFS}/dev/urandom" c 1 9 >>>>>> -chmod 640 "${ROOTFS}/dev/urandom" >>>>>> -chown 0:0 "${ROOTFS}/dev/urandom" >>>>>> >>>>> >>>>> That should finally obsolete my /dev/null fix. But after the problem >>>>> with my bind-mount attempt for /dev, I'd like to validate. What is the >>>>> baseline this patch should go upon? >>>>> >>>> >>>> Rebased patch to the latest 'next', please try 'asmirnov/devel' branch. >>>> Also it would be helpful if you could test double ^C and broken build. >>>> >>>> It would be perfect if it works :-) >>>> >>>> NOTE: umounts are done within next 2-3 seconds after your build stopped >>>> It seems this time is needed to stop bitbake server context. >>> >>> Will run a couple of tests (I'm particularly good generating broken one) >>> and let you know. On first glance, it works as designed. Even ^C^C >>> followed by an immediate restart work fine (the second run waits for the >>> first to finish and does not complain about any dangling mounts). >>> >> >> OK, something that was not yet addressed: Unmounting of the workdir from >> the buildchroot on ^C^C. I suppose that is doable in an analogous way, >> right? Yes, I've already thought about this, implement something like this: for a in $BUILDCHROOT_DIR/home/builder umount xxx in isar-events.bbclass. >> > > We also need this for the isar-image build (I'm left with /proc still > mounted when interrupt that task). For now it's only about buildchroot. > > And when I restart the build that I interrupt after buildchroot was long > finished, that task is still rebuild - and that fails, maybe because of > preexisting artifacts. That's the known problem, we have no do_clean task implemented. Re-run on existing tree is not officially supported for now. So, I'm right that there are no mounted /dev, /proc and /sys in your buildchroot after termination? Alex