Re: [PATCH 2/2] buildchroot: Permanently mount '/dev', '/sys' and '/proc'

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/05/2018 08:50 PM, Jan Kiszka wrote:
> On 2018-02-05 18:32, Jan Kiszka wrote:
>> On 2018-02-05 18:30, [ext] Jan Kiszka wrote:
>>> On 2018-02-05 18:16, Alexander Smirnov wrote:
>>>> On 02/05/2018 07:53 PM, Jan Kiszka wrote:
>>>>> On 2018-02-05 17:43, Alexander Smirnov wrote:
>>>>>> Mount these folders permanently before creating buildchroot. Mounts are
>>>>>> performed in dedicated task do_prepare, so it will also work if you run
>>>>>> build with already existing buildchroot.
>>>>>>
>>>>>> BuildCompleted event is used for clean up, so we should not take care
>>>>>> about this manually.
>>>>>>
>>>>>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>>>>>> ---
>>>>>>    meta/classes/isar-events.bbclass                    |  3 +++
>>>>>>    meta/recipes-devtools/buildchroot/buildchroot.bb    | 21
>>>>>> +++++++--------------
>>>>>>    .../buildchroot/files/configscript.sh               |  4 ----
>>>>>>    .../buildchroot/files/download_dev-random           | 13 -------------
>>>>>>    4 files changed, 10 insertions(+), 31 deletions(-)
>>>>>>    delete mode 100644
>>>>>> meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>>
>>>>>> diff --git a/meta/classes/isar-events.bbclass
>>>>>> b/meta/classes/isar-events.bbclass
>>>>>> index 55fc106..9f581a6 100644
>>>>>> --- a/meta/classes/isar-events.bbclass
>>>>>> +++ b/meta/classes/isar-events.bbclass
>>>>>> @@ -15,6 +15,9 @@ python isar_handler () {
>>>>>>              # Clean up buildchroot
>>>>>>            subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>> '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>> '/dev || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>> '/proc || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>> '/sys || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>>          devnull.close()
>>>>>>    }
>>>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>> b/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>> index 0044a71..b8aaf28 100644
>>>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>> @@ -12,7 +12,6 @@ FILESPATH =.
>>>>>> "${LAYERDIR_core}/recipes-devtools/buildchroot/files:"
>>>>>>    SRC_URI = "file://multistrap.conf.in \
>>>>>>               file://configscript.sh \
>>>>>>               file://setup.sh \
>>>>>> -           file://download_dev-random \
>>>>>>               file://build.sh"
>>>>>>    PV = "1.0"
>>>>>>    @@ -32,16 +31,21 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
>>>>>>    WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>>>>>      do_prepare[nostamp] = "1"
>>>>>> -do_prepare[dirs] = "${BUILDCHROOT_DIR}/isar-apt"
>>>>>> +do_prepare[dirs] = "${BUILDCHROOT_DIR}/isar-apt \
>>>>>> +                    ${BUILDCHROOT_DIR}/dev \
>>>>>> +                    ${BUILDCHROOT_DIR}/proc \
>>>>>> +                    ${BUILDCHROOT_DIR}/sys"
>>>>>>      do_prepare() {
>>>>>>        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO}
>>>>>> ${BUILDCHROOT_DIR}/isar-apt
>>>>>> +    sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev
>>>>>> +    sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>>>>>> +    sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys
>>>>>>    }
>>>>>>      addtask prepare before do_build after do_unpack
>>>>>>      do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>>>>> -do_build[dirs] = "${WORKDIR}/hooks_multistrap"
>>>>>>    do_build[depends] = "isar-apt:do_cache_config"
>>>>>>      do_build() {
>>>>>> @@ -49,7 +53,6 @@ do_build() {
>>>>>>          chmod +x "${WORKDIR}/setup.sh"
>>>>>>        chmod +x "${WORKDIR}/configscript.sh"
>>>>>> -    install -m 755 "${WORKDIR}/download_dev-random"
>>>>>> "${WORKDIR}/hooks_multistrap/"
>>>>>>          # Multistrap accepts only relative path in configuration
>>>>>> files, so get it:
>>>>>>        cd ${TOPDIR}
>>>>>> @@ -68,15 +71,6 @@ do_build() {
>>>>>>            -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
>>>>>>               "${WORKDIR}/multistrap.conf.in" >
>>>>>> "${WORKDIR}/multistrap.conf"
>>>>>>    -    [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555
>>>>>> ${BUILDCHROOT_DIR}/proc
>>>>>> -    sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>>>>>> -    _do_build_cleanup() {
>>>>>> -        ret=$?
>>>>>> -        sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true
>>>>>> -        (exit $ret) || bb_exit_handler
>>>>>> -    }
>>>>>> -    trap '_do_build_cleanup' EXIT
>>>>>> -
>>>>>>        # Create root filesystem
>>>>>>        sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f
>>>>>> "${WORKDIR}/multistrap.conf"
>>>>>>    @@ -85,5 +79,4 @@ do_build() {
>>>>>>          # Configure root filesystem
>>>>>>        sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
>>>>>> -    _do_build_cleanup
>>>>>>    }
>>>>>> diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>> b/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>> index 458c94b..a5942b5 100644
>>>>>> --- a/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>> +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>> @@ -39,9 +39,5 @@ export LC_ALL=C LANGUAGE=C LANG=C
>>>>>>    #run pre installation script
>>>>>>    /var/lib/dpkg/info/dash.preinst install
>>>>>>    -# apt-get http method, gpg require /dev/null
>>>>>> -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev
>>>>>> -
>>>>>>    #configuring packages
>>>>>>    dpkg --configure -a
>>>>>> -umount /dev
>>>>>> diff --git
>>>>>> a/meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>> b/meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>> deleted file mode 100644
>>>>>> index 5b5b96b..0000000
>>>>>> --- a/meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>> +++ /dev/null
>>>>>> @@ -1,13 +0,0 @@
>>>>>> -#!/bin/sh
>>>>>> -
>>>>>> -set -e
>>>>>> -
>>>>>> -readonly ROOTFS="$1"
>>>>>> -
>>>>>> -mknod "${ROOTFS}/dev/random" c 1 8
>>>>>> -chmod 640 "${ROOTFS}/dev/random"
>>>>>> -chown 0:0 "${ROOTFS}/dev/random"
>>>>>> -
>>>>>> -mknod "${ROOTFS}/dev/urandom" c 1 9
>>>>>> -chmod 640 "${ROOTFS}/dev/urandom"
>>>>>> -chown 0:0 "${ROOTFS}/dev/urandom"
>>>>>>
>>>>>
>>>>> That should finally obsolete my /dev/null fix. But after the problem
>>>>> with my bind-mount attempt for /dev, I'd like to validate. What is the
>>>>> baseline this patch should go upon?
>>>>>
>>>>
>>>> Rebased patch to the latest 'next', please try 'asmirnov/devel' branch.
>>>> Also it would be helpful if you could test double ^C and broken build.
>>>>
>>>> It would be perfect if it works :-)
>>>>
>>>> NOTE: umounts are done within next 2-3 seconds after your build stopped
>>>> It seems this time is needed to stop bitbake server context.
>>>
>>> Will run a couple of tests (I'm particularly good generating broken one)
>>> and let you know. On first glance, it works as designed. Even ^C^C
>>> followed by an immediate restart work fine (the second run waits for the
>>> first to finish and does not complain about any dangling mounts).
>>>
>>
>> OK, something that was not yet addressed: Unmounting of the workdir from
>> the buildchroot on ^C^C. I suppose that is doable in an analogous way,
>> right?

Yes, I've already thought about this, implement something like this:

for a in $BUILDCHROOT_DIR/home/builder
     umount xxx

in isar-events.bbclass.

>>
> 
> We also need this for the isar-image build (I'm left with /proc still
> mounted when interrupt that task).

For now it's only about buildchroot.

> 
> And when I restart the build that I interrupt after buildchroot was long
> finished, that task is still rebuild - and that fails, maybe because of
> preexisting artifacts.

That's the known problem, we have no do_clean task implemented. Re-run 
on existing tree is not officially supported for now.

So, I'm right that there are no mounted /dev, /proc and /sys in your 
buildchroot after termination?

Alex