Re: [PATCH 2/2] buildchroot: Permanently mount '/dev', '/sys' and '/proc'

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-05 19:03, Alexander Smirnov wrote:
> 
> 
> On 02/05/2018 08:50 PM, Jan Kiszka wrote:
>> On 2018-02-05 18:32, Jan Kiszka wrote:
>>> On 2018-02-05 18:30, [ext] Jan Kiszka wrote:
>>>> On 2018-02-05 18:16, Alexander Smirnov wrote:
>>>>> On 02/05/2018 07:53 PM, Jan Kiszka wrote:
>>>>>> On 2018-02-05 17:43, Alexander Smirnov wrote:
>>>>>>> Mount these folders permanently before creating buildchroot.
>>>>>>> Mounts are
>>>>>>> performed in dedicated task do_prepare, so it will also work if
>>>>>>> you run
>>>>>>> build with already existing buildchroot.
>>>>>>>
>>>>>>> BuildCompleted event is used for clean up, so we should not take
>>>>>>> care
>>>>>>> about this manually.
>>>>>>>
>>>>>>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>>>>>>> ---
>>>>>>>    meta/classes/isar-events.bbclass                    |  3 +++
>>>>>>>    meta/recipes-devtools/buildchroot/buildchroot.bb    | 21
>>>>>>> +++++++--------------
>>>>>>>    .../buildchroot/files/configscript.sh               |  4 ----
>>>>>>>    .../buildchroot/files/download_dev-random           | 13
>>>>>>> -------------
>>>>>>>    4 files changed, 10 insertions(+), 31 deletions(-)
>>>>>>>    delete mode 100644
>>>>>>> meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>>>
>>>>>>> diff --git a/meta/classes/isar-events.bbclass
>>>>>>> b/meta/classes/isar-events.bbclass
>>>>>>> index 55fc106..9f581a6 100644
>>>>>>> --- a/meta/classes/isar-events.bbclass
>>>>>>> +++ b/meta/classes/isar-events.bbclass
>>>>>>> @@ -15,6 +15,9 @@ python isar_handler () {
>>>>>>>              # Clean up buildchroot
>>>>>>>            subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>>> '/isar-apt || /bin/true', stdout=devnull, stderr=devnull,
>>>>>>> shell=True)
>>>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>>> '/dev || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>>> '/proc || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>>>>>>> '/sys || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>>>>>>>          devnull.close()
>>>>>>>    }
>>>>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>>> b/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>>> index 0044a71..b8aaf28 100644
>>>>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
>>>>>>> @@ -12,7 +12,6 @@ FILESPATH =.
>>>>>>> "${LAYERDIR_core}/recipes-devtools/buildchroot/files:"
>>>>>>>    SRC_URI = "file://multistrap.conf.in \
>>>>>>>               file://configscript.sh \
>>>>>>>               file://setup.sh \
>>>>>>> -           file://download_dev-random \
>>>>>>>               file://build.sh"
>>>>>>>    PV = "1.0"
>>>>>>>    @@ -32,16 +31,21 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
>>>>>>>    WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>>>>>>      do_prepare[nostamp] = "1"
>>>>>>> -do_prepare[dirs] = "${BUILDCHROOT_DIR}/isar-apt"
>>>>>>> +do_prepare[dirs] = "${BUILDCHROOT_DIR}/isar-apt \
>>>>>>> +                    ${BUILDCHROOT_DIR}/dev \
>>>>>>> +                    ${BUILDCHROOT_DIR}/proc \
>>>>>>> +                    ${BUILDCHROOT_DIR}/sys"
>>>>>>>      do_prepare() {
>>>>>>>        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO}
>>>>>>> ${BUILDCHROOT_DIR}/isar-apt
>>>>>>> +    sudo mount --bind /dev ${BUILDCHROOT_DIR}/dev
>>>>>>> +    sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>>>>>>> +    sudo mount -t sysfs none ${BUILDCHROOT_DIR}/sys
>>>>>>>    }
>>>>>>>      addtask prepare before do_build after do_unpack
>>>>>>>      do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>>>>>> -do_build[dirs] = "${WORKDIR}/hooks_multistrap"
>>>>>>>    do_build[depends] = "isar-apt:do_cache_config"
>>>>>>>      do_build() {
>>>>>>> @@ -49,7 +53,6 @@ do_build() {
>>>>>>>          chmod +x "${WORKDIR}/setup.sh"
>>>>>>>        chmod +x "${WORKDIR}/configscript.sh"
>>>>>>> -    install -m 755 "${WORKDIR}/download_dev-random"
>>>>>>> "${WORKDIR}/hooks_multistrap/"
>>>>>>>          # Multistrap accepts only relative path in configuration
>>>>>>> files, so get it:
>>>>>>>        cd ${TOPDIR}
>>>>>>> @@ -68,15 +71,6 @@ do_build() {
>>>>>>>            -e
>>>>>>> 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
>>>>>>>               "${WORKDIR}/multistrap.conf.in" >
>>>>>>> "${WORKDIR}/multistrap.conf"
>>>>>>>    -    [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555
>>>>>>> ${BUILDCHROOT_DIR}/proc
>>>>>>> -    sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>>>>>>> -    _do_build_cleanup() {
>>>>>>> -        ret=$?
>>>>>>> -        sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true
>>>>>>> -        (exit $ret) || bb_exit_handler
>>>>>>> -    }
>>>>>>> -    trap '_do_build_cleanup' EXIT
>>>>>>> -
>>>>>>>        # Create root filesystem
>>>>>>>        sudo -E multistrap -a ${DISTRO_ARCH} -d
>>>>>>> "${BUILDCHROOT_DIR}" -f
>>>>>>> "${WORKDIR}/multistrap.conf"
>>>>>>>    @@ -85,5 +79,4 @@ do_build() {
>>>>>>>          # Configure root filesystem
>>>>>>>        sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
>>>>>>> -    _do_build_cleanup
>>>>>>>    }
>>>>>>> diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>>> b/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>>> index 458c94b..a5942b5 100644
>>>>>>> --- a/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>>> +++ b/meta/recipes-devtools/buildchroot/files/configscript.sh
>>>>>>> @@ -39,9 +39,5 @@ export LC_ALL=C LANGUAGE=C LANG=C
>>>>>>>    #run pre installation script
>>>>>>>    /var/lib/dpkg/info/dash.preinst install
>>>>>>>    -# apt-get http method, gpg require /dev/null
>>>>>>> -mount -t devtmpfs -o mode=0755,nosuid devtmpfs /dev
>>>>>>> -
>>>>>>>    #configuring packages
>>>>>>>    dpkg --configure -a
>>>>>>> -umount /dev
>>>>>>> diff --git
>>>>>>> a/meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>>> b/meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>>> deleted file mode 100644
>>>>>>> index 5b5b96b..0000000
>>>>>>> --- a/meta/recipes-devtools/buildchroot/files/download_dev-random
>>>>>>> +++ /dev/null
>>>>>>> @@ -1,13 +0,0 @@
>>>>>>> -#!/bin/sh
>>>>>>> -
>>>>>>> -set -e
>>>>>>> -
>>>>>>> -readonly ROOTFS="$1"
>>>>>>> -
>>>>>>> -mknod "${ROOTFS}/dev/random" c 1 8
>>>>>>> -chmod 640 "${ROOTFS}/dev/random"
>>>>>>> -chown 0:0 "${ROOTFS}/dev/random"
>>>>>>> -
>>>>>>> -mknod "${ROOTFS}/dev/urandom" c 1 9
>>>>>>> -chmod 640 "${ROOTFS}/dev/urandom"
>>>>>>> -chown 0:0 "${ROOTFS}/dev/urandom"
>>>>>>>
>>>>>>
>>>>>> That should finally obsolete my /dev/null fix. But after the problem
>>>>>> with my bind-mount attempt for /dev, I'd like to validate. What is
>>>>>> the
>>>>>> baseline this patch should go upon?
>>>>>>
>>>>>
>>>>> Rebased patch to the latest 'next', please try 'asmirnov/devel'
>>>>> branch.
>>>>> Also it would be helpful if you could test double ^C and broken build.
>>>>>
>>>>> It would be perfect if it works :-)
>>>>>
>>>>> NOTE: umounts are done within next 2-3 seconds after your build
>>>>> stopped
>>>>> It seems this time is needed to stop bitbake server context.
>>>>
>>>> Will run a couple of tests (I'm particularly good generating broken
>>>> one)
>>>> and let you know. On first glance, it works as designed. Even ^C^C
>>>> followed by an immediate restart work fine (the second run waits for
>>>> the
>>>> first to finish and does not complain about any dangling mounts).
>>>>
>>>
>>> OK, something that was not yet addressed: Unmounting of the workdir from
>>> the buildchroot on ^C^C. I suppose that is doable in an analogous way,
>>> right?
> 
> Yes, I've already thought about this, implement something like this:
> 
> for a in $BUILDCHROOT_DIR/home/builder
>     umount xxx
> 
> in isar-events.bbclass.
> 
>>>
>>
>> We also need this for the isar-image build (I'm left with /proc still
>> mounted when interrupt that task).
> 
> For now it's only about buildchroot.
> 
>>
>> And when I restart the build that I interrupt after buildchroot was long
>> finished, that task is still rebuild - and that fails, maybe because of
>> preexisting artifacts.
> 
> That's the known problem, we have no do_clean task implemented. Re-run
> on existing tree is not officially supported for now.

That should be fixed, or our turn-around times for Isar development will
explode!

> 
> So, I'm right that there are no mounted /dev, /proc and /sys in your
> buildchroot after termination?

Yes, that works as expected.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux