[RFC] using lightweight containers instead of chroot

[Cedric Hombourger <cedric_hombourger@mentor.com>]

Hello,

This is a RFC for us to replace uses of chroot within the Isar classes 
to use Linux namespaces.

Motivation

    We will be submitting another RFC for per-recipe buildchroots but we
    should note that it would likely
    increase the number of active bind mounts during our builds. So
    before we discuss that, we may want to
    discuss how we run our build scripts today using chroots.

    There are numerous challenges with our current implementation and
    attempts to , Examples would include:

        1e31d5a events: Warn if mounted paths left
        d21d495 dpkg: Make mount buildroot reliable
        22c42de dpkg-base: Warn about unmounting problems
        ...

    And you have found yourself in the situation where Isar wiped out
    /dev or your layers (at least I did)

    chroot is a powerful tool but it is starting to show its age and we
    see how much burden is on us to setup and
    shutdown our chroot environment

Proposal

    We may want to use unshare(1) to create a mount namespace where we
    will create our bind mounts,
    chroot into the buildchroot and run the specified command/script

    The immediate benefit of this approach is that all mounts
    automatically disappear as the supplied
    command exits (whether it aborts prematurely because of an error or
    normally on completion).

    Another nice benefit is that bind mounts we created within this
    namespace are not (directly) visible
    from the parent namespace

    However, we found that running scripts within an unshare environment
    may not be as easy as
    chroot. We would welcome feedback on the code snippets provided
    below if you happen to have
    some better ideas.

    def isar_user_spec():
         import os
         return '%d:%d' % (os.getuid(), os.getgid())

    ISAR_USER_SPEC    = "${@ isar_user_spec() }"
    ISAR_UNSHARE_CMD  = "sudo unshare --pid --fork --ipc --mount sh -ex"
    ISAR_CHROOT_SHELL = "sh -ex"
    ISAR_CHROOT_ROOT  = "chroot ${BUILDCHROOT_DIR} ${ISAR_CHROOT_SHELL}"
    ISAR_CHROOT_USER  = "chroot --userspec='${ISAR_USER_SPEC}'
    ${BUILDCHROOT_DIR} ${ISAR_CHROOT_SHELL}"

    # Would be similar to buildchroot_do_mounts but will happen in a
    separate mount namespace
    BUILDCHROOT_DO_MOUNTS =
    "                                                         \
         mount --bind '${REPO_ISAR_DIR}/${DISTRO}'
    '${BUILDCHROOT_DIR}/isar-apt'     ; \
         mount --bind '${DL_DIR}'
    '${BUILDCHROOT_DIR}/downloads'                     ; \
         mount -t proc none
    '${BUILDCHROOT_DIR}/proc'                                ; \
         mount --rbind /sys
    '${BUILDCHROOT_DIR}/sys'                                 ; \
         mount -t tmpfs -o rw,nosuid,nodev,seclabel none
    ${BUILDCHROOT_DIR}/dev/shm  ; \
         mount -o bind /dev/pts
    ${BUILDCHROOT_DIR}/dev/pts                             \
    "

    # Would be similar to dpkg_do_mounts but will happen in a separate
    mount namespace
    DPKG_DO_MOUNTS = "                         \
         ${BUILDCHROOT_DO_MOUNTS}             ; \
         mkdir -p ${BUILDROOT}                ; \
         mount --bind ${WORKDIR} ${BUILDROOT}   \
    "

    # Build package from sources using build script
    _runbuild() {
         export arch=${1}

         E="${@ isar_export_proxies(d)}"
         (   cat <<"        UNSHARE"
                 ${DPKG_DO_MOUNTS}
                 (   cat <<"                SCRIPT"
                         export DEB_BUILD_OPTIONS="${DEB_BUILD_OPTIONS}"
                         export DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES}"
                         export PARALLEL_MAKE="${PARALLEL_MAKE}"
                         /isar/build.sh ${PP}/${PPS} ${arch}
                     SCRIPT
                 ) | ${ISAR_CHROOT_USER}
             UNSHARE
         ) | envsubst '$arch' | ${ISAR_UNSHARE_CMD}
    }

    dpkg_runbuild() {
         ( _runbuild ${PACKAGE_ARCH} )
    }

    PS: I am not very happy with the need to feed the script to execute
    under unshare
         via stdin, if there are better ways, we would be happy to
    consider them!

Proposed Next steps

    1. Collect feedback and answer questions
        a. is the use of unshare a good idea? (no is an OK answer!)
        b. can we come up with a better code construct?

    2. Check compatibility with containerized-builds (e.g. from within a
    kas build)

    3. Check for better ways to spawn scripts

    4. Implement and submit RFC patches

Future RFCs

    - per-recipe buildchroots (mimic sbuild)

Cedric