Re: PRoot Isar summary

[Benedikt Niedermayr <benbrenson89@googlemail.com>]

[-- Attachment #1: Type: text/plain, Size: 4187 bytes --]

Am 10.11.2017 um 20:42 schrieb Alexander Smirnov:
> Hi,
>
> On 11/10/2017 09:59 PM, Jan Kiszka wrote:
>> On 2017-11-09 10:57, Alexander Smirnov wrote:
>>> Hello everybody,
>>>
>>> I've tried to completely switch Isar to PRoot, so here are the problems
>>> I've faced with:
>>>
>>> 1. PRoot doesn't work with UID/GID, all the files in PRoot are owned by
>>> root. The command 'chown' doesn't have any effect.
>>>
>>> 2. Some system commands are failed in PRoot: passwd, chpasswd. I see
>>> message: System error, no other clues (but for Wheezy these commands 
>>> work).
>>>
>>> 3. mkfs.ext4 doesn't work under proot, lots of files are dropped in
>>> resulting image.
>>>
>>> So, summary:
>>> ============
>>>
>>> 1. PRoot could be an intermediate option for:
>>>   + Buildchroot creation.
>>>   + Packages building.
>>>   - Drawback: works slowly.
>>
>> Aren't issues 1 and 2 from above affecting these use cases as well?
>>
>
> For now I don't have any facts about problems with buildchroot, but my 
> test includes only 'hello' and 'example-raw' applications.
>
>  - Regarding UID/GID, what I've seen for now, these manipulations are 
> done in postinst scripts.
>  - Passwd/chpasswd commands are also used in postinst scripts (for 
> example initrd package), there is no need to have passwords in 
> buildchroot because we are working under root.
>
> So, roughly speaking, buildchroot is only needed to compile and pack 
> the binary package, what doesn't require multi-UID/GID and passwords 
> support.
>
> But for sure, it needs to build much more real packages to have more 
> precise statistics. :-(
>
> So I've created dedicated branch 'asmirnov/proot' for possible 
> experiments in future.
>
>>>
>>> 2. For image generation the other tool should be considered.
>>>
>>
>> What is plan B now? Plan C remains falling back to VM builds, I suppose.
>
> So there are 2 options remain for evaluation:
>  - fakeroot
>  - pseudo
>
> I'd like to evaluate these tools for the features, that are uncovered 
> now:
>
>  - rootfs with UID/GID support: in general PRoot is able to generate 
> multistrap rootfs with just *upacked* Debian packages, all the 
> problems occur when I try to run 'dpkg-configure -a' inside this rootfs.
>
>  - ext2fs image generation (AFAIK this already is supported by Yocto, 
> but unfortunately I don't know too much, I need to take a look first).
>
> From this evaluation I'd like to get two points:
>
> 1. Could we somehow implement quick PoC to drop 'sudo' for Isar. This 
> PoC could be based on several tools in parallel.
>
> 2. If the item above is possible - then choose one dedicated tool and 
> try to adapt it for our needs.
>
> Alex
>
Ok bad news, but I faced the same problems, when trying to use one of 
these tools. Each tool has own drawbacks and benefits, but no tool 
combines multiple drawbacks to fit our needs.

I don't want to bother you, but why can't we try to focus on running 
builds with docker support?

Create a wrapper around "bitbake" which first performs a docker 
container setup and then runs the bitbake build.

Using such wrapper can make the docker thing almost transparent.


I know there are some problems getting a docker container secure, but 
maybe a focus on trying to get a docker based isar build secure, is 
easier to reach than the our current approaches?

It is possible to drop some capabilities for docker in order to make it 
more secure (e.g. don't allow to create dev files).

A mount command is also not required since, new versions of mkfs have 
the "-d" option included (specifiy a directory, which copied into the 
filesystem image). So no sys_admin capabilities would be needed.

It is also possible to customize other things within the container to 
make it more secure:

- Add only required commands to sudoers file.

- Modify permissions to files/directories.

- Think about which commands within isar really need root privileges, 
and drop those.


I think, if somebody seriously wants to exploit the container, he will 
also reach that with a non-root based build.

The attachment contains a (very basic and rudimentary) approach of 
running docker based isar builds.


Benedikt








[-- Attachment #2: isar --]
[-- Type: text/plain, Size: 8130 bytes --]

#!/usr/bin/env python3
#
# Copyright (C) 2017 Mixed Mode GmbH
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
# Main ISAR executable. Bitbake itself cannot be executed, instead call this wrapper for
# running builds.
#

import argparse
import sys
import docker
import logging
import traceback
import os
from threading import Thread

# Remove capabilities from container increasing
# host system savety.
CAP_DROP = ['MKNOD']


def get_logger():
    logger = logging.getLogger('isar')
    formatter = logging.Formatter('%(asctime)s-%(name)s-%(levelname)s: %(message)s')

    logger.setLevel(logging.DEBUG)

    shandler = logging.StreamHandler()
    shandler.setLevel(logging.DEBUG)
    shandler.setFormatter(formatter)

    logger.addHandler(shandler)

    return logger

# Add function tracing
def addlog(func):
    def wrapper(*args, **kwargs):
        log.debug('Starting %s """%s"""' % (func.__name__, func.__doc__))
        func(*args, **kwargs)
        log.debug('Finished %s """%s"""' % (func.__name__, func.__doc__))
    return wrapper


log = get_logger()

def docker_log(stream_handler):
    """ Threaded handler for receiving current container and also image build logs.

        Print status lines in place instead of writing those logs into next newlines:

        {"status":"Extracting","progressDetail":{"current":40370176,"total":45129088},"progress":"[============================================\u003e      ] 40.37 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":40828928,"total":45129088},"progress":"[=============================================\u003e     ] 40.83 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":41287680,"total":45129088},"progress":"[=============================================\u003e     ] 41.29 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":41746432,"total":45129088},"progress":"[==============================================\u003e    ] 41.75 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":42205184,"total":45129088},"progress":"[==============================================\u003e    ] 42.21 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":42663936,"total":45129088},"progress":"[===============================================\u003e   ] 42.66 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":43122688,"total":45129088},"progress":"[===============================================\u003e   ] 43.12 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":43581440,"total":45129088},"progress":"[================================================\u003e  ] 43.58 MB/45.13 MB","id":"3e17c6eae66c"}
        {"status":"Extracting","progressDetail":{"current":44040192,"total":45129088},"progress":"[================================================\u003e  ] 44.04 MB/45.13 MB","id":"3e17c6eae66c"}

    """
    for i in stream_handler:

        if isinstance(i, dict):
            # Image build logs

            #
            # different keys possible (stream, status)
            #
            #i = i['stream']
            print(i.strip())
        else:
            # Container logs
            print(i.decode().strip())


class IsarDocker():

    def __init__(self, args):
        try:
            self.client = docker.from_env()
        except:
            traceback.print_exc()
            log.error('Cannot connect to the docker socket! Exciting now...')
            exit(2) # ENOENT

        self.builddir   = os.environ['BUILDDIR']
        self.bspdir     = os.path.realpath(os.path.join(self.builddir, '..'))
        self.dockerdir  = args.path

        self.cap_drop   = CAP_DROP
        self.dockerdir
        self.volume_binds = {
            self.bspdir : {
                                'bind' : self.bspdir,
                                'mode' : 'rw',
                          }
        }

        self.hostconfig = self.client.create_host_config(privileged=False, cap_drop=self.cap_drop, binds=self.volume_binds)


    @addlog
    def build(self):
        """ Build the isar docker image. """
        os.chdir(self.builddir)

        self.blogs = self.client.build(path=self.dockerdir, rm=True, tag='isar_image:nanopi', decode=False)
        thread = Thread(target=docker_log, args=(self.blogs, ))
        thread.start()
        thread.join()


    @addlog
    def _create(self, cmd):
        """ Create the container without starting it."""
        try:
            self.container = self.client.create_container(image='isar_image:nanopi', command=cmd, host_config=self.hostconfig)
        except:
            traceback.print_exc()
            log.error('Cannot create container! Exciting now...')
            exit(1)

    @addlog
    def run(self, cmd):
        """ Run a command in the container. """
        #
        # TODO: Use exec_create instead, so _create is not required anymore.
        #       BUT!!: Is it possible to drop capabilities then?
        #
        self._create(cmd)

        try:
            self.client.start(container=self.container.get('Id'))
            self.clogs = self.client.logs(container=self.container.get('Id'), stream=True)

            # Thread for receiving current container logs
            thread = Thread(target=docker_log, args=(self.clogs, ))
            thread.start()

            self.retcode = self.client.wait(container=self.container.get('Id'))
            thread.join()

            if self.retcode != 0:
                log.warning('Command return non zero status!')
        except:
            traceback.print_exc()
            log.error('Executing docker container! Exciting now...')
            exit(1)


def run_docker(args):
    log.debug('Running docker subcommand')
    dc = IsarDocker(args)

    if args.setup:
        dc.build()
    elif args.run:
        dc.run(args.run)

def run_bitbake(args):
    log.debug('Running bitbake subcommand')
    dc = IsarDocker(args)
    builddir = os.path.basename(dc.builddir)

    log.debug('Args: %s' % args.args)
    log.debug('Build directory: %s' % builddir)
    cmd = "bash -c 'cd %s; source setup-environment %s; bitbake %s'" % (dc.bspdir, builddir, args.args)

    log.debug(cmd)
    dc.run(cmd)



#
# Cli
#
parser = argparse.ArgumentParser(prog='isar')
parser.add_argument('--path', type=str, default='./docker', help='Path to the directory holding the dockerfile. Defaults to $BUILDDIR/docker')
subparsers = parser.add_subparsers()

# create the parser for the docker command
docker_parser = subparsers.add_parser('docker', help='docker --help')
docker_parser.add_argument('--create', action='store_true', help='Creates the basic docker image where isar builds run.'
                                                                 'The dockerfile specified by --path is used.')
docker_parser.add_argument('--run', type=str, help='Run a command in the docker container.')
docker_parser.set_defaults(func=run_docker)


# create the parser for the bitbake command
bitbake_parser = subparsers.add_parser('bitbake', help='bitbake --help')
bitbake_parser.add_argument('--args', type=str, required=True, help='Arguments forwarded to bitbake command.'
                                        'The complete argument string will be appended to the bitbake command.')
bitbake_parser.set_defaults(func=run_bitbake)

args = parser.parse_args(sys.argv[1:])

args.func(args)