Re: [PATCH] images: wic: limit use of sudo and enable manual call again

[Henning Schild <henning.schild@siemens.com>]

In this one i tried to get rid of sudo as far as possible. But thinking
about what patches will have to follow in the future, sudo will most
likely be reintroduced. And here is why:
wic has the concept of a native_sysroot where tools like mkfs, parted,
or grub-install etc. are found. These tools have a direct influence on
the image layout and content and should match the image as close as
possible. Right now we just take the tools and the bootloader from the
host, which is clearly wrong. And our patched version wic did not even
support the native_sysroot anymore.

In order to get as close as we possibly can, i suggest using
buildchroot as native_sysroot. And the way to execute stuff in there is
to call "sudo chroot", so a lot of sudos will have to come back. As a
result we will not need grub-efi and other partitioning related things
on the host anymore.

Henning

Am Thu, 1 Feb 2018 13:41:06 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> Issues:
>   1. after the wic rework wic was called under a big sudo
>   2. and calling it manually - like stated in the doc - did not work
>      anymore
> 
> Impact:
>   This patch solves both issues. Just like before sudo is only used
> for "du" and "mkfs". And by applying some tricks and wrapping we now
> can call "isar-wic" just like "wic" before.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  doc/user_manual.md                           |  4 +--
>  meta/classes/wic-img.bbclass                 |  6 ++--
>  meta/recipes-devtools/wic-tools/wic-tools.bb |  4 +++
>  scripts/isar-wic                             | 27 ++++++++++++++
>  scripts/isar-wic-handler                     | 53
> ++++++++++++++++++++++++++++
> scripts/wic_fakeroot                         | 37 -------------------
> 6 files changed, 90 insertions(+), 41 deletions(-) create mode 100644
> meta/recipes-devtools/wic-tools/wic-tools.bb create mode 100755
> scripts/isar-wic create mode 100755 scripts/isar-wic-handler
>  delete mode 100755 scripts/wic_fakeroot
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 969f6d2..e9284a0 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -208,9 +208,9 @@ Once the image artifacts have been built (c.f.
> previous section), full EFI disk Currently, only the `i386` and
> `amd64` target architectures are supported: ```
>   # Generate an EFI image for the `i386` target architecture
> - $ wic create -D sdimage-efi -o . -e
> multiconfig:qemui386-stretch:isar-image-base
> + $ isar-wic create -D sdimage-efi -o . -e
> multiconfig:qemui386-stretch:isar-image-base # Similarly, for the
> `amd64` target architecture
> - $ wic create -D sdimage-efi -o . -e
> multiconfig:qemuamd64-stretch:isar-image-base
> + $ isar-wic create -D sdimage-efi -o . -e
> multiconfig:qemuamd64-stretch:isar-image-base ```
>  
>  In order to run the images with `qemu`, an EFI firmware is required
> and available at the following address: diff --git
> a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass index
> e8d2678..d4afde6 100644 --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -14,7 +14,7 @@ STAGING_DATADIR ?= "/usr/share/"
>  STAGING_LIBDIR ?= "/usr/lib/"
>  STAGING_DIR ?= "${TMPDIR}"
>  IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}"
> -FAKEROOTCMD ?= "wic_fakeroot"
> +FAKEROOTCMD ?= "/builder/isar/scripts/isar-wic-handler"
>  RECIPE_SYSROOT_NATIVE ?= "/"
>  
>  do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> @@ -57,8 +57,10 @@ do_rootfs_wicenv[prefuncs] = 'set_image_size'
>  
>  do_wic_image() {
>      export BUILDDIR="${BUILDDIR}"
> +    export FAKEROOTCMD="${FAKEROOTCMD}"
> +    export TMPDIR="${TMPDIR}"
>  
> -    sudo -E
> PATH="$PATH:/builder/isar/bitbake/bin:/builder/isar/scripts" /builder/isar/scripts/wic
> create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o
> ${DEPLOY_DIR_IMAGE} -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
> +    isar-wic create ${WKS_FILE} --vars
> "${STAGING_DIR}/${MACHINE}/imgdata/" -o ${DEPLOY_DIR_IMAGE} -e
> ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS} } 
>  addtask wic_image before do_build after do_copy_boot_files
> diff --git a/meta/recipes-devtools/wic-tools/wic-tools.bb
> b/meta/recipes-devtools/wic-tools/wic-tools.bb new file mode 100644
> index 0000000..50ba664
> --- /dev/null
> +++ b/meta/recipes-devtools/wic-tools/wic-tools.bb
> @@ -0,0 +1,4 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2018 Siemens AG
> +# This is just a dummy  because wic might call "bitbake -e
> wic-tools" to learn wic variables +inherit wic-img
> diff --git a/scripts/isar-wic b/scripts/isar-wic
> new file mode 100755
> index 0000000..4e4d0dd
> --- /dev/null
> +++ b/scripts/isar-wic
> @@ -0,0 +1,27 @@
> +#!/bin/sh
> +#
> +# This script is a wrapper to wic that prepares everything for Isar
> specific +# needs.
> +#
> +# This software is a part of Isar.
> +# Copyright (C) 2018 Siemens AG
> +
> +set -e
> +
> +[ -z $FAKEROOTCMD ] &&
> FAKEROOTCMD="/builder/isar/scripts/isar-wic-handler" +[ -z $TMPDIR ]
> && TMPDIR=$( mktemp -d ) +
> +export MTOOLS_SKIP_CHECK=1
> +
> +# Play a dirty trick to redirect "du" and "mkfs.*" to FAKEROOTCMD
> +TRICK_SYSROOT="${TMPDIR}/trick_wic_sysroot/"
> +mkdir -p ${TRICK_SYSROOT}/sbin
> +mkdir -p ${TRICK_SYSROOT}/usr/bin
> +for fstype in btrfs ext2 ext3 ext4 vfat; do
> +  ln -sf ${FAKEROOTCMD} ${TRICK_SYSROOT}/sbin/mkfs.${fstype}
> +done
> +ln -sf ${FAKEROOTCMD} ${TRICK_SYSROOT}/usr/bin/du
> +
> +export
> PATH="${TRICK_SYSROOT}/sbin:${TRICK_SYSROOT}/usr/sbin:${TRICK_SYSROOT}/usr/bin:${PATH}"
> + +exec wic $@
> diff --git a/scripts/isar-wic-handler b/scripts/isar-wic-handler
> new file mode 100755
> index 0000000..01fe4fe
> --- /dev/null
> +++ b/scripts/isar-wic-handler
> @@ -0,0 +1,53 @@
> +#!/usr/bin/env python3
> +#
> +# This script is used to handle Isar specifics in wic without having
> to change +# wic. It is called in two cases:
> +#  1. if wic calls exec_native_cmd with pseudo != ""
> +#  2. if wic calls exec_cmd on one of our trick symlinks
> +#
> +# This software is a part of Isar.
> +# Copyright (C) 2018 Siemens AG
> +#
> +import os
> +import sys
> +import shutil
> +import subprocess
> +
> +use_sudo = False
> +native = False
> +
> +args = sys.argv
> +args[0] = os.path.basename(args[0])
> +
> +# first thing we do is remove the PATH hack that took us here
> +os.environ['PATH'] = ':'.join(os.environ['PATH'].split(':')[3:])
> +
> +if args[0] == 'isar-wic-handler':
> +    native = True
> +    args.pop(0)
> +
> +# run only "mkfs.*" and "du" with sudo, in "exec_native_cmd" and
> "exec_cmd" +if (args[0].startswith('mkfs.') or args[0] == 'du'):
> +    use_sudo = True
> +else:
> +    if not native:
> +        print('ERROR: wic_fakeroot cmd "%s" not supported in
> non-native mode.'
> +              % args[0], file=sys.stderr)
> +        sys.exit(1)
> +
> +cmd = args[0]
> +args.pop(0)
> +
> +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
> affected) +# treat 1 as safe ... the filesystem was successfully
> repaired and is OK +if cmd.startswith('fsck.'):
> +    ret = subprocess.call([cmd] + args)
> +    if ret == 0 or ret == 1:
> +        sys.exit(0)
> +    sys.exit(ret)
> +
> +if use_sudo:
> +    args = ['-E', 'PATH="%s"' % os.environ['PATH'], cmd ] + args
> +    cmd = 'sudo'
> +
> +os.execv(shutil.which(cmd), args)
> diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
> deleted file mode 100755
> index 9e01c38..0000000
> --- a/scripts/wic_fakeroot
> +++ /dev/null
> @@ -1,37 +0,0 @@
> -#!/usr/bin/env python3
> -#
> -# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar we
> do/can not -# use pseudo. And we call wic as root to begin with, so
> this script could be a -# dummy doing nothing. It is almost a
> dummy ... -#
> -# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" can
> be used -#
> -# This software is a part of Isar.
> -# Copyright (C) 2018 Siemens AG
> -#
> -import os
> -import sys
> -import shutil
> -import subprocess
> -
> -args = sys.argv
> -args.pop(0)
> -cmd = args[0]
> -
> -# expect to be running as root
> -# we could loosen that and execv(sudo, args) but even some early
> -# "du"s fail, which do not use the fakeroot-wrapper
> -#  i.e. in wics partition.py the "du -ks" fails on
> -#    var/cache/apt/archives/partial
> -#    rootfs/root ...
> -assert 'root' == os.environ["USER"]
> -
> -# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
> affected) -# treat 1 as safe ... the filesystem was successfully
> repaired and is OK -if cmd.startswith('fsck.'):
> -    ret = subprocess.call(args)
> -    if ret == 0 or ret == 1:
> -        sys.exit(0)
> -    sys.exit(ret)
> -
> -os.execv(shutil.which(cmd), args)