From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6517147827419742208 X-Received: by 10.46.42.66 with SMTP id q63mr795700ljq.37.1518462473268; Mon, 12 Feb 2018 11:07:53 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.193.87 with SMTP id r84ls269167lff.9.gmail; Mon, 12 Feb 2018 11:07:52 -0800 (PST) X-Google-Smtp-Source: AH8x224hm+pP2VYSa0VRbJSSy3fM4LjiQAowRacY6FPpJl+EHHE0ekqhiA5hob4ZueBNJF7ezNwp X-Received: by 10.25.205.202 with SMTP id d193mr882259lfg.16.1518462472589; Mon, 12 Feb 2018 11:07:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518462472; cv=none; d=google.com; s=arc-20160816; b=XOGTGSXm9s//9ZUpa3NqfZ8I9Z5GsYIf7y6qeytLwC15IXls2GjTLYFOd3ah5cy4Vh utOjnyvtACORBX2y1cbk0UT3Bke34tUWsvH3ckyS43ry8ESX97OdcMiDwFiDPC76XqbR AqKMTTXbVQKHo2kn8lYH4nuG0FakWqtrWoILBBhXOLantBJKleLt/DfKwBgpb8y2QNhq wzTK3nVkPMh7z+ICMFErKg5js9Drxmrl4MUagNwbdk+Nr+mRfg6538Fv6k9mB7suL8L8 VTObZubPOre/sXb2Y+QGP1yCtqPRGrkOnYW7XmVLvnvXfnHuHEtDUQOqZv17uyGHhZfy tcyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=kC137EM5gueM6Umo3xee1bbxvDmPanU6OfMkHIY6b9s=; b=rxiHqapCRNAbYwOPEYnpbK9wSk1N8s12+xTthgVv7eir8zf5BII0sIUc+WYDgOkEJX nR5xtunptZq9eZcyIaCQ1Vgy7Z13c+2gP8wMplP72v+8GFNwfQKWziavwcZDp74eVjqb krSl6iIXO0AfGuCpNqm7fZE3vrX4sWHJby3bRVTy8lxLpI/XPQ0s5AkquO09e2Nez24M gDswIRqnLFX3q1LdqZEd4r3gf0JCVBP+LmqCKQaOjMGfIogU5MfX4rlqeqGHfOZhKz0e +jz3PxKhzaTzTbLaWYlaAYsv9kYtC4SvsYAqgJkmx+aOCmeikdPmJ4ShMqLnocInzCAD Gvsg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id o26si471938ljc.5.2018.02.12.11.07.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Feb 2018 11:07:52 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w1CJ7pdl008888 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 12 Feb 2018 20:07:51 +0100 Received: from mmd1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w1CJ7pOl024370; Mon, 12 Feb 2018 20:07:51 +0100 Date: Mon, 12 Feb 2018 20:07:53 +0100 From: Henning Schild To: Cc: Baurzhan Ismagulov , Alexander Smirnov Subject: Re: [PATCH] images: wic: limit use of sudo and enable manual call again Message-ID: <20180212200753.2a34e502@mmd1pvb1c.ad001.siemens.net> In-Reply-To: <20180201124106.29397-1-henning.schild@siemens.com> References: <20180201124106.29397-1-henning.schild@siemens.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: J3qkMa6Iqtu2 In this one i tried to get rid of sudo as far as possible. But thinking about what patches will have to follow in the future, sudo will most likely be reintroduced. And here is why: wic has the concept of a native_sysroot where tools like mkfs, parted, or grub-install etc. are found. These tools have a direct influence on the image layout and content and should match the image as close as possible. Right now we just take the tools and the bootloader from the host, which is clearly wrong. And our patched version wic did not even support the native_sysroot anymore. In order to get as close as we possibly can, i suggest using buildchroot as native_sysroot. And the way to execute stuff in there is to call "sudo chroot", so a lot of sudos will have to come back. As a result we will not need grub-efi and other partitioning related things on the host anymore. Henning Am Thu, 1 Feb 2018 13:41:06 +0100 schrieb Henning Schild : > Issues: > 1. after the wic rework wic was called under a big sudo > 2. and calling it manually - like stated in the doc - did not work > anymore > > Impact: > This patch solves both issues. Just like before sudo is only used > for "du" and "mkfs". And by applying some tricks and wrapping we now > can call "isar-wic" just like "wic" before. > > Signed-off-by: Henning Schild > --- > doc/user_manual.md | 4 +-- > meta/classes/wic-img.bbclass | 6 ++-- > meta/recipes-devtools/wic-tools/wic-tools.bb | 4 +++ > scripts/isar-wic | 27 ++++++++++++++ > scripts/isar-wic-handler | 53 > ++++++++++++++++++++++++++++ > scripts/wic_fakeroot | 37 ------------------- > 6 files changed, 90 insertions(+), 41 deletions(-) create mode 100644 > meta/recipes-devtools/wic-tools/wic-tools.bb create mode 100755 > scripts/isar-wic create mode 100755 scripts/isar-wic-handler > delete mode 100755 scripts/wic_fakeroot > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index 969f6d2..e9284a0 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -208,9 +208,9 @@ Once the image artifacts have been built (c.f. > previous section), full EFI disk Currently, only the `i386` and > `amd64` target architectures are supported: ``` > # Generate an EFI image for the `i386` target architecture > - $ wic create -D sdimage-efi -o . -e > multiconfig:qemui386-stretch:isar-image-base > + $ isar-wic create -D sdimage-efi -o . -e > multiconfig:qemui386-stretch:isar-image-base # Similarly, for the > `amd64` target architecture > - $ wic create -D sdimage-efi -o . -e > multiconfig:qemuamd64-stretch:isar-image-base > + $ isar-wic create -D sdimage-efi -o . -e > multiconfig:qemuamd64-stretch:isar-image-base ``` > > In order to run the images with `qemu`, an EFI firmware is required > and available at the following address: diff --git > a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass index > e8d2678..d4afde6 100644 --- a/meta/classes/wic-img.bbclass > +++ b/meta/classes/wic-img.bbclass > @@ -14,7 +14,7 @@ STAGING_DATADIR ?= "/usr/share/" > STAGING_LIBDIR ?= "/usr/lib/" > STAGING_DIR ?= "${TMPDIR}" > IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}" > -FAKEROOTCMD ?= "wic_fakeroot" > +FAKEROOTCMD ?= "/builder/isar/scripts/isar-wic-handler" > RECIPE_SYSROOT_NATIVE ?= "/" > > do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}" > @@ -57,8 +57,10 @@ do_rootfs_wicenv[prefuncs] = 'set_image_size' > > do_wic_image() { > export BUILDDIR="${BUILDDIR}" > + export FAKEROOTCMD="${FAKEROOTCMD}" > + export TMPDIR="${TMPDIR}" > > - sudo -E > PATH="$PATH:/builder/isar/bitbake/bin:/builder/isar/scripts" /builder/isar/scripts/wic > create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o > ${DEPLOY_DIR_IMAGE} -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS} > + isar-wic create ${WKS_FILE} --vars > "${STAGING_DIR}/${MACHINE}/imgdata/" -o ${DEPLOY_DIR_IMAGE} -e > ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS} } > addtask wic_image before do_build after do_copy_boot_files > diff --git a/meta/recipes-devtools/wic-tools/wic-tools.bb > b/meta/recipes-devtools/wic-tools/wic-tools.bb new file mode 100644 > index 0000000..50ba664 > --- /dev/null > +++ b/meta/recipes-devtools/wic-tools/wic-tools.bb > @@ -0,0 +1,4 @@ > +# This software is a part of ISAR. > +# Copyright (C) 2018 Siemens AG > +# This is just a dummy because wic might call "bitbake -e > wic-tools" to learn wic variables +inherit wic-img > diff --git a/scripts/isar-wic b/scripts/isar-wic > new file mode 100755 > index 0000000..4e4d0dd > --- /dev/null > +++ b/scripts/isar-wic > @@ -0,0 +1,27 @@ > +#!/bin/sh > +# > +# This script is a wrapper to wic that prepares everything for Isar > specific +# needs. > +# > +# This software is a part of Isar. > +# Copyright (C) 2018 Siemens AG > + > +set -e > + > +[ -z $FAKEROOTCMD ] && > FAKEROOTCMD="/builder/isar/scripts/isar-wic-handler" +[ -z $TMPDIR ] > && TMPDIR=$( mktemp -d ) + > +export MTOOLS_SKIP_CHECK=1 > + > +# Play a dirty trick to redirect "du" and "mkfs.*" to FAKEROOTCMD > +TRICK_SYSROOT="${TMPDIR}/trick_wic_sysroot/" > +mkdir -p ${TRICK_SYSROOT}/sbin > +mkdir -p ${TRICK_SYSROOT}/usr/bin > +for fstype in btrfs ext2 ext3 ext4 vfat; do > + ln -sf ${FAKEROOTCMD} ${TRICK_SYSROOT}/sbin/mkfs.${fstype} > +done > +ln -sf ${FAKEROOTCMD} ${TRICK_SYSROOT}/usr/bin/du > + > +export > PATH="${TRICK_SYSROOT}/sbin:${TRICK_SYSROOT}/usr/sbin:${TRICK_SYSROOT}/usr/bin:${PATH}" > + +exec wic $@ > diff --git a/scripts/isar-wic-handler b/scripts/isar-wic-handler > new file mode 100755 > index 0000000..01fe4fe > --- /dev/null > +++ b/scripts/isar-wic-handler > @@ -0,0 +1,53 @@ > +#!/usr/bin/env python3 > +# > +# This script is used to handle Isar specifics in wic without having > to change +# wic. It is called in two cases: > +# 1. if wic calls exec_native_cmd with pseudo != "" > +# 2. if wic calls exec_cmd on one of our trick symlinks > +# > +# This software is a part of Isar. > +# Copyright (C) 2018 Siemens AG > +# > +import os > +import sys > +import shutil > +import subprocess > + > +use_sudo = False > +native = False > + > +args = sys.argv > +args[0] = os.path.basename(args[0]) > + > +# first thing we do is remove the PATH hack that took us here > +os.environ['PATH'] = ':'.join(os.environ['PATH'].split(':')[3:]) > + > +if args[0] == 'isar-wic-handler': > + native = True > + args.pop(0) > + > +# run only "mkfs.*" and "du" with sudo, in "exec_native_cmd" and > "exec_cmd" +if (args[0].startswith('mkfs.') or args[0] == 'du'): > + use_sudo = True > +else: > + if not native: > + print('ERROR: wic_fakeroot cmd "%s" not supported in > non-native mode.' > + % args[0], file=sys.stderr) > + sys.exit(1) > + > +cmd = args[0] > +args.pop(0) > + > +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before > affected) +# treat 1 as safe ... the filesystem was successfully > repaired and is OK +if cmd.startswith('fsck.'): > + ret = subprocess.call([cmd] + args) > + if ret == 0 or ret == 1: > + sys.exit(0) > + sys.exit(ret) > + > +if use_sudo: > + args = ['-E', 'PATH="%s"' % os.environ['PATH'], cmd ] + args > + cmd = 'sudo' > + > +os.execv(shutil.which(cmd), args) > diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot > deleted file mode 100755 > index 9e01c38..0000000 > --- a/scripts/wic_fakeroot > +++ /dev/null > @@ -1,37 +0,0 @@ > -#!/usr/bin/env python3 > -# > -# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar we > do/can not -# use pseudo. And we call wic as root to begin with, so > this script could be a -# dummy doing nothing. It is almost a > dummy ... -# > -# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" can > be used -# > -# This software is a part of Isar. > -# Copyright (C) 2018 Siemens AG > -# > -import os > -import sys > -import shutil > -import subprocess > - > -args = sys.argv > -args.pop(0) > -cmd = args[0] > - > -# expect to be running as root > -# we could loosen that and execv(sudo, args) but even some early > -# "du"s fail, which do not use the fakeroot-wrapper > -# i.e. in wics partition.py the "du -ks" fails on > -# var/cache/apt/archives/partial > -# rootfs/root ... > -assert 'root' == os.environ["USER"] > - > -# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before > affected) -# treat 1 as safe ... the filesystem was successfully > repaired and is OK -if cmd.startswith('fsck.'): > - ret = subprocess.call(args) > - if ret == 0 or ret == 1: > - sys.exit(0) > - sys.exit(ret) > - > -os.execv(shutil.which(cmd), args)