From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6517147827419742208 X-Received: by 10.80.149.81 with SMTP id v17mr713435eda.10.1518538005988; Tue, 13 Feb 2018 08:06:45 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.202.5 with SMTP id d5ls4124531edi.8.gmail; Tue, 13 Feb 2018 08:06:45 -0800 (PST) X-Google-Smtp-Source: AH8x227x5VPP5ZKDDTLSVUT0qir0Jg/spmw+oiysvn9wHnaUeHNxP3Xdw2a58qidOZctduV4GWlt X-Received: by 10.80.202.201 with SMTP id f9mr716573edi.12.1518538005274; Tue, 13 Feb 2018 08:06:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518538005; cv=none; d=google.com; s=arc-20160816; b=B6wPY0Phl2aXVJ+jxSDdIThsq8b1Cfty0Ybq89es0Sr0733Ys4No6kMHO91LrhRU9b gRUm9rxTxqeLar+qbi1y5R0UcI3PZS+Edc0uzdPxrVLotXequANUUnETAI9LviGUyvGk OOdQUQH7DfhOEtskjMrz9OLa1g50kj0YtnrjLdGccxWkc1g/0rywOtlqlPX+wtMVP4vE 6WPY64FSuBwHgMenTfPdJuCCBi2UsnCUVPQaoreOFbt4fMleXDvWwjsl8nC6e9AOim3F 7e/ThR5mF5/jW+ycD9SCwQcjMLXKvCScSaHbXMdHtknuS91joAFO1wuHzNuW0WN+QNSx Om2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=RQRLqeWyC8F24gjqe5//+QJQiAOuMVwBZC5x4Q5vhrc=; b=tnrKb+cSgmrsYLCYyGKytUlnpOgnf0eyd63wdWDLkB4c/DraFkM837hNiFPejhbs11 9j+VjIpxF1PdLy5fU0xAFbqJHe7emb5QEvSnoYRgkVamljTBzuRPh/ifv8bWgSNf4MRm hopkkNjATep4EOsh/vWcS/06dm2Ui10233AAWDc4kH0RKQT5dTs9nWCYfqbV6WwGopvI mWXtfmZ4j0TyTYif8fd1B1fQTXNxgYpv5E5bd5rj0A/u77ipvCKAZrIUanX7baVoD+eF JaA9OTl84kU1eDQjPFohws9cFoqPYasf4jBdPgpDGrGg63VDEzOQIpWDPn+fYr7mg44q T2aw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id g10si221953edm.0.2018.02.13.08.06.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Feb 2018 08:06:45 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w1DG6i58010190 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2018 17:06:44 +0100 Received: from mmd1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40] (may be forged)) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w1DG6iCT025317; Tue, 13 Feb 2018 17:06:44 +0100 Date: Tue, 13 Feb 2018 17:06:43 +0100 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH 2/9] images: new class wic-img for wic intregration Message-ID: <20180213170643.1c1b434f@mmd1pvb1c.ad001.siemens.net> In-Reply-To: <9342ef1b-ea13-06cf-3f31-8e86eac2cf76@ilbers.de> References: <122454433490bad5b449b9a984c4578da525a03a.1517390790.git.henning.schild@siemens.com> <9342ef1b-ea13-06cf-3f31-8e86eac2cf76@ilbers.de> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: NPLPaOO7JmdS Am Tue, 13 Feb 2018 17:44:24 +0300 schrieb Alexander Smirnov : > On 01/31/2018 12:41 PM, Henning Schild wrote: > > This patch integrates wic into the bitbake workflow, wic will be > > used for the imaging step, no need to call it manually. > > > > The target of this patch is to work with an unmodified wic, that is > > why it contains a few wic-specifics, mostly taken from OE. > > > > There is one major difference to OE, we assume that wic is run as > > root. And we smuggle in a wrapper that patches calls to an fsck > > that does not behave like the one on OE. > > > > Issues: > > - wic was never integrated > > - you always had to build an ext4-img to create a wic image later > > - there was never a way to control the size of wic > > disks/partition, only directly in the wks > > > > Impact: > > The patch solves the Issues without changing the default > > behaviour of Isar. > > > > Signed-off-by: Henning Schild > > --- > > meta/classes/wic-img.bbclass | 64 > > ++++++++++++++++++++++++++++++++++++++++++++ > > scripts/wic_fakeroot | 37 +++++++++++++++++++++++++ 2 files > > changed, 101 insertions(+) create mode 100644 > > meta/classes/wic-img.bbclass create mode 100755 scripts/wic_fakeroot > > > > diff --git a/meta/classes/wic-img.bbclass > > b/meta/classes/wic-img.bbclass new file mode 100644 > > index 0000000..e8d2678 > > --- /dev/null > > +++ b/meta/classes/wic-img.bbclass > > @@ -0,0 +1,64 @@ > > +# This software is a part of ISAR. > > +# Copyright (C) 2018 Siemens AG > > +# > > +# this class is heavily inspired by > > OEs ./meta/classes/image_types_wic.bbclass +# > > + > > +WKS_FILE ?= "sdimage-efi" > > + > > +# TODO here we leak buildenv into the image > > +# this needs to come from buildchroot > > +# and the isar efi plugin has the same problem > > +# syslinux in debian has different folder structure, need to for > > those plugins +STAGING_DATADIR ?= "/usr/share/" > > +STAGING_LIBDIR ?= "/usr/lib/" > > +STAGING_DIR ?= "${TMPDIR}" > > +IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}" > > +FAKEROOTCMD ?= "wic_fakeroot" > > +RECIPE_SYSROOT_NATIVE ?= "/" > > + > > +do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}" > > + > > +WIC_CREATE_EXTRA_ARGS ?= "" > > + > > +WICVARS ?= "\ > > + BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD > > IMAGE_BASENAME IMAGE_BOOT_FILES \ > > + IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD > > INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \ > > + ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR > > TARGET_SYS TRANSLATED_TARGET_ARCH" + > > +# Isar specific vars used in our plugins > > +WICVARS += "KERNEL_IMAGE INITRD_IMAGE DISTRO_ARCH" > > + > > +python do_rootfs_wicenv () { > > + wicvars = d.getVar('WICVARS', True) > > + if not wicvars: > > + return > > + > > + stdir = d.getVar('STAGING_DIR', True) > > + outdir = os.path.join(stdir, d.getVar('MACHINE', True), > > 'imgdata') > > + bb.utils.mkdirhier(outdir) > > + basename = d.getVar('IMAGE_BASENAME', True) > > + with open(os.path.join(outdir, basename) + '.env', 'w') as > > envf: > > + for var in wicvars.split(): > > + value = d.getVar(var, True) > > + if value: > > + envf.write('%s="%s"\n' % (var, value.strip())) > > + > > + # this part is stolen from > > OE ./meta/recipes-core/meta/wic-tools.bb > > + with open(os.path.join(outdir, "wic-tools.env"), 'w') as envf: > > + for var in ('RECIPE_SYSROOT_NATIVE', 'STAGING_DATADIR', > > 'STAGING_LIBDIR'): > > + envf.write('%s="%s"\n' % (var, d.getVar(var, > > True).strip())) + > > +} > > + > > +addtask do_rootfs_wicenv after do_copy_boot_files before > > do_wic_image +do_rootfs_wicenv[vardeps] += "${WICVARS}" > > +do_rootfs_wicenv[prefuncs] = 'set_image_size' > > + > > +do_wic_image() { > > + export BUILDDIR="${BUILDDIR}" > > + > > + sudo -E > > PATH="$PATH:/builder/isar/bitbake/bin:/builder/isar/scripts" /builder/isar/scripts/wic > > create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o > > ${DEPLOY_DIR_IMAGE} -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS} > > +} > > How this hardcoded path intended to work? You are right, that is the result of us building in docker where you can assume the path to be fixed. Will make it variable. Henning > Alex > > > + > > +addtask wic_image before do_build after do_copy_boot_files > > diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot > > new file mode 100755 > > index 0000000..9e01c38 > > --- /dev/null > > +++ b/scripts/wic_fakeroot > > @@ -0,0 +1,37 @@ > > +#!/usr/bin/env python3 > > +# > > +# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar > > we do/can not +# use pseudo. And we call wic as root to begin with, > > so this script could be a +# dummy doing nothing. It is almost a > > dummy ... +# > > +# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" > > can be used +# > > +# This software is a part of Isar. > > +# Copyright (C) 2018 Siemens AG > > +# > > +import os > > +import sys > > +import shutil > > +import subprocess > > + > > +args = sys.argv > > +args.pop(0) > > +cmd = args[0] > > + > > +# expect to be running as root > > +# we could loosen that and execv(sudo, args) but even some early > > +# "du"s fail, which do not use the fakeroot-wrapper > > +# i.e. in wics partition.py the "du -ks" fails on > > +# var/cache/apt/archives/partial > > +# rootfs/root ... > > +assert 'root' == os.environ["USER"] > > + > > +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before > > affected) +# treat 1 as safe ... the filesystem was successfully > > repaired and is OK +if cmd.startswith('fsck.'): > > + ret = subprocess.call(args) > > + if ret == 0 or ret == 1: > > + sys.exit(0) > > + sys.exit(ret) > > + > > +os.execv(shutil.which(cmd), args) > >