Re: [PATCH 2/9] images: new class wic-img for wic intregration

[Henning Schild <henning.schild@siemens.com>]

Am Tue, 13 Feb 2018 17:44:24 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> On 01/31/2018 12:41 PM, Henning Schild wrote:
> > This patch integrates wic into the bitbake workflow, wic will be
> > used for the imaging step, no need to call it manually.
> > 
> > The target of this patch is to work with an unmodified wic, that is
> > why it contains a few wic-specifics, mostly taken from OE.
> > 
> > There is one major difference to OE, we assume that wic is run as
> > root. And we smuggle in a wrapper that patches calls to an fsck
> > that does not behave like the one on OE.
> > 
> > Issues:
> >    - wic was never integrated
> >    - you always had to build an ext4-img to create a wic image later
> >    - there was never a way to control the size of wic
> > disks/partition, only directly in the wks
> > 
> > Impact:
> >    The patch solves the Issues without changing the default
> > behaviour of Isar.
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >   meta/classes/wic-img.bbclass | 64
> > ++++++++++++++++++++++++++++++++++++++++++++
> > scripts/wic_fakeroot         | 37 +++++++++++++++++++++++++ 2 files
> > changed, 101 insertions(+) create mode 100644
> > meta/classes/wic-img.bbclass create mode 100755 scripts/wic_fakeroot
> > 
> > diff --git a/meta/classes/wic-img.bbclass
> > b/meta/classes/wic-img.bbclass new file mode 100644
> > index 0000000..e8d2678
> > --- /dev/null
> > +++ b/meta/classes/wic-img.bbclass
> > @@ -0,0 +1,64 @@
> > +# This software is a part of ISAR.
> > +# Copyright (C) 2018 Siemens AG
> > +#
> > +# this class is heavily inspired by
> > OEs ./meta/classes/image_types_wic.bbclass +#
> > +
> > +WKS_FILE ?= "sdimage-efi"
> > +
> > +# TODO here we leak buildenv into the image
> > +# this needs to come from buildchroot
> > +# and the isar efi plugin has the same problem
> > +# syslinux in debian has different folder structure, need to for
> > those plugins +STAGING_DATADIR ?= "/usr/share/"
> > +STAGING_LIBDIR ?= "/usr/lib/"
> > +STAGING_DIR ?= "${TMPDIR}"
> > +IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}"
> > +FAKEROOTCMD ?= "wic_fakeroot"
> > +RECIPE_SYSROOT_NATIVE ?= "/"
> > +
> > +do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> > +
> > +WIC_CREATE_EXTRA_ARGS ?= ""
> > +
> > +WICVARS ?= "\
> > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> > INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
> > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > TARGET_SYS TRANSLATED_TARGET_ARCH" +
> > +# Isar specific vars used in our plugins
> > +WICVARS += "KERNEL_IMAGE INITRD_IMAGE DISTRO_ARCH"
> > +
> > +python do_rootfs_wicenv () {
> > +    wicvars = d.getVar('WICVARS', True)
> > +    if not wicvars:
> > +        return
> > +
> > +    stdir = d.getVar('STAGING_DIR', True)
> > +    outdir = os.path.join(stdir, d.getVar('MACHINE', True),
> > 'imgdata')
> > +    bb.utils.mkdirhier(outdir)
> > +    basename = d.getVar('IMAGE_BASENAME', True)
> > +    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > envf:
> > +        for var in wicvars.split():
> > +            value = d.getVar(var, True)
> > +            if value:
> > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > +
> > +    # this part is stolen from
> > OE ./meta/recipes-core/meta/wic-tools.bb
> > +    with open(os.path.join(outdir, "wic-tools.env"), 'w') as envf:
> > +        for var in ('RECIPE_SYSROOT_NATIVE', 'STAGING_DATADIR',
> > 'STAGING_LIBDIR'):
> > +            envf.write('%s="%s"\n' % (var, d.getVar(var,
> > True).strip())) +
> > +}
> > +
> > +addtask do_rootfs_wicenv after do_copy_boot_files before
> > do_wic_image +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > +
> > +do_wic_image() {
> > +    export BUILDDIR="${BUILDDIR}"
> > +
> > +    sudo -E
> > PATH="$PATH:/builder/isar/bitbake/bin:/builder/isar/scripts" /builder/isar/scripts/wic
> > create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o
> > ${DEPLOY_DIR_IMAGE} -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
> > +}  
> 
> How this hardcoded path intended to work?

You are right, that is the result of us building in docker where you
can assume the path to be fixed. Will make it variable.

Henning

> Alex
> 
> > +
> > +addtask wic_image before do_build after do_copy_boot_files
> > diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
> > new file mode 100755
> > index 0000000..9e01c38
> > --- /dev/null
> > +++ b/scripts/wic_fakeroot
> > @@ -0,0 +1,37 @@
> > +#!/usr/bin/env python3
> > +#
> > +# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar
> > we do/can not +# use pseudo. And we call wic as root to begin with,
> > so this script could be a +# dummy doing nothing. It is almost a
> > dummy ... +#
> > +# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;"
> > can be used +#
> > +# This software is a part of Isar.
> > +# Copyright (C) 2018 Siemens AG
> > +#
> > +import os
> > +import sys
> > +import shutil
> > +import subprocess
> > +
> > +args = sys.argv
> > +args.pop(0)
> > +cmd = args[0]
> > +
> > +# expect to be running as root
> > +# we could loosen that and execv(sudo, args) but even some early
> > +# "du"s fail, which do not use the fakeroot-wrapper
> > +#  i.e. in wics partition.py the "du -ks" fails on
> > +#    var/cache/apt/archives/partial
> > +#    rootfs/root ...
> > +assert 'root' == os.environ["USER"]
> > +
> > +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
> > affected) +# treat 1 as safe ... the filesystem was successfully
> > repaired and is OK +if cmd.startswith('fsck.'):
> > +    ret = subprocess.call(args)
> > +    if ret == 0 or ret == 1:
> > +        sys.exit(0)
> > +    sys.exit(ret)
> > +
> > +os.execv(shutil.which(cmd), args)
> >