From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6864480010557718528 X-Received: by 2002:adf:e852:: with SMTP id d18mr1725904wrn.40.1599296459111; Sat, 05 Sep 2020 02:00:59 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:1b8a:: with SMTP id b132ls5698356wmb.2.gmail; Sat, 05 Sep 2020 02:00:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzbyPo1LWz/YrgOVyQ99QFD4GPfANH0AZbzFrpwGq/mHbuInkvc37u6wwQ1m0FwS8gjExb9 X-Received: by 2002:a1c:5605:: with SMTP id k5mr6188500wmb.142.1599296458048; Sat, 05 Sep 2020 02:00:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1599296458; cv=none; d=google.com; s=arc-20160816; b=DsgcuZgSWz+wVfpC0k1uUyScoaKdskok70l10wF0E9Kbz2u+1sEvF8sBQXYLxJifez hkNAHUXaEkUrkAUccB7ClukmAOG/2tGuujC02Q3R4y9+CrjQkmyBZeu17LbIPAcKbn27 B4M5gSGrRvkrE8P3uzzwik9fBP6GPe8/0IRGs9YgCczCvHGj+8VigIUekSLuQoqknhb+ y2KF3S4eYShLnOzPraNUc2yS6KRYbwsjyH4PDyGZnAHBGKihyrIXhoeyo8II5n9woofu ZMO4XD3xMx8hXzJ3Nw8ONRJA6aOyN2UZop/WRXiQJALh9V7G2FxqMBezw4WhGKhs5StD f4Aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=6Y2xkAVmMTfafB34J6ONSgLXvlesM6DvoWLEP2M+P0Q=; b=RMRPUC0C9D0FXa4pyx4TKiGhM4LeXhwDGFztb2x8q8uxr+dWV14i/ReiYbxCDddMOp XALHixOSSpBFiq6w7mlSQGSF5PA8BriqhouaA1Z0VX+6F07lT2jo506icGSkdcya59ZB vKkSgo3rfYU/VAx632Uz3a+ddVHYlyl+xs7cccZYBgaWpPuv4jzmIEHB9ehBafzAiHzy 7xdzm2BRsBrHAGI4ppCUVIN8Iny6mvvnCzsTdrMYlDYoU8PfqKUYlBImBETUQo4TNwbN x2tOoEqZ63yPcr3tjN9CzhGk4lRugmDN1mdICNaL+H4e0MsrI/WeokOn/uRp9gCo7t3c leEA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id k14si274535wrx.1.2020.09.05.02.00.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Sep 2020 02:00:58 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 08590vO5005185 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 5 Sep 2020 11:00:57 +0200 Received: from md1za8fc.ad001.siemens.net ([167.87.13.147]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 08590u8J021917; Sat, 5 Sep 2020 11:00:56 +0200 Date: Sat, 5 Sep 2020 11:00:55 +0200 From: Henning Schild To: Vijai Kumar K Cc: Subject: Re: [PATCH v2 01/10] wic: Update to the latest wic from openembedded core Message-ID: <20200905110055.6d9ce1f1@md1za8fc.ad001.siemens.net> In-Reply-To: <20200902185624.15044-2-Vijaikumar_Kanagarajan@mentor.com> References: <20200902185624.15044-1-Vijaikumar_Kanagarajan@mentor.com> <20200902185624.15044-2-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: Claws Mail 3.17.6 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 15DTVLdfzgjC I assume if i took that commit and copied it over i would end up with 100% the same patch, right? On Thu, 3 Sep 2020 00:26:15 +0530 Vijai Kumar K wrote: > Update to the latest wic from OE-core. > > OE-core Revision: 532ae127c52c9f7b1d2e4ca5cbca91881d23a2ac > > Signed-off-by: Vijai Kumar K > --- > scripts/lib/scriptpath.py | 32 ++ > scripts/lib/wic/__init__.py | 14 +- > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > .../directdisk-bootloader-config.cfg | 8 +- > .../lib/wic/canned-wks/efi-bootdisk.wks.in | 3 + > scripts/lib/wic/canned-wks/mkhybridiso.wks | 2 +- > scripts/lib/wic/canned-wks/qemuriscv.wks | 3 + > .../lib/wic/canned-wks/qemux86-directdisk.wks | 2 +- > .../lib/wic/canned-wks/sdimage-bootpart.wks | 4 +- > .../lib/wic/canned-wks/systemd-bootdisk.wks | 4 +- > scripts/lib/wic/engine.py | 421 +++++++++++++++- > scripts/lib/wic/filemap.py | 170 ++++--- > scripts/lib/wic/help.py | 401 ++++++++++++++-- > scripts/lib/wic/ksparser.py | 115 +++-- > scripts/lib/wic/{utils => }/misc.py | 99 ++-- > scripts/lib/wic/partition.py | 234 ++++----- > scripts/lib/wic/pluginbase.py | 36 +- > scripts/lib/wic/plugins/imager/direct.py | 175 ++++--- > .../wic/plugins/source/bootimg-biosplusefi.py | 213 +++++++++ > scripts/lib/wic/plugins/source/bootimg-efi.py | 111 +++-- > .../wic/plugins/source/bootimg-partition.py | 153 ++++-- > .../lib/wic/plugins/source/bootimg-pcbios.py | 91 ++-- > scripts/lib/wic/plugins/source/fsimage.py | 56 --- > .../wic/plugins/source/isoimage-isohybrid.py | 185 +++---- > scripts/lib/wic/plugins/source/rawcopy.py | 44 +- > scripts/lib/wic/plugins/source/rootfs.py | 159 ++++-- > scripts/lib/wic/utils/__init__.py | 0 > scripts/lib/wic/utils/runner.py | 114 ----- > scripts/wic | 452 > +++++++++++++----- 29 files changed, 2283 insertions(+), 1020 > deletions(-) create mode 100644 scripts/lib/scriptpath.py > create mode 100644 scripts/lib/wic/canned-wks/efi-bootdisk.wks.in > create mode 100644 scripts/lib/wic/canned-wks/qemuriscv.wks > rename scripts/lib/wic/{utils => }/misc.py (70%) > create mode 100644 > scripts/lib/wic/plugins/source/bootimg-biosplusefi.py delete mode > 100644 scripts/lib/wic/plugins/source/fsimage.py delete mode 100644 > scripts/lib/wic/utils/__init__.py delete mode 100644 > scripts/lib/wic/utils/runner.py > > diff --git a/scripts/lib/scriptpath.py b/scripts/lib/scriptpath.py > new file mode 100644 > index 0000000..f32326d > --- /dev/null > +++ b/scripts/lib/scriptpath.py > @@ -0,0 +1,32 @@ > +# Path utility functions for OE python scripts > +# > +# Copyright (C) 2012-2014 Intel Corporation > +# Copyright (C) 2011 Mentor Graphics Corporation > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > + > +import sys > +import os > +import os.path > + > +def add_oe_lib_path(): > + basepath = os.path.abspath(os.path.dirname(__file__) + '/../..') > + newpath = basepath + '/meta/lib' > + sys.path.insert(0, newpath) > + > +def add_bitbake_lib_path(): > + basepath = os.path.abspath(os.path.dirname(__file__) + '/../..') > + bitbakepath = None > + if os.path.exists(basepath + '/bitbake/lib/bb'): > + bitbakepath = basepath + '/bitbake' > + else: > + # look for bitbake/bin dir in PATH > + for pth in os.environ['PATH'].split(':'): > + if os.path.exists(os.path.join(pth, '../lib/bb')): > + bitbakepath = os.path.abspath(os.path.join(pth, > '..')) > + break > + > + if bitbakepath: > + sys.path.insert(0, bitbakepath + '/lib') > + return bitbakepath > diff --git a/scripts/lib/wic/__init__.py b/scripts/lib/wic/__init__.py > index 85876b1..8556793 100644 > --- a/scripts/lib/wic/__init__.py > +++ b/scripts/lib/wic/__init__.py > @@ -1,20 +1,10 @@ > -#!/usr/bin/env python -tt > +#!/usr/bin/env python3 > # > # Copyright (c) 2007 Red Hat, Inc. > # Copyright (c) 2011 Intel, Inc. > # > -# This program is free software; you can redistribute it and/or > modify it -# under the terms of the GNU General Public License as > published by the Free -# Software Foundation; version 2 of the License > +# SPDX-License-Identifier: GPL-2.0-only > # > -# This program is distributed in the hope that it will be useful, but > -# WITHOUT ANY WARRANTY; without even the implied warranty of > MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > General Public License -# for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA > 02111-1307, USA. > class WicError(Exception): > pass > diff --git a/scripts/lib/wic/canned-wks/common.wks.inc > b/scripts/lib/wic/canned-wks/common.wks.inc index 5cf2fd1..89880b4 > 100644 --- a/scripts/lib/wic/canned-wks/common.wks.inc > +++ b/scripts/lib/wic/canned-wks/common.wks.inc > @@ -1,3 +1,3 @@ > # This file is included into 3 canned wks files from this directory > part /boot --source bootimg-pcbios --ondisk sda --label boot > --active --align 1024 -part / --source rootfs --ondisk sda > --fstype=ext4 --label platform --align 1024 +part / --source rootfs > --use-uuid --fstype=ext4 --label platform --align 1024 diff --git > a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg > b/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg index > d5a07d2..c58e74a 100644 --- > a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg +++ > b/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg @@ > -12,16 +12,16 @@ DEFAULT Graphics console boot LABEL Graphics console > boot KERNEL /vmlinuz > -APPEND label=boot root=/dev/sda2 rootwait > +APPEND label=boot rootwait > > LABEL Serial console boot > KERNEL /vmlinuz > -APPEND label=boot root=/dev/sda2 rootwait console=ttyS0,115200 > +APPEND label=boot rootwait console=ttyS0,115200 > > LABEL Graphics console install > KERNEL /vmlinuz > -APPEND label=install root=/dev/sda2 rootwait > +APPEND label=install rootwait > > LABEL Serial console install > KERNEL /vmlinuz > -APPEND label=install root=/dev/sda2 rootwait console=ttyS0,115200 > +APPEND label=install rootwait console=ttyS0,115200 > diff --git a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in > b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in new file mode 100644 > index 0000000..7300e65 > --- /dev/null > +++ b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in > @@ -0,0 +1,3 @@ > +bootloader --ptable gpt > +part /boot --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/boot > --fstype=vfat --label boot --active --align 1024 --use-uuid > --overhead-factor 1.0 +part / --source rootfs --fstype=ext4 --label > root --align 1024 --exclude-path boot/ diff --git > a/scripts/lib/wic/canned-wks/mkhybridiso.wks > b/scripts/lib/wic/canned-wks/mkhybridiso.wks index 9d34e9b..48c5ac4 > 100644 --- a/scripts/lib/wic/canned-wks/mkhybridiso.wks +++ > b/scripts/lib/wic/canned-wks/mkhybridiso.wks @@ -2,6 +2,6 @@ # > long-description: Creates an EFI and legacy bootable hybrid ISO image > # which can be used on optical media as well as USB media. > -part /boot --source isoimage-isohybrid > --sourceparams="loader=grub-efi,image_name=HYBRID_ISO_IMG" --ondisk > cd --label HYBRIDISO --fstype=ext4 +part /boot --source > isoimage-isohybrid > --sourceparams="loader=grub-efi,image_name=HYBRID_ISO_IMG" --ondisk > cd --label HYBRIDISO bootloader --timeout=15 --append="" diff --git > a/scripts/lib/wic/canned-wks/qemuriscv.wks > b/scripts/lib/wic/canned-wks/qemuriscv.wks new file mode 100644 index > 0000000..12c68b7 --- /dev/null +++ > b/scripts/lib/wic/canned-wks/qemuriscv.wks @@ -0,0 +1,3 @@ > +# short-description: Create qcow2 image for RISC-V QEMU machines > + > +part / --source rootfs --fstype=ext4 --label root --align 4096 > --size 5G diff --git > a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks > b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks index > a6518a0..22b4521 100644 --- > a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks +++ > b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks @@ -4,5 +4,5 @@ > include common.wks.inc > > -bootloader --timeout=0 --append="vga=0 > uvesafb.mode_option=640x480-32 root=/dev/vda2 rw mem=256M > ip=192.168.7.2::192.168.7.1:255.255.255.0 oprofile.timer=1 > rootfstype=ext4 " +bootloader --timeout=0 --append="rw > oprofile.timer=1 rootfstype=ext4 " diff --git > a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks > b/scripts/lib/wic/canned-wks/sdimage-bootpart.wks index > 7ffd632..63bc4da 100644 --- > a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks +++ > b/scripts/lib/wic/canned-wks/sdimage-bootpart.wks @@ -2,5 +2,5 @@ # > long-description: Creates a partitioned SD card image. Boot files # > are located in the first vfat partition. -part /boot --source > bootimg-partition --ondisk mmcblk --fstype=vfat --label boot --active > --align 4 --size 16 -part / --source rootfs --ondisk mmcblk > --fstype=ext4 --label root --align 4 +part /boot --source > bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot > --active --align 4 --size 16 +part / --source rootfs --ondisk mmcblk0 > --fstype=ext4 --label root --align 4 diff --git > a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks > b/scripts/lib/wic/canned-wks/systemd-bootdisk.wks index > 4bd9d6a..95d7b97 100644 --- > a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks +++ > b/scripts/lib/wic/canned-wks/systemd-bootdisk.wks @@ -2,10 +2,10 @@ # > long-description: Creates a partitioned EFI disk image that the user > # can directly dd to boot media. The selected bootloader is > systemd-boot. -part /boot --source bootimg-efi > --sourceparams="loader=systemd-boot" --ondisk sda --label msdos > --active --align 1024 +part /boot --source bootimg-efi > --sourceparams="loader=systemd-boot" --ondisk sda --label msdos > --active --align 1024 --use-uuid part / --source rootfs --ondisk sda > --fstype=ext4 --label platform --align 1024 --use-uuid -part swap > --ondisk sda --size 44 --label swap1 --fstype=swap +part swap > --ondisk sda --size 44 --label swap1 --fstype=swap --use-uuid > bootloader --ptable gpt --timeout=5 --append="rootwait > rootfstype=ext4 console=ttyS0,115200 console=tty0" diff --git > a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py index > f59821f..018815b 100644 --- a/scripts/lib/wic/engine.py +++ > b/scripts/lib/wic/engine.py @@ -1,21 +1,7 @@ -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2013, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > > @@ -30,10 +16,18 @@ > > import logging > import os > +import tempfile > +import json > +import subprocess > +import re > + > +from collections import namedtuple, OrderedDict > +from distutils.spawn import find_executable > > from wic import WicError > +from wic.filemap import sparse_copy > from wic.pluginbase import PluginMgr > -from wic.utils.misc import get_bitbake_var > +from wic.misc import get_bitbake_var, exec_cmd > > logger = logging.getLogger('wic') > > @@ -82,7 +76,8 @@ def find_canned_image(scripts_path, wks_file): > for fname in files: > if fname.endswith("~") or fname.endswith("#"): > continue > - if fname.endswith(".wks") and wks_file + ".wks" == > fname: > + if ((fname.endswith(".wks") and wks_file + ".wks" == > fname) or \ > + (fname.endswith(".wks.in") and wks_file + > ".wks.in" == fname)): fullpath = os.path.join(canned_wks_dir, fname) > return fullpath > return None > @@ -99,7 +94,7 @@ def list_canned_images(scripts_path): > for fname in files: > if fname.endswith("~") or fname.endswith("#"): > continue > - if fname.endswith(".wks"): > + if fname.endswith(".wks") or > fname.endswith(".wks.in"): fullpath = os.path.join(canned_wks_dir, > fname) with open(fullpath) as wks: > for line in wks: > @@ -108,7 +103,7 @@ def list_canned_images(scripts_path): > if idx != -1: > desc = line[idx + > len("short-description:"):].strip() break > - basename = os.path.splitext(fname)[0] > + basename = fname.split('.')[0] > print(" %s\t\t%s" % (basename.ljust(30), desc)) > > > @@ -184,7 +179,7 @@ def wic_create(wks_file, rootfs_dir, bootimg_dir, > kernel_dir, if not os.path.exists(options.outdir): > os.makedirs(options.outdir) > > - pname = 'direct' > + pname = options.imager > plugin_class = PluginMgr.get_plugins('imager').get(pname) > if not plugin_class: > raise WicError('Unknown plugin: %s' % pname) > @@ -201,17 +196,18 @@ def wic_list(args, scripts_path): > """ > Print the list of images or source plugins. > """ > - if len(args) < 1: > + if args.list_type is None: > return False > > - if args == ["images"]: > + if args.list_type == "images": > + > list_canned_images(scripts_path) > return True > - elif args == ["source-plugins"]: > + elif args.list_type == "source-plugins": > list_source_plugins() > return True > - elif len(args) == 2 and args[1] == "help": > - wks_file = args[0] > + elif len(args.help_for) == 1 and args.help_for[0] == 'help': > + wks_file = args.list_type > fullpath = find_canned_image(scripts_path, wks_file) > if not fullpath: > raise WicError("No image named %s found, exiting. " > @@ -224,6 +220,381 @@ def wic_list(args, scripts_path): > > return False > > + > +class Disk: > + def __init__(self, imagepath, native_sysroot, fstypes=('fat', > 'ext')): > + self.imagepath = imagepath > + self.native_sysroot = native_sysroot > + self.fstypes = fstypes > + self._partitions = None > + self._partimages = {} > + self._lsector_size = None > + self._psector_size = None > + self._ptable_format = None > + > + # find parted > + # read paths from $PATH environment variable > + # if it fails, use hardcoded paths > + pathlist = "/bin:/usr/bin:/usr/sbin:/sbin/" > + try: > + self.paths = os.environ['PATH'] + ":" + pathlist > + except KeyError: > + self.paths = pathlist > + > + if native_sysroot: > + for path in pathlist.split(':'): > + self.paths = "%s%s:%s" % (native_sysroot, path, > self.paths) + > + self.parted = find_executable("parted", self.paths) > + if not self.parted: > + raise WicError("Can't find executable parted") > + > + self.partitions = self.get_partitions() > + > + def __del__(self): > + for path in self._partimages.values(): > + os.unlink(path) > + > + def get_partitions(self): > + if self._partitions is None: > + self._partitions = OrderedDict() > + out = exec_cmd("%s -sm %s unit B print" % (self.parted, > self.imagepath)) > + parttype = namedtuple("Part", "pnum start end size > fstype") > + splitted = out.splitlines() > + # skip over possible errors in exec_cmd output > + try: > + idx =splitted.index("BYT;") > + except ValueError: > + raise WicError("Error getting partition information > from %s" % (self.parted)) > + lsector_size, psector_size, self._ptable_format = > splitted[idx + 1].split(":")[3:6] > + self._lsector_size = int(lsector_size) > + self._psector_size = int(psector_size) > + for line in splitted[idx + 2:]: > + pnum, start, end, size, fstype = line.split(':')[:5] > + partition = parttype(int(pnum), int(start[:-1]), > int(end[:-1]), > + int(size[:-1]), fstype) > + self._partitions[pnum] = partition > + > + return self._partitions > + > + def __getattr__(self, name): > + """Get path to the executable in a lazy way.""" > + if name in ("mdir", "mcopy", "mdel", "mdeltree", "sfdisk", > "e2fsck", > + "resize2fs", "mkswap", "mkdosfs", > "debugfs","blkid"): > + aname = "_%s" % name > + if aname not in self.__dict__: > + setattr(self, aname, find_executable(name, > self.paths)) > + if aname not in self.__dict__ or > self.__dict__[aname] is None: > + raise WicError("Can't find executable > '{}'".format(name)) > + return self.__dict__[aname] > + return self.__dict__[name] > + > + def _get_part_image(self, pnum): > + if pnum not in self.partitions: > + raise WicError("Partition %s is not in the image" % pnum) > + part = self.partitions[pnum] > + # check if fstype is supported > + for fstype in self.fstypes: > + if part.fstype.startswith(fstype): > + break > + else: > + raise WicError("Not supported fstype: > {}".format(part.fstype)) > + if pnum not in self._partimages: > + tmpf = tempfile.NamedTemporaryFile(prefix="wic-part") > + dst_fname = tmpf.name > + tmpf.close() > + sparse_copy(self.imagepath, dst_fname, skip=part.start, > length=part.size) > + self._partimages[pnum] = dst_fname > + > + return self._partimages[pnum] > + > + def _put_part_image(self, pnum): > + """Put partition image into partitioned image.""" > + sparse_copy(self._partimages[pnum], self.imagepath, > + seek=self.partitions[pnum].start) > + > + def dir(self, pnum, path): > + if pnum not in self.partitions: > + raise WicError("Partition %s is not in the image" % pnum) > + > + if self.partitions[pnum].fstype.startswith('ext'): > + return exec_cmd("{} {} -R 'ls -l > {}'".format(self.debugfs, > + > self._get_part_image(pnum), > + path), > as_shell=True) > + else: # fat > + return exec_cmd("{} -i {} ::{}".format(self.mdir, > + > self._get_part_image(pnum), > + path)) > + > + def copy(self, src, dest): > + """Copy partition image into wic image.""" > + pnum = dest.part if isinstance(src, str) else src.part > + > + if self.partitions[pnum].fstype.startswith('ext'): > + if isinstance(src, str): > + cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\ > + format(os.path.dirname(dest.path), src, > os.path.basename(src), > + self.debugfs, > self._get_part_image(pnum)) > + else: # copy from wic > + # run both dump and rdump to support both files and > directory > + cmd = "printf 'cd {}\ndump /{} {}\nrdump /{} {}\n' | > {} {}".\ > + format(os.path.dirname(src.path), src.path, > + dest, src.path, dest, self.debugfs, > + self._get_part_image(pnum)) > + else: # fat > + if isinstance(src, str): > + cmd = "{} -i {} -snop {} ::{}".format(self.mcopy, > + > self._get_part_image(pnum), > + src, dest.path) > + else: > + cmd = "{} -i {} -snop ::{} {}".format(self.mcopy, > + > self._get_part_image(pnum), > + src.path, dest) > + > + exec_cmd(cmd, as_shell=True) > + self._put_part_image(pnum) > + > + def remove_ext(self, pnum, path, recursive): > + """ > + Remove files/dirs and their contents from the partition. > + This only applies to ext* partition. > + """ > + abs_path = re.sub('\/\/+', '/', path) > + cmd = "{} {} -wR 'rm \"{}\"'".format(self.debugfs, > + > self._get_part_image(pnum), > + abs_path) > + out = exec_cmd(cmd , as_shell=True) > + for line in out.splitlines(): > + if line.startswith("rm:"): > + if "file is a directory" in line: > + if recursive: > + # loop through content and delete them one > by one if > + # flaged with -r > + subdirs = iter(self.dir(pnum, > abs_path).splitlines()) > + next(subdirs) > + for subdir in subdirs: > + dir = subdir.split(':')[1].split(" ", > 1)[1] > + if not dir == "." and not dir == "..": > + self.remove_ext(pnum, "%s/%s" % > (abs_path, dir), recursive) + > + rmdir_out = exec_cmd("{} {} -wR 'rmdir > \"{}\"'".format(self.debugfs, > + > self._get_part_image(pnum), > + > abs_path.rstrip('/')) > + , as_shell=True) > + > + for rmdir_line in rmdir_out.splitlines(): > + if "directory not empty" in rmdir_line: > + raise WicError("Could not complete > operation: \n%s \n" > + "use -r to remove > non-empty directory" % rmdir_line) > + if rmdir_line.startswith("rmdir:"): > + raise WicError("Could not complete > operation: \n%s " > + "\n%s" % (str(line), > rmdir_line)) + > + else: > + raise WicError("Could not complete operation: > \n%s " > + "\nUnable to remove %s" % > (str(line), abs_path)) + > + def remove(self, pnum, path, recursive): > + """Remove files/dirs from the partition.""" > + partimg = self._get_part_image(pnum) > + if self.partitions[pnum].fstype.startswith('ext'): > + self.remove_ext(pnum, path, recursive) > + > + else: # fat > + cmd = "{} -i {} ::{}".format(self.mdel, partimg, path) > + try: > + exec_cmd(cmd) > + except WicError as err: > + if "not found" in str(err) or "non empty" in > str(err): > + # mdel outputs 'File ... not found' or > 'directory .. non empty" > + # try to use mdeltree as path could be a > directory > + cmd = "{} -i {} ::{}".format(self.mdeltree, > + partimg, path) > + exec_cmd(cmd) > + else: > + raise err > + self._put_part_image(pnum) > + > + def write(self, target, expand): > + """Write disk image to the media or file.""" > + def write_sfdisk_script(outf, parts): > + for key, val in parts['partitiontable'].items(): > + if key in ("partitions", "device", "firstlba", > "lastlba"): > + continue > + if key == "id": > + key = "label-id" > + outf.write("{}: {}\n".format(key, val)) > + outf.write("\n") > + for part in parts['partitiontable']['partitions']: > + line = '' > + for name in ('attrs', 'name', 'size', 'type', > 'uuid'): > + if name == 'size' and part['type'] == 'f': > + # don't write size for extended partition > + continue > + val = part.get(name) > + if val: > + line += '{}={}, '.format(name, val) > + if line: > + line = line[:-2] # strip ', ' > + if part.get('bootable'): > + line += ' ,bootable' > + outf.write("{}\n".format(line)) > + outf.flush() > + > + def read_ptable(path): > + out = exec_cmd("{} -J {}".format(self.sfdisk, path)) > + return json.loads(out) > + > + def write_ptable(parts, target): > + with tempfile.NamedTemporaryFile(prefix="wic-sfdisk-", > mode='w') as outf: > + write_sfdisk_script(outf, parts) > + cmd = "{} --no-reread {} < {} ".format(self.sfdisk, > target, outf.name) > + exec_cmd(cmd, as_shell=True) > + > + if expand is None: > + sparse_copy(self.imagepath, target) > + else: > + # copy first sectors that may contain bootloader > + sparse_copy(self.imagepath, target, length=2048 * > self._lsector_size) + > + # copy source partition table to the target > + parts = read_ptable(self.imagepath) > + write_ptable(parts, target) > + > + # get size of unpartitioned space > + free = None > + for line in exec_cmd("{} -F {}".format(self.sfdisk, > target)).splitlines(): > + if line.startswith("Unpartitioned space ") and > line.endswith("sectors"): > + free = int(line.split()[-2]) > + # Align free space to a 2048 sector boundary. > YOCTO #12840. > + free = free - (free % 2048) > + if free is None: > + raise WicError("Can't get size of unpartitioned > space") + > + # calculate expanded partitions sizes > + sizes = {} > + num_auto_resize = 0 > + for num, part in > enumerate(parts['partitiontable']['partitions'], 1): > + if num in expand: > + if expand[num] != 0: # don't resize partition if > size is set to 0 > + sectors = expand[num] // self._lsector_size > + free -= sectors - part['size'] > + part['size'] = sectors > + sizes[num] = sectors > + elif part['type'] != 'f': > + sizes[num] = -1 > + num_auto_resize += 1 > + > + for num, part in > enumerate(parts['partitiontable']['partitions'], 1): > + if sizes.get(num) == -1: > + part['size'] += free // num_auto_resize > + > + # write resized partition table to the target > + write_ptable(parts, target) > + > + # read resized partition table > + parts = read_ptable(target) > + > + # copy partitions content > + for num, part in > enumerate(parts['partitiontable']['partitions'], 1): > + pnum = str(num) > + fstype = self.partitions[pnum].fstype > + > + # copy unchanged partition > + if part['size'] == self.partitions[pnum].size // > self._lsector_size: > + logger.info("copying unchanged partition > {}".format(pnum)) > + sparse_copy(self._get_part_image(pnum), target, > seek=part['start'] * self._lsector_size) > + continue > + > + # resize or re-create partitions > + if fstype.startswith('ext') or > fstype.startswith('fat') or \ > + fstype.startswith('linux-swap'): > + > + partfname = None > + with > tempfile.NamedTemporaryFile(prefix="wic-part{}-".format(pnum)) as > partf: > + partfname = partf.name > + > + if fstype.startswith('ext'): > + logger.info("resizing ext partition > {}".format(pnum)) > + partimg = self._get_part_image(pnum) > + sparse_copy(partimg, partfname) > + exec_cmd("{} -pf {}".format(self.e2fsck, > partfname)) > + exec_cmd("{} {} {}s".format(\ > + self.resize2fs, partfname, > part['size'])) > + elif fstype.startswith('fat'): > + logger.info("copying content of the fat > partition {}".format(pnum)) > + with > tempfile.TemporaryDirectory(prefix='wic-fatdir-') as tmpdir: > + # copy content to the temporary directory > + cmd = "{} -snompi {} :: > {}".format(self.mcopy, > + > self._get_part_image(pnum), > + > tmpdir) > + exec_cmd(cmd) > + # create new msdos partition > + label = part.get("name") > + label_str = "-n {}".format(label) if > label else '' + > + cmd = "{} {} -C {} > {}".format(self.mkdosfs, label_str, partfname, > + > part['size']) > + exec_cmd(cmd) > + # copy content from the temporary > directory to the new partition > + cmd = "{} -snompi {} {}/* > ::".format(self.mcopy, partfname, tmpdir) > + exec_cmd(cmd, as_shell=True) > + elif fstype.startswith('linux-swap'): > + logger.info("creating swap partition > {}".format(pnum)) > + label = part.get("name") > + label_str = "-L {}".format(label) if label > else '' > + out = exec_cmd("{} --probe > {}".format(self.blkid, self._get_part_image(pnum))) > + uuid = > out[out.index("UUID=\"")+6:out.index("UUID=\"")+42] > + uuid_str = "-U {}".format(uuid) if uuid else > '' > + with open(partfname, 'w') as sparse: > + os.ftruncate(sparse.fileno(), > part['size'] * self._lsector_size) > + exec_cmd("{} {} {} {}".format(self.mkswap, > label_str, uuid_str, partfname)) > + sparse_copy(partfname, target, > seek=part['start'] * self._lsector_size) > + os.unlink(partfname) > + elif part['type'] != 'f': > + logger.warning("skipping partition {}: > unsupported fstype {}".format(pnum, fstype)) + > +def wic_ls(args, native_sysroot): > + """List contents of partitioned image or vfat partition.""" > + disk = Disk(args.path.image, native_sysroot) > + if not args.path.part: > + if disk.partitions: > + print('Num Start End Size > Fstype') > + for part in disk.partitions.values(): > + print("{:2d} {:12d} {:12d} {:12d} {}".format(\ > + part.pnum, part.start, part.end, > + part.size, part.fstype)) > + else: > + path = args.path.path or '/' > + print(disk.dir(args.path.part, path)) > + > +def wic_cp(args, native_sysroot): > + """ > + Copy file or directory to/from the vfat/ext partition of > + partitioned image. > + """ > + if isinstance(args.dest, str): > + disk = Disk(args.src.image, native_sysroot) > + else: > + disk = Disk(args.dest.image, native_sysroot) > + disk.copy(args.src, args.dest) > + > + > +def wic_rm(args, native_sysroot): > + """ > + Remove files or directories from the vfat partition of > + partitioned image. > + """ > + disk = Disk(args.path.image, native_sysroot) > + disk.remove(args.path.part, args.path.path, > args.recursive_delete) + > +def wic_write(args, native_sysroot): > + """ > + Write image to a target device. > + """ > + disk = Disk(args.image, native_sysroot, ('fat', 'ext', > 'linux-swap')) > + disk.write(args.target, args.expand) > + > def find_canned(scripts_path, file_name): > """ > Find a file either by its path or by name in the canned files > dir. diff --git a/scripts/lib/wic/filemap.py > b/scripts/lib/wic/filemap.py index 080668e..4d9da28 100644 > --- a/scripts/lib/wic/filemap.py > +++ b/scripts/lib/wic/filemap.py > @@ -1,13 +1,8 @@ > +# > # Copyright (c) 2012 Intel, Inc. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License, > version 2, -# as published by the Free Software Foundation. > +# SPDX-License-Identifier: GPL-2.0-only > # > -# This program is distributed in the hope that it will be useful, but > -# WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > -# General Public License for more details. > > """ > This module implements python implements a way to get file block. > Two methods @@ -22,6 +17,7 @@ and returns an instance of the class. > # * Too many instance attributes (R0902) > # pylint: disable=R0902 > > +import errno > import os > import struct > import array > @@ -34,14 +30,23 @@ def get_block_size(file_obj): > Returns block size for file object 'file_obj'. Errors are > indicated by the 'IOError' exception. > """ > - > - from fcntl import ioctl > - import struct > - > # Get the block size of the host file-system for the image file > by calling # the FIGETBSZ ioctl (number 2). > - binary_data = ioctl(file_obj, 2, struct.pack('I', 0)) > - return struct.unpack('I', binary_data)[0] > + try: > + binary_data = fcntl.ioctl(file_obj, 2, struct.pack('I', 0)) > + bsize = struct.unpack('I', binary_data)[0] > + except OSError: > + bsize = None > + > + # If ioctl causes OSError or give bsize to zero failback to > os.fstat > + if not bsize: > + import os > + stat = os.fstat(file_obj.fileno()) > + if hasattr(stat, 'st_blksize'): > + bsize = stat.st_blksize > + else: > + raise IOError("Unable to determine block size") > + return bsize > > class ErrorNotSupp(Exception): > """ > @@ -137,15 +142,6 @@ class _FilemapBase(object): > > raise Error("the method is not implemented") > > - def block_is_unmapped(self, block): # pylint: disable=W0613,R0201 > - """ > - This method has has to be implemented by child classes. It > returns > - 'True' if block number 'block' of the image file is not > mapped (hole) > - and 'False' otherwise. > - """ > - > - raise Error("the method is not implemented") > - > def get_mapped_ranges(self, start, count): # pylint: > disable=W0613,R0201 """ > This method has has to be implemented by child classes. This > is a @@ -159,15 +155,6 @@ class _FilemapBase(object): > > raise Error("the method is not implemented") > > - def get_unmapped_ranges(self, start, count): # pylint: > disable=W0613,R0201 > - """ > - This method has has to be implemented by child classes. Just > like > - 'get_mapped_ranges()', but yields unmapped block ranges > instead > - (holes). > - """ > - > - raise Error("the method is not implemented") > - > > # The 'SEEK_HOLE' and 'SEEK_DATA' options of the file seek system > call _SEEK_DATA = 3 > @@ -185,9 +172,9 @@ def _lseek(file_obj, offset, whence): > except OSError as err: > # The 'lseek' system call returns the ENXIO if there is no > data or # hole starting from the specified offset. > - if err.errno == os.errno.ENXIO: > + if err.errno == errno.ENXIO: > return -1 > - elif err.errno == os.errno.EINVAL: > + elif err.errno == errno.EINVAL: > raise ErrorNotSupp("the kernel or file-system does not > support " "\"SEEK_HOLE\" and \"SEEK_DATA\"") > else: > @@ -228,7 +215,7 @@ class FilemapSeek(_FilemapBase): > try: > tmp_obj = tempfile.TemporaryFile("w+", dir=directory) > except IOError as err: > - raise ErrorNotSupp("cannot create a temporary in \"%s\": > %s" > + raise ErrorNotSupp("cannot create a temporary in \"%s\": > %s" \ % (directory, err)) > > try: > @@ -260,15 +247,10 @@ class FilemapSeek(_FilemapBase): > % (block, result)) > return result > > - def block_is_unmapped(self, block): > - """Refer the '_FilemapBase' class for the documentation.""" > - return not self.block_is_mapped(block) > - > def _get_ranges(self, start, count, whence1, whence2): > """ > - This function implements 'get_mapped_ranges()' and > - 'get_unmapped_ranges()' depending on what is passed in the > 'whence1' > - and 'whence2' arguments. > + This function implements 'get_mapped_ranges()' depending > + on what is passed in the 'whence1' and 'whence2' arguments. > """ > > assert whence1 != whence2 > @@ -298,12 +280,6 @@ class FilemapSeek(_FilemapBase): > % (start, count, start + count - 1)) > return self._get_ranges(start, count, _SEEK_DATA, _SEEK_HOLE) > > - def get_unmapped_ranges(self, start, count): > - """Refer the '_FilemapBase' class for the documentation.""" > - self._log.debug("FilemapSeek: get_unmapped_ranges(%d, > %d(%d))" > - % (start, count, start + count - 1)) > - return self._get_ranges(start, count, _SEEK_HOLE, _SEEK_DATA) > - > > # Below goes the FIEMAP ioctl implementation, which is not very > readable # because it deals with the rather complex FIEMAP ioctl. To > understand the @@ -390,12 +366,12 @@ class > FilemapFiemap(_FilemapBase): except IOError as err: > # Note, the FIEMAP ioctl is supported by the Linux > kernel starting # from version 2.6.28 (year 2008). > - if err.errno == os.errno.EOPNOTSUPP: > + if err.errno == errno.EOPNOTSUPP: > errstr = "FilemapFiemap: the FIEMAP ioctl is not > supported " \ "by the file-system" > self._log.debug(errstr) > raise ErrorNotSupp(errstr) > - if err.errno == os.errno.ENOTTY: > + if err.errno == errno.ENOTTY: > errstr = "FilemapFiemap: the FIEMAP ioctl is not > supported " \ "by the kernel" > self._log.debug(errstr) > @@ -417,10 +393,6 @@ class FilemapFiemap(_FilemapBase): > % (block, result)) > return result > > - def block_is_unmapped(self, block): > - """Refer the '_FilemapBase' class for the documentation.""" > - return not self.block_is_mapped(block) > - > def _unpack_fiemap_extent(self, index): > """ > Unpack a 'struct fiemap_extent' structure object number > 'index' from @@ -497,23 +469,28 @@ class FilemapFiemap(_FilemapBase): > % (first_prev, last_prev)) > yield (first_prev, last_prev) > > - def get_unmapped_ranges(self, start, count): > +class FilemapNobmap(_FilemapBase): > + """ > + This class is used when both the 'SEEK_DATA/HOLE' and FIEMAP are > not > + supported by the filesystem or kernel. > + """ > + > + def __init__(self, image, log=None): > """Refer the '_FilemapBase' class for the documentation.""" > - self._log.debug("FilemapFiemap: get_unmapped_ranges(%d, > %d(%d))" > - % (start, count, start + count - 1)) > - hole_first = start > - for first, last in self._do_get_mapped_ranges(start, count): > - if first > hole_first: > - self._log.debug("FilemapFiemap: yielding range (%d, > %d)" > - % (hole_first, first - 1)) > - yield (hole_first, first - 1) > > - hole_first = last + 1 > + # Call the base class constructor first > + _FilemapBase.__init__(self, image, log) > + self._log.debug("FilemapNobmap: initializing") > > - if hole_first < start + count: > - self._log.debug("FilemapFiemap: yielding range (%d, %d)" > - % (hole_first, start + count - 1)) > - yield (hole_first, start + count - 1) > + def block_is_mapped(self, block): > + """Refer the '_FilemapBase' class for the documentation.""" > + return True > + > + def get_mapped_ranges(self, start, count): > + """Refer the '_FilemapBase' class for the documentation.""" > + self._log.debug("FilemapNobmap: get_mapped_ranges(%d, > %d(%d))" > + % (start, count, start + count - 1)) > + yield (start, start + count -1) > > def filemap(image, log=None): > """ > @@ -528,26 +505,56 @@ def filemap(image, log=None): > try: > return FilemapFiemap(image, log) > except ErrorNotSupp: > - return FilemapSeek(image, log) > + try: > + return FilemapSeek(image, log) > + except ErrorNotSupp: > + return FilemapNobmap(image, log) > > -def sparse_copy(src_fname, dst_fname, offset=0, skip=0): > - """Efficiently copy sparse file to or into another file.""" > - fmap = filemap(src_fname) > +def sparse_copy(src_fname, dst_fname, skip=0, seek=0, > + length=0, api=None): > + """ > + Efficiently copy sparse file to or into another file. > + > + src_fname: path to source file > + dst_fname: path to destination file > + skip: skip N bytes at thestart of src > + seek: seek N bytes from the start of dst > + length: read N bytes from src and write them to dst > + api: FilemapFiemap or FilemapSeek object > + """ > + if not api: > + api = filemap > + fmap = api(src_fname) > try: > dst_file = open(dst_fname, 'r+b') > except IOError: > dst_file = open(dst_fname, 'wb') > - dst_file.truncate(os.path.getsize(src_fname)) > + if length: > + dst_size = length + seek > + else: > + dst_size = os.path.getsize(src_fname) + seek - skip > + dst_file.truncate(dst_size) > > + written = 0 > for first, last in fmap.get_mapped_ranges(0, fmap.blocks_cnt): > start = first * fmap.block_size > end = (last + 1) * fmap.block_size > > + if skip >= end: > + continue > + > if start < skip < end: > - fmap._f_image.seek(skip, os.SEEK_SET) > - else: > - fmap._f_image.seek(start, os.SEEK_SET) > - dst_file.seek(offset + start, os.SEEK_SET) > + start = skip > + > + fmap._f_image.seek(start, os.SEEK_SET) > + > + written += start - skip - written > + if length and written >= length: > + dst_file.seek(seek + length, os.SEEK_SET) > + dst_file.close() > + return > + > + dst_file.seek(seek + start - skip, os.SEEK_SET) > > chunk_size = 1024 * 1024 > to_read = end - start > @@ -556,7 +563,14 @@ def sparse_copy(src_fname, dst_fname, offset=0, > skip=0): while read < to_read: > if read + chunk_size > to_read: > chunk_size = to_read - read > - chunk = fmap._f_image.read(chunk_size) > + size = chunk_size > + if length and written + size > length: > + size = length - written > + chunk = fmap._f_image.read(size) > dst_file.write(chunk) > - read += chunk_size > + read += size > + written += size > + if written == length: > + dst_file.close() > + return > dst_file.close() > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > index 148da89..bd3a2b9 100644 > --- a/scripts/lib/wic/help.py > +++ b/scripts/lib/wic/help.py > @@ -1,21 +1,6 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > -# > # Copyright (c) 2013, Intel Corporation. > -# All rights reserved. > -# > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > # > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This module implements some basic help invocation functions along > @@ -56,7 +41,7 @@ def wic_help(args, usage_str, subcommands): > """ > Subcommand help dispatcher. > """ > - if len(args) == 1 or not display_help(args[1], subcommands): > + if args.help_topic == None or not display_help(args.help_topic, > subcommands): print(usage_str) > > > @@ -82,19 +67,20 @@ def invoke_subcommand(args, parser, > main_command_usage, subcommands): Dispatch to subcommand handler > borrowed from combo-layer. Should use argparse, but has to work in > 2.6. """ > - if not args: > + if not args.command: > logger.error("No subcommand specified, exiting") > parser.print_help() > return 1 > - elif args[0] == "help": > + elif args.command == "help": > wic_help(args, main_command_usage, subcommands) > - elif args[0] not in subcommands: > - logger.error("Unsupported subcommand %s, exiting\n", args[0]) > + elif args.command not in subcommands: > + logger.error("Unsupported subcommand %s, exiting\n", > args.command) parser.print_help() > return 1 > else: > - usage = subcommands.get(args[0], subcommand_error)[1] > - subcommands.get(args[0], subcommand_error)[0](args[1:], > usage) > + subcmd = subcommands.get(args.command, subcommand_error) > + usage = subcmd[1] > + subcmd[0](args, usage) > > > ## > @@ -130,10 +116,10 @@ wic_create_usage = """ > Create a new OpenEmbedded image > > usage: wic create [-o | --outdir > ] > - [-i | --infile ] > [-e | --image-name] [-s, --skip-build-check] [-D, > --debug] [-r, --rootfs-dir] [-b, --bootimg-dir] > [-k, --kernel-dir] [-n, --native-sysroot] [-f, > --build-rootfs] > + [-c, --compress-with] [-m, --bmap] > > This command creates an OpenEmbedded image based on the 'OE > kickstart commands' found in the . > @@ -154,7 +140,7 @@ SYNOPSIS > [-e | --image-name] [-s, --skip-build-check] [-D, --debug] > [-r, --rootfs-dir] [-b, --bootimg-dir] > [-k, --kernel-dir] [-n, --native-sysroot] [-f, > --build-rootfs] > - [-c, --compress-with] [-m, --bmap] > + [-c, --compress-with] [-m, --bmap] [--no-fstab-update] > > DESCRIPTION > This command creates an OpenEmbedded image based on the 'OE > @@ -226,6 +212,11 @@ DESCRIPTION > > The -m option is used to produce .bmap file for the image. This > file can be used to flash image using bmaptool utility. > + > + The --no-fstab-update option is used to doesn't change fstab > file. When > + using this option the final fstab file will be same that in > rootfs and > + wic doesn't update file, e.g adding a new mount point. User can > control > + the fstab file content in base-files recipe. > """ > > wic_list_usage = """ > @@ -283,6 +274,243 @@ DESCRIPTION > details. > """ > > +wic_ls_usage = """ > + > + List content of a partitioned image > + > + usage: wic ls [:[]] [--native-sysroot > ] + > + This command outputs either list of image partitions or directory > contents > + of vfat and ext* partitions. > + > + See 'wic help ls' for more detailed instructions. > + > +""" > + > +wic_ls_help = """ > + > +NAME > + wic ls - List contents of partitioned image or partition > + > +SYNOPSIS > + wic ls > + wic ls : > + wic ls : > + wic ls : --native-sysroot > + > +DESCRIPTION > + This command lists either partitions of the image or directory > contents > + of vfat or ext* partitions. > + > + The first form it lists partitions of the image. > + For example: > + $ wic ls > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic > + Num Start End Size Fstype > + 1 1048576 24438783 23390208 fat16 > + 2 25165824 50315263 25149440 ext4 > + > + Second and third form list directory content of the partition: > + $ wic ls > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 > + Volume in drive : is boot > + Volume Serial Number is 2DF2-5F02 > + Directory for ::/ > + > + efi 2017-05-11 10:54 > + startup nsh 26 2017-05-11 10:54 > + vmlinuz 6922288 2017-05-11 10:54 > + 3 files 6 922 314 bytes > + 15 818 752 bytes free > + > + > + $ wic ls > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/EFI/boot/ > + Volume in drive : is boot > + Volume Serial Number is 2DF2-5F02 > + Directory for ::/EFI/boot > + > + . 2017-05-11 10:54 > + .. 2017-05-11 10:54 > + grub cfg 679 2017-05-11 10:54 > + bootx64 efi 571392 2017-05-11 10:54 > + 4 files 572 071 bytes > + 15 818 752 bytes free > + > + The -n option is used to specify the path to the native sysroot > + containing the tools(parted and mtools) to use. > + > +""" > + > +wic_cp_usage = """ > + > + Copy files and directories to/from the vfat or ext* partition > + > + usage: wic cp [--native-sysroot ] > + > + source/destination image in format :[] > + > + This command copies files or directories either > + - from local to vfat or ext* partitions of partitioned image > + - from vfat or ext* partitions of partitioned image to local > + > + See 'wic help cp' for more detailed instructions. > + > +""" > + > +wic_cp_help = """ > + > +NAME > + wic cp - copy files and directories to/from the vfat or ext* > partitions + > +SYNOPSIS > + wic cp : > + wic cp : > + wic cp : > + wic cp : --native-sysroot > + > +DESCRIPTION > + This command copies files or directories either > + - from local to vfat or ext* partitions of partitioned image > + - from vfat or ext* partitions of partitioned image to local > + > + The first form of it copies file or directory to the root > directory of > + the partition: > + $ wic cp test.wks > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 > + $ wic ls > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 > + Volume in drive : is boot > + Volume Serial Number is DB4C-FD4C > + Directory for ::/ > + > + efi 2017-05-24 18:15 > + loader 2017-05-24 18:15 > + startup nsh 26 2017-05-24 18:15 > + vmlinuz 6926384 2017-05-24 18:15 > + test wks 628 2017-05-24 21:22 > + 5 files 6 927 038 bytes > + 15 677 440 bytes free > + > + The second form of the command copies file or directory to the > specified directory > + on the partition: > + $ wic cp test > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/ > + $ wic ls > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/ > + Volume in drive : is boot > + Volume Serial Number is DB4C-FD4C > + Directory for ::/efi > + > + . 2017-05-24 18:15 > + .. 2017-05-24 18:15 > + boot 2017-05-24 18:15 > + test 2017-05-24 21:27 > + 4 files 0 bytes > + 15 675 392 bytes free > + > + The third form of the command copies file or directory from the > specified directory > + on the partition to local: > + $ wic cp > tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/vmlinuz > test + > + The -n option is used to specify the path to the native sysroot > + containing the tools(parted and mtools) to use. > +""" > + > +wic_rm_usage = """ > + > + Remove files or directories from the vfat or ext* partitions > + > + usage: wic rm : [--native-sysroot ] > + > + This command removes files or directories from the vfat or ext* > partitions of > + the partitioned image. > + > + See 'wic help rm' for more detailed instructions. > + > +""" > + > +wic_rm_help = """ > + > +NAME > + wic rm - remove files or directories from the vfat or ext* > partitions + > +SYNOPSIS > + wic rm : > + wic rm : --native-sysroot > + wic rm -r : > + > +DESCRIPTION > + This command removes files or directories from the vfat or ext* > partition of the > + partitioned image: > + > + $ wic ls > ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 > + Volume in drive : is boot > + Volume Serial Number is 11D0-DE21 > + Directory for ::/ > + > + libcom32 c32 186500 2017-06-02 15:15 > + libutil c32 24148 2017-06-02 15:15 > + syslinux cfg 209 2017-06-02 15:15 > + vesamenu c32 27104 2017-06-02 15:15 > + vmlinuz 6926384 2017-06-02 15:15 > + 5 files 7 164 345 bytes > + 16 582 656 bytes free > + > + $ wic rm > ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/libutil.c32 > + > + $ wic ls > ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1 > + Volume in drive : is boot > + Volume Serial Number is 11D0-DE21 > + Directory for ::/ > + > + libcom32 c32 186500 2017-06-02 15:15 > + syslinux cfg 209 2017-06-02 15:15 > + vesamenu c32 27104 2017-06-02 15:15 > + vmlinuz 6926384 2017-06-02 15:15 > + 4 files 7 140 197 bytes > + 16 607 232 bytes free > + > + The -n option is used to specify the path to the native sysroot > + containing the tools(parted and mtools) to use. > + > + The -r option is used to remove directories and their contents > + recursively,this only applies to ext* partition. > +""" > + > +wic_write_usage = """ > + > + Write image to a device > + > + usage: wic write [--expand [rules]] > [--native-sysroot ] + > + This command writes partitioned image to a target device (USB > stick, SD card etc). + > + See 'wic help write' for more detailed instructions. > + > +""" > + > +wic_write_help = """ > + > +NAME > + wic write - write an image to a device > + > +SYNOPSIS > + wic write > + wic write --expand auto > + wic write --expand 1:100M,2:300M > + wic write --native-sysroot > + > +DESCRIPTION > + This command writes an image to a target device (USB stick, SD > card etc) + > + $ wic write > ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic > /dev/sdb + > + The --expand option is used to resize image partitions. > + --expand auto expands partitions to occupy all free space > available on the target device. > + It's also possible to specify expansion rules in a format > + :[,:...] for one or more > partitions. > + Specifying size 0 will keep partition unmodified. > + Note: Resizing boot partition can result in non-bootable image > for non-EFI images. It is > + recommended to use size 0 for boot partition to keep image > bootable. + > + The --native-sysroot option is used to specify the path to the > native sysroot > + containing the tools(parted, resize2fs) to use. > +""" > + > wic_plugins_help = """ > > NAME > @@ -308,7 +536,8 @@ DESCRIPTION > > Source plugins can also be implemented and added by external > layers - any plugins found in a scripts/lib/wic/plugins/source/ > - directory in an external layer will also be made available. > + or lib/wic/plugins/source/ directory in an external layer will > + also be made available. > > When the wic implementation needs to invoke a partition-specific > implementation, it looks for the plugin that has the same name as > @@ -346,6 +575,10 @@ DESCRIPTION > partition. In other words, it 'prepares' the final > partition image which will be incorporated into the disk image. > > + do_post_partition() > + Called after the partition is created. It is useful to add > post > + operations e.g. signing the partition. > + > do_configure_partition() > Called before do_prepare_partition(), typically used to > create custom configuration files for a partition, for > @@ -632,8 +865,11 @@ DESCRIPTION > Partitions with a specified will be > automatically mounted. This is achieved by wic adding entries to the > fstab during image generation. In order for a valid fstab to be > generated one of the > - --ondrive, --ondisk or --use-uuid partition options must be > used for > - each partition that specifies a mountpoint. > + --ondrive, --ondisk, --use-uuid or --use-label partition > options must > + be used for each partition that specifies a mountpoint. Note > that with > + --use-{uuid,label} and non-root , including swap, > the mount > + program must understand the PARTUUID or LABEL syntax. This > currently > + excludes the busybox versions of these applications. > > > The following are supported 'part' options: > @@ -687,6 +923,8 @@ DESCRIPTION > apply to partitions created using '--source rootfs' (see > --source above). Valid values are: > > + vfat > + msdos > ext2 > ext3 > ext4 > @@ -706,6 +944,14 @@ DESCRIPTION > label is already in use by another > filesystem, a new label is created for the partition. > > + --use-label: This option is specific to wic. It makes wic > to use the > + label in /etc/fstab to specify a partition. If > the > + --use-label and --use-uuid are used at the > same time, > + we prefer the uuid because it is less likely > to cause > + name confliction. We don't support using this > parameter > + on the root partition since it requires an > initramfs to > + parse this value and we do not currently > support that. + > --active: Marks the partition as active. > > --align (in KBytes): This option is specific to wic and says > @@ -719,11 +965,31 @@ DESCRIPTION > bootloaders. > > --exclude-path: This option is specific to wic. It excludes > the given > - absolute path from the resulting image. If > the path > + relative path from the resulting image. If > the path ends with a slash, only the content of the directory > is omitted, not the directory itself. This > option only has an effect with the rootfs source plugin. > > + --include-path: This option is specific to wic. It adds the > contents > + of the given path or a rootfs to the > resulting image. > + The option contains two fields, the origin > and the > + destination. When the origin is a rootfs, > it follows > + the same logic as the rootfs-dir argument > and the > + permissions and owners are kept. When the > origin is a > + path, it is relative to the directory in > which wic is > + running not the rootfs itself so use of an > absolute > + path is recommended, and the owner and > group is set to > + root:root. If no destination is given it is > + automatically set to the root of the > rootfs. This > + option only has an effect with the rootfs > source > + plugin. > + > + --change-directory: This option is specific to wic. It > changes to the > + given directory before copying the > files. This > + option is useful when we want to split > a rootfs in > + multiple partitions and we want to keep > the right > + permissions and usernames in all the > partitions. + > --extra-space: This option is specific to wic. It adds extra > space after the space filled by the content > of the partition. The final size can go > @@ -738,6 +1004,8 @@ DESCRIPTION > This option cannot be used with > --fixed-size option. > > + --part-name: This option is specific to wic. It specifies > name for GPT partitions. + > --part-type: This option is specific to wic. It specifies > partition type GUID for GPT partitions. > List of partition type GUIDS can be found here: > @@ -752,10 +1020,21 @@ DESCRIPTION > in bootloader configuration before running wic. In > this case .wks file can be generated or modified to set preconfigured > parition UUID using this option. > + --fsuuid: This option is specific to wic. It specifies > filesystem UUID. > + It's useful if preconfigured filesystem UUID is > added to kernel command line > + in bootloader configuration before running wic. > In this case .wks file can > + be generated or modified to set preconfigured > filesystem UUID using this option. + > --system-id: This option is specific to wic. It specifies > partition system id. It's useful for the harware that requires > non-default partition system ids. The parameter in one byte long hex > number either with 0x prefix or without it. > + --mkfs-extraopts: This option specifies extra options to > pass to mkfs utility. > + NOTE, that wic uses default options for > some filesystems, for example > + '-S 512' for mkfs.fat or '-F -i 8192' for > mkfs.ext. Those options will > + not take effect when --mkfs-extraopts is > used. This should be taken into > + account when using --mkfs-extraopts. > + > * bootloader > > This command allows the user to specify various bootloader > @@ -793,3 +1072,67 @@ DESCRIPTION > .wks files. > > """ > + > +wic_help_help = """ > +NAME > + wic help - display a help topic > + > +DESCRIPTION > + Specify a help topic to display it. Topics are shown above. > +""" > + > + > +wic_help = """ > +Creates a customized OpenEmbedded image. > + > +Usage: wic [--version] > + wic help [COMMAND or TOPIC] > + wic COMMAND [ARGS] > + > + usage 1: Returns the current version of Wic > + usage 2: Returns detailed help for a COMMAND or TOPIC > + usage 3: Executes COMMAND > + > + > +COMMAND: > + > + list - List available canned images and source plugins > + ls - List contents of partitioned image or partition > + rm - Remove files or directories from the vfat or ext* > partitions > + help - Show help for a wic COMMAND or TOPIC > + write - Write an image to a device > + cp - Copy files and directories to the vfat or ext* > partitions > + create - Create a new OpenEmbedded image > + > + > +TOPIC: > + overview - Presents an overall overview of Wic > + plugins - Presents an overview and API for Wic plugins > + kickstart - Presents a Wic kicstart file reference > + > + > +Examples: > + > + $ wic --version > + > + Returns the current version of Wic > + > + > + $ wic help cp > + > + Returns the SYNOPSIS and DESCRIPTION for the Wic "cp" command. > + > + > + $ wic list images > + > + Returns the list of canned images (i.e. *.wks files located in > + the /scripts/lib/wic/canned-wks directory. > + > + > + $ wic create mkefidisk -e core-image-minimal > + > + Creates an EFI disk image from artifacts used in a previous > + core-image-minimal build in standard BitBake locations > + (e.g. Cooked Mode). > + > +""" > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py > index a039300..3453d9c 100644 > --- a/scripts/lib/wic/ksparser.py > +++ b/scripts/lib/wic/ksparser.py > @@ -1,21 +1,8 @@ > -#!/usr/bin/env python -tt > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > +#!/usr/bin/env python3 > # > # Copyright (c) 2016 Intel, Inc. > # > -# This program is free software; you can redistribute it and/or > modify it -# under the terms of the GNU General Public License as > published by the Free -# Software Foundation; version 2 of the License > -# > -# This program is distributed in the hope that it will be useful, but > -# WITHOUT ANY WARRANTY; without even the implied warranty of > MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > General Public License -# for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA > 02111-1307, USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This module provides parser for kickstart format > @@ -28,14 +15,30 @@ > import os > import shlex > import logging > +import re > > from argparse import ArgumentParser, ArgumentError, ArgumentTypeError > > from wic.engine import find_canned > from wic.partition import Partition > +from wic.misc import get_bitbake_var > > logger = logging.getLogger('wic') > > +__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}") > + > +def expand_line(line): > + while True: > + m = __expand_var_regexp__.search(line) > + if not m: > + return line > + key = m.group()[2:-1] > + val = get_bitbake_var(key) > + if val is None: > + logger.warning("cannot expand variable %s" % key) > + return line > + line = line[:m.start()] + val + line[m.end():] > + > class KickStartError(Exception): > """Custom exception.""" > pass > @@ -48,26 +51,31 @@ class KickStartParser(ArgumentParser): > def error(self, message): > raise ArgumentError(None, message) > > -def sizetype(arg): > - """ > - Custom type for ArgumentParser > - Converts size string in [K|k|M|G] format into the integer > value > - """ > - if arg.isdigit(): > - return int(arg) * 1024 > - > - if not arg[:-1].isdigit(): > - raise ArgumentTypeError("Invalid size: %r" % arg) > +def sizetype(default): > + def f(arg): > + """ > + Custom type for ArgumentParser > + Converts size string in [K|k|M|G] format into the > integer value > + """ > + try: > + suffix = default > + size = int(arg) > + except ValueError: > + try: > + suffix = arg[-1:] > + size = int(arg[:-1]) > + except ValueError: > + raise ArgumentTypeError("Invalid size: %r" % arg) > > - size = int(arg[:-1]) > - if arg.endswith("k") or arg.endswith("K"): > - return size > - if arg.endswith("M"): > - return size * 1024 > - if arg.endswith("G"): > - return size * 1024 * 1024 > + if suffix == "k" or suffix == "K": > + return size > + if suffix == "M": > + return size * 1024 > + if suffix == "G": > + return size * 1024 * 1024 > > - raise ArgumentTypeError("Invalid size: %r" % arg) > + raise ArgumentTypeError("Invalid size: %r" % arg) > + return f > > def overheadtype(arg): > """ > @@ -114,7 +122,7 @@ def systemidtype(arg): > return arg > > class KickStart(): > - """"Kickstart parser implementation.""" > + """Kickstart parser implementation.""" > > DEFAULT_EXTRA_SPACE = 10*1024 > DEFAULT_OVERHEAD_FACTOR = 1.3 > @@ -133,30 +141,41 @@ class KickStart(): > part.add_argument('mountpoint', nargs='?') > part.add_argument('--active', action='store_true') > part.add_argument('--align', type=int) > + part.add_argument('--offset', type=sizetype("K")) > part.add_argument('--exclude-path', nargs='+') > - part.add_argument("--extra-space", type=sizetype) > + part.add_argument('--include-path', nargs='+', > action='append') > + part.add_argument('--change-directory') > + part.add_argument("--extra-space", type=sizetype("M")) > part.add_argument('--fsoptions', dest='fsopts') > - part.add_argument('--fstype') > + part.add_argument('--fstype', default='vfat', > + choices=('ext2', 'ext3', 'ext4', 'btrfs', > + 'squashfs', 'vfat', 'msdos', > 'swap')) > + part.add_argument('--mkfs-extraopts', default='') > part.add_argument('--label') > + part.add_argument('--use-label', action='store_true') > part.add_argument('--no-table', action='store_true') > part.add_argument('--ondisk', '--ondrive', dest='disk', > default='sda') part.add_argument("--overhead-factor", > type=overheadtype) > + part.add_argument('--part-name') > part.add_argument('--part-type') > part.add_argument('--rootfs-dir') > + part.add_argument('--type', default='primary', > + choices = ('primary', 'logical')) > > # --size and --fixed-size cannot be specified together; > options # ----extra-space and --overhead-factor should also raise a > parser # --error, but since nesting mutually exclusive groups does > not work, # ----extra-space/--overhead-factor are handled later > sizeexcl = part.add_mutually_exclusive_group() > - sizeexcl.add_argument('--size', type=sizetype, default=0) > - sizeexcl.add_argument('--fixed-size', type=sizetype, > default=0) > + sizeexcl.add_argument('--size', type=sizetype("M"), > default=0) > + sizeexcl.add_argument('--fixed-size', type=sizetype("M"), > default=0) > part.add_argument('--source') > part.add_argument('--sourceparams') > part.add_argument('--system-id', type=systemidtype) > part.add_argument('--use-uuid', action='store_true') > part.add_argument('--uuid') > + part.add_argument('--fsuuid') > > bootloader = subparsers.add_parser('bootloader') > bootloader.add_argument('--append') > @@ -184,6 +203,7 @@ class KickStart(): > line = line.strip() > lineno += 1 > if line and line[0] != '#': > + line = expand_line(line) > try: > line_args = shlex.split(line) > parsed = parser.parse_args(line_args) > @@ -191,6 +211,20 @@ class KickStart(): > raise KickStartError('%s:%d: %s' % \ > (confpath, lineno, err)) > if line.startswith('part'): > + # SquashFS does not support filesystem UUID > + if parsed.fstype == 'squashfs': > + if parsed.fsuuid: > + err = "%s:%d: SquashFS does not > support UUID" \ > + % (confpath, lineno) > + raise KickStartError(err) > + if parsed.label: > + err = "%s:%d: SquashFS does not > support LABEL" \ > + % (confpath, lineno) > + raise KickStartError(err) > + if parsed.use_label and not parsed.label: > + err = "%s:%d: Must set the label with > --label" \ > + % (confpath, lineno) > + raise KickStartError(err) > # using ArgumentParser one cannot easily > tell if option # was passed as argument, if said option has a default > # value; --overhead-factor/--extra-space > cannot be used @@ -219,6 +253,11 @@ class KickStart(): > elif line.startswith('bootloader'): > if not self.bootloader: > self.bootloader = parsed > + # Concatenate the strings set in APPEND > + append_var = get_bitbake_var("APPEND") > + if append_var: > + self.bootloader.append = ' > '.join(filter(None, \ > + > (self.bootloader.append, append_var))) else: > err = "%s:%d: more than one bootloader > specified" \ % (confpath, lineno) > diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/misc.py > similarity index 70% > rename from scripts/lib/wic/utils/misc.py > rename to scripts/lib/wic/misc.py > index c941112..91975ba 100644 > --- a/scripts/lib/wic/utils/misc.py > +++ b/scripts/lib/wic/misc.py > @@ -1,21 +1,7 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2013, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This module provides a place to collect various wic-related utils > @@ -29,12 +15,12 @@ > import logging > import os > import re > +import subprocess > > from collections import defaultdict > from distutils import spawn > > from wic import WicError > -from wic.utils import runner > > logger = logging.getLogger('wic') > > @@ -43,6 +29,9 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools", > "grub-mkimage": "grub-efi", > "isohybrid": "syslinux", > "mcopy": "mtools", > + "mdel" : "mtools", > + "mdeltree" : "mtools", > + "mdir" : "mtools", > "mkdosfs": "dosfstools", > "mkisofs": "cdrtools", > "mkfs.btrfs": "btrfs-tools", > @@ -52,14 +41,48 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools", > "mkfs.vfat": "dosfstools", > "mksquashfs": "squashfs-tools", > "mkswap": "util-linux", > - "mmd": "syslinux", > + "mmd": "mtools", > "parted": "parted", > "sfdisk": "util-linux", > "sgdisk": "gptfdisk", > - "syslinux": "syslinux" > + "syslinux": "syslinux", > + "tar": "tar" > } > > -def _exec_cmd(cmd_and_args, as_shell=False, catch=3): > +def runtool(cmdln_or_args): > + """ wrapper for most of the subprocess calls > + input: > + cmdln_or_args: can be both args and cmdln str (shell=True) > + return: > + rc, output > + """ > + if isinstance(cmdln_or_args, list): > + cmd = cmdln_or_args[0] > + shell = False > + else: > + import shlex > + cmd = shlex.split(cmdln_or_args)[0] > + shell = True > + > + sout = subprocess.PIPE > + serr = subprocess.STDOUT > + > + try: > + process = subprocess.Popen(cmdln_or_args, stdout=sout, > + stderr=serr, shell=shell) > + sout, serr = process.communicate() > + # combine stdout and stderr, filter None out and decode > + out = ''.join([out.decode('utf-8') for out in [sout, serr] > if out]) > + except OSError as err: > + if err.errno == 2: > + # [Errno 2] No such file or directory > + raise WicError('Cannot run command: %s, lost > dependency?' % cmd) > + else: > + raise # relay > + > + return process.returncode, out > + > +def _exec_cmd(cmd_and_args, as_shell=False): > """ > Execute command, catching stderr, stdout > > @@ -70,9 +93,9 @@ def _exec_cmd(cmd_and_args, as_shell=False, > catch=3): logger.debug(args) > > if as_shell: > - ret, out = runner.runtool(cmd_and_args, catch) > + ret, out = runtool(cmd_and_args) > else: > - ret, out = runner.runtool(args, catch) > + ret, out = runtool(args) > out = out.strip() > if ret != 0: > raise WicError("_exec_cmd: %s returned '%s' instead of > 0\noutput: %s" % \ @@ -84,14 +107,23 @@ def _exec_cmd(cmd_and_args, > as_shell=False, catch=3): return ret, out > > > -def exec_cmd(cmd_and_args, as_shell=False, catch=3): > +def exec_cmd(cmd_and_args, as_shell=False): > """ > Execute command, return output > """ > - return _exec_cmd(cmd_and_args, as_shell, catch)[1] > + return _exec_cmd(cmd_and_args, as_shell)[1] > + > +def find_executable(cmd, paths): > + recipe = cmd > + if recipe in NATIVE_RECIPES: > + recipe = NATIVE_RECIPES[recipe] > + provided = get_bitbake_var("ASSUME_PROVIDED") > + if provided and "%s-native" % recipe in provided: > + return True > > + return spawn.find_executable(cmd, paths) > > -def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, > pseudo=""): +def exec_native_cmd(cmd_and_args, native_sysroot, > pseudo=""): """ > Execute native command, catching stderr, stdout > > @@ -106,19 +138,16 @@ def exec_native_cmd(cmd_and_args, > native_sysroot, catch=3, pseudo=""): if pseudo: > cmd_and_args = pseudo + cmd_and_args > > - wtools_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", > "wic-tools") > + native_paths = "%s/sbin:%s/usr/sbin:%s/usr/bin" % \ > + (native_sysroot, native_sysroot, native_sysroot) > > - native_paths = \ > - > "%s/sbin:%s/usr/sbin:%s/usr/bin:%s/sbin:%s/usr/sbin:%s/usr/bin" % \ > - (wtools_sysroot, wtools_sysroot, wtools_sysroot, > - native_sysroot, native_sysroot, native_sysroot) > native_cmd_and_args = "export PATH=%s:$PATH;%s" % \ > - (native_paths, cmd_and_args) > + (native_paths, cmd_and_args) > logger.debug("exec_native_cmd: %s", native_cmd_and_args) > > # If the command isn't in the native sysroot say we failed. > - if spawn.find_executable(args[0], native_paths): > - ret, out = _exec_cmd(native_cmd_and_args, True, catch) > + if find_executable(args[0], native_paths): > + ret, out = _exec_cmd(native_cmd_and_args, True) > else: > ret = 127 > out = "can't find native executable %s in %s" % (args[0], > native_paths) @@ -131,8 +160,8 @@ def exec_native_cmd(cmd_and_args, > native_sysroot, catch=3, pseudo=""): "was not found (see details > above).\n\n" % prog recipe = NATIVE_RECIPES.get(prog) > if recipe: > - msg += "Please bake it with 'bitbake %s-native' "\ > - "and try again.\n" % recipe > + msg += "Please make sure wic-tools have %s-native in its > DEPENDS, "\ > + "build it with 'bitbake wic-tools' and try > again.\n" % recipe else: > msg += "Wic failed to find a recipe to build native %s. > Please "\ "file a bug against wic.\n" % prog > @@ -153,7 +182,7 @@ class BitbakeVars(defaultdict): > self.default_image = None > self.vars_dir = None > > - def _parse_line(self, line, image, > matcher=re.compile(r"^(\w+)=(.+)")): > + def _parse_line(self, line, image, > matcher=re.compile(r"^([a-zA-Z0-9\-_+./~]+)=(.*)")): """ > Parse one line from bitbake -e output or from .env file. > Put result key-value pair into the storage. > diff --git a/scripts/lib/wic/partition.py > b/scripts/lib/wic/partition.py index 8e32afc..85eb15c 100644 > --- a/scripts/lib/wic/partition.py > +++ b/scripts/lib/wic/partition.py > @@ -1,21 +1,7 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2013-2016 Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This module provides the OpenEmbedded partition object definitions. > @@ -26,10 +12,10 @@ > > import logging > import os > -import tempfile > +import uuid > > from wic import WicError > -from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var > +from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var > from wic.pluginbase import PluginMgr > > logger = logging.getLogger('wic') > @@ -44,13 +30,19 @@ class Partition(): > self.device = None > self.extra_space = args.extra_space > self.exclude_path = args.exclude_path > + self.include_path = args.include_path > + self.change_directory = args.change_directory > self.fsopts = args.fsopts > self.fstype = args.fstype > self.label = args.label > + self.use_label = args.use_label > + self.mkfs_extraopts = args.mkfs_extraopts > self.mountpoint = args.mountpoint > self.no_table = args.no_table > self.num = None > + self.offset = args.offset > self.overhead_factor = args.overhead_factor > + self.part_name = args.part_name > self.part_type = args.part_type > self.rootfs_dir = args.rootfs_dir > self.size = args.size > @@ -60,10 +52,11 @@ class Partition(): > self.system_id = args.system_id > self.use_uuid = args.use_uuid > self.uuid = args.uuid > + self.fsuuid = args.fsuuid > + self.type = args.type > > self.lineno = lineno > self.source_file = "" > - self.sourceparams_dict = {} > > def get_extra_block_count(self, current_blocks): > """ > @@ -136,22 +129,24 @@ class Partition(): > "specify a non-zero > --size/--fixed-size for that " "partition." % self.mountpoint) > > - if self.fstype and self.fstype == "swap": > + if self.fstype == "swap": > self.prepare_swap_partition(cr_workdir, oe_builddir, > native_sysroot) > self.source_file = "%s/fs.%s" % (cr_workdir, > self.fstype) > - elif self.fstype: > + else: > + if self.fstype == 'squashfs': > + raise WicError("It's not possible to create > empty squashfs " > + "partition '%s'" % > (self.mountpoint)) + > rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label, > self.lineno, > self.fstype) if os.path.isfile(rootfs): > os.remove(rootfs) > - for prefix in ("ext", "btrfs", "vfat", "squashfs"): > - if self.fstype.startswith(prefix): > - method = getattr(self, > - "prepare_empty_partition_" > + prefix) > - method(rootfs, oe_builddir, native_sysroot) > - self.source_file = rootfs > - break > + > + prefix = "ext" if self.fstype.startswith("ext") else > self.fstype > + method = getattr(self, "prepare_empty_partition_" + > prefix) > + method(rootfs, oe_builddir, native_sysroot) > + self.source_file = rootfs > return > > plugins = PluginMgr.get_plugins('source') > @@ -168,7 +163,7 @@ class Partition(): > # Split sourceparams string of the form > key1=val1[,key2=val2,...] # into a dict. Also accepts valueless keys > i.e. without = splitted = self.sourceparams.split(',') > - srcparams_dict = dict(par.split('=') for par in splitted > if par) > + srcparams_dict = dict(par.split('=', 1) for par in > splitted if par) > plugin = PluginMgr.get_plugins('source')[self.source] > plugin.do_configure_partition(self, srcparams_dict, creator, > @@ -180,6 +175,9 @@ class Partition(): > plugin.do_prepare_partition(self, srcparams_dict, creator, > cr_workdir, oe_builddir, > bootimg_dir, kernel_dir, rootfs_dir, native_sysroot) > + plugin.do_post_partition(self, srcparams_dict, creator, > + cr_workdir, oe_builddir, > bootimg_dir, > + kernel_dir, rootfs_dir, > native_sysroot) > # further processing required Partition.size to be an > integer, make # sure that it is one > @@ -193,74 +191,57 @@ class Partition(): > "larger (%d kB) than its allowed size %d > kB" % (self.mountpoint, self.size, self.fixed_size)) > > - def prepare_rootfs_from_fs_image(self, cr_workdir, oe_builddir, > - rootfs_dir): > - """ > - Handle an already-created partition e.g. xxx.ext3 > - """ > - rootfs = oe_builddir > - du_cmd = "du -Lbks %s" % rootfs > - out = exec_cmd(du_cmd) > - rootfs_size = out.split()[0] > - > - self.size = int(rootfs_size) > - self.source_file = rootfs > - > def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir, > - native_sysroot): > + native_sysroot, real_rootfs = True, > pseudo_dir = None): """ > Prepare content for a rootfs partition i.e. create a > partition and fill it from a /rootfs dir. > > - Currently handles ext2/3/4, btrfs and vfat. > + Currently handles ext2/3/4, btrfs, vfat and squashfs. > """ > p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" % > native_sysroot) > - p_localstatedir = os.environ.get("PSEUDO_LOCALSTATEDIR", > - "%s/../pseudo" % rootfs_dir) > - p_passwd = os.environ.get("PSEUDO_PASSWD", rootfs_dir) > - p_nosymlinkexp = os.environ.get("PSEUDO_NOSYMLINKEXP", "1") > - pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix > - pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % p_localstatedir > - pseudo += "export PSEUDO_PASSWD=%s;" % p_passwd > - pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp > - pseudo += "%s " % get_bitbake_var("FAKEROOTCMD") > + if (pseudo_dir): > + pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix > + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir > + pseudo += "export PSEUDO_PASSWD=%s;" % rootfs_dir > + pseudo += "export PSEUDO_NOSYMLINKEXP=1;" > + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD") > + else: > + pseudo = None > > rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label, > self.lineno, self.fstype) > if os.path.isfile(rootfs): > os.remove(rootfs) > > - if not self.fstype: > - raise WicError("File system for partition %s not > specified in " > - "kickstart, use --fstype option" % > self.mountpoint) - > - # Get rootfs size from bitbake variable if it's not set in > .ks file > - if not self.size: > - # Bitbake variable ROOTFS_SIZE is calculated in > - # Image._get_rootfs_size method from meta/lib/oe/image.py > - # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, > - # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE > + if not self.size and real_rootfs: > + # The rootfs size is not set in .ks file so try to get it > + # from bitbake variable > rsize_bb = get_bitbake_var('ROOTFS_SIZE') > - if rsize_bb: > - logger.warning('overhead-factor was specified, but > size was not,' > - ' so bitbake variables will be used > for the size.' > - ' In this case both > IMAGE_OVERHEAD_FACTOR and ' > - '--overhead-factor will be applied') > + rdir = get_bitbake_var('IMAGE_ROOTFS') > + if rsize_bb and rdir == rootfs_dir: > + # Bitbake variable ROOTFS_SIZE is calculated in > + # Image._get_rootfs_size method from > meta/lib/oe/image.py > + # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, > + # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE > self.size = int(round(float(rsize_bb))) > - > - for prefix in ("ext", "btrfs", "vfat", "squashfs"): > - if self.fstype.startswith(prefix): > - method = getattr(self, "prepare_rootfs_" + prefix) > - method(rootfs, oe_builddir, rootfs_dir, > native_sysroot, pseudo) - > - self.source_file = rootfs > - > - # get the rootfs size in the right units for > kickstart (kB) > - du_cmd = "du -Lbks %s" % rootfs > + else: > + # Bitbake variable ROOTFS_SIZE is not defined so > compute it > + # from the rootfs_dir size using the same logic > found in > + # get_rootfs_size() from meta/classes/image.bbclass > + du_cmd = "du -ks %s" % rootfs_dir > out = exec_cmd(du_cmd) > self.size = int(out.split()[0]) > > - break > + prefix = "ext" if self.fstype.startswith("ext") else > self.fstype > + method = getattr(self, "prepare_rootfs_" + prefix) > + method(rootfs, oe_builddir, rootfs_dir, native_sysroot, > pseudo) > + self.source_file = rootfs > + > + # get the rootfs size in the right units for kickstart (kB) > + du_cmd = "du -Lbks %s" % rootfs > + out = exec_cmd(du_cmd) > + self.size = int(out.split()[0]) > > def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir, > native_sysroot, pseudo): > @@ -276,25 +257,23 @@ class Partition(): > with open(rootfs, 'w') as sparse: > os.ftruncate(sparse.fileno(), rootfs_size * 1024) > > - extra_imagecmd = "-i 8192" > + extraopts = self.mkfs_extraopts or "-F -i 8192" > > label_str = "" > if self.label: > label_str = "-L %s" % self.label > > - mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \ > - (self.fstype, extra_imagecmd, rootfs, label_str, > rootfs_dir) > + mkfs_cmd = "mkfs.%s %s %s %s -U %s -d %s" % \ > + (self.fstype, extraopts, rootfs, label_str, self.fsuuid, > rootfs_dir) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) > > - mkfs_cmd = "fsck.%s -fy %s" % (self.fstype, rootfs) > + mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs) > exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) > > def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir, > native_sysroot, pseudo): > """ > Prepare content for a btrfs rootfs partition. > - > - Currently handles ext2/3/4 and btrfs. > """ > du_cmd = "du -ks %s" % rootfs_dir > out = exec_cmd(du_cmd) > @@ -309,14 +288,15 @@ class Partition(): > if self.label: > label_str = "-L %s" % self.label > > - mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \ > - (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, > rootfs) > + mkfs_cmd = "mkfs.%s -b %d -r %s %s %s -U %s %s" % \ > + (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, > + self.mkfs_extraopts, self.fsuuid, rootfs) > exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) > > - def prepare_rootfs_vfat(self, rootfs, oe_builddir, rootfs_dir, > - native_sysroot, pseudo): > + def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir, > + native_sysroot, pseudo): > """ > - Prepare content for a vfat rootfs partition. > + Prepare content for a msdos/vfat rootfs partition. > """ > du_cmd = "du -bks %s" % rootfs_dir > out = exec_cmd(du_cmd) > @@ -328,7 +308,15 @@ class Partition(): > if self.label: > label_str = "-n %s" % self.label > > - dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, > rootfs, rootfs_size) > + size_str = "" > + if self.fstype == 'msdos': > + size_str = "-F 16" # FAT 16 > + > + extraopts = self.mkfs_extraopts or '-S 512' > + > + dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \ > + (label_str, self.fsuuid, size_str, extraopts, > rootfs, > + rootfs_size) > exec_native_cmd(dosfs_cmd, native_sysroot) > > mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir) > @@ -337,13 +325,16 @@ class Partition(): > chmod_cmd = "chmod 644 %s" % rootfs > exec_cmd(chmod_cmd) > > + prepare_rootfs_vfat = prepare_rootfs_msdos > + > def prepare_rootfs_squashfs(self, rootfs, oe_builddir, > rootfs_dir, native_sysroot, pseudo): > """ > Prepare content for a squashfs rootfs partition. > """ > - squashfs_cmd = "mksquashfs %s %s -noappend" % \ > - (rootfs_dir, rootfs) > + extraopts = self.mkfs_extraopts or '-noappend' > + squashfs_cmd = "mksquashfs %s %s %s" % \ > + (rootfs_dir, rootfs, extraopts) > exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo) > > def prepare_empty_partition_ext(self, rootfs, oe_builddir, > @@ -355,14 +346,14 @@ class Partition(): > with open(rootfs, 'w') as sparse: > os.ftruncate(sparse.fileno(), size * 1024) > > - extra_imagecmd = "-i 8192" > + extraopts = self.mkfs_extraopts or "-i 8192" > > label_str = "" > if self.label: > label_str = "-L %s" % self.label > > - mkfs_cmd = "mkfs.%s -F %s %s %s" % \ > - (self.fstype, extra_imagecmd, label_str, rootfs) > + mkfs_cmd = "mkfs.%s -F %s %s -U %s %s" % \ > + (self.fstype, extraopts, label_str, self.fsuuid, rootfs) > exec_native_cmd(mkfs_cmd, native_sysroot) > > def prepare_empty_partition_btrfs(self, rootfs, oe_builddir, > @@ -378,12 +369,13 @@ class Partition(): > if self.label: > label_str = "-L %s" % self.label > > - mkfs_cmd = "mkfs.%s -b %d %s %s" % \ > - (self.fstype, self.size * 1024, label_str, rootfs) > + mkfs_cmd = "mkfs.%s -b %d %s -U %s %s %s" % \ > + (self.fstype, self.size * 1024, label_str, > self.fsuuid, > + self.mkfs_extraopts, rootfs) > exec_native_cmd(mkfs_cmd, native_sysroot) > > - def prepare_empty_partition_vfat(self, rootfs, oe_builddir, > - native_sysroot): > + def prepare_empty_partition_msdos(self, rootfs, oe_builddir, > + native_sysroot): > """ > Prepare an empty vfat partition. > """ > @@ -393,40 +385,22 @@ class Partition(): > if self.label: > label_str = "-n %s" % self.label > > - dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, > rootfs, blocks) > - exec_native_cmd(dosfs_cmd, native_sysroot) > + size_str = "" > + if self.fstype == 'msdos': > + size_str = "-F 16" # FAT 16 > > - chmod_cmd = "chmod 644 %s" % rootfs > - exec_cmd(chmod_cmd) > + extraopts = self.mkfs_extraopts or '-S 512' > > - def prepare_empty_partition_squashfs(self, cr_workdir, > oe_builddir, > - native_sysroot): > - """ > - Prepare an empty squashfs partition. > - """ > - logger.warning("Creating of an empty squashfs %s partition > was attempted. " > - "Proceeding as requested.", self.mountpoint) > - > - path = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype) > - if os.path.isfile(path): > - os.remove(path) > + dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \ > + (label_str, self.fsuuid, extraopts, size_str, > rootfs, > + blocks) > > - # it is not possible to create a squashfs without source > data, > - # thus prepare an empty temp dir that is used as source > - tmpdir = tempfile.mkdtemp() > - > - squashfs_cmd = "mksquashfs %s %s -noappend" % \ > - (tmpdir, path) > - exec_native_cmd(squashfs_cmd, native_sysroot) > - > - os.rmdir(tmpdir) > + exec_native_cmd(dosfs_cmd, native_sysroot) > > - # get the rootfs size in the right units for kickstart (kB) > - du_cmd = "du -Lbks %s" % path > - out = exec_cmd(du_cmd) > - fs_size = out.split()[0] > + chmod_cmd = "chmod 644 %s" % rootfs > + exec_cmd(chmod_cmd) > > - self.size = int(fs_size) > + prepare_empty_partition_vfat = prepare_empty_partition_msdos > > def prepare_swap_partition(self, cr_workdir, oe_builddir, > native_sysroot): """ > @@ -437,9 +411,9 @@ class Partition(): > with open(path, 'w') as sparse: > os.ftruncate(sparse.fileno(), self.size * 1024) > > - import uuid > label_str = "" > if self.label: > label_str = "-L %s" % self.label > - mkswap_cmd = "mkswap %s -U %s %s" % (label_str, > str(uuid.uuid1()), path) + > + mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid, > path) exec_native_cmd(mkswap_cmd, native_sysroot) > diff --git a/scripts/lib/wic/pluginbase.py > b/scripts/lib/wic/pluginbase.py index fb3d179..d9b4e57 100644 > --- a/scripts/lib/wic/pluginbase.py > +++ b/scripts/lib/wic/pluginbase.py > @@ -1,19 +1,9 @@ > -#!/usr/bin/env python -tt > +#!/usr/bin/env python3 > # > # Copyright (c) 2011 Intel, Inc. > # > -# This program is free software; you can redistribute it and/or > modify it -# under the terms of the GNU General Public License as > published by the Free -# Software Foundation; version 2 of the License > +# SPDX-License-Identifier: GPL-2.0-only > # > -# This program is distributed in the hope that it will be useful, but > -# WITHOUT ANY WARRANTY; without even the implied warranty of > MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > General Public License -# for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA > 02111-1307, USA. > __all__ = ['ImagerPlugin', 'SourcePlugin'] > > @@ -24,11 +14,11 @@ from collections import defaultdict > from importlib.machinery import SourceFileLoader > > from wic import WicError > -from wic.utils.misc import get_bitbake_var > +from wic.misc import get_bitbake_var > > PLUGIN_TYPES = ["imager", "source"] > > -SCRIPTS_PLUGIN_DIR = "scripts/lib/wic/plugins" > +SCRIPTS_PLUGIN_DIR = ["scripts/lib/wic/plugins", "lib/wic/plugins"] > > logger = logging.getLogger('wic') > > @@ -48,10 +38,11 @@ class PluginMgr: > cls._plugin_dirs = > [os.path.join(os.path.dirname(__file__), 'plugins')] layers = > get_bitbake_var("BBLAYERS") or '' for layer_path in layers.split(): > - path = os.path.join(layer_path, SCRIPTS_PLUGIN_DIR) > - path = os.path.abspath(os.path.expanduser(path)) > - if path not in cls._plugin_dirs and > os.path.isdir(path): > - cls._plugin_dirs.insert(0, path) > + for script_plugin_dir in SCRIPTS_PLUGIN_DIR: > + path = os.path.join(layer_path, > script_plugin_dir) > + path = os.path.abspath(os.path.expanduser(path)) > + if path not in cls._plugin_dirs and > os.path.isdir(path): > + cls._plugin_dirs.insert(0, path) > > if ptype not in PLUGINS: > # load all ptype plugins > @@ -138,3 +129,12 @@ class SourcePlugin(metaclass=PluginMeta): > """ > logger.debug("SourcePlugin: do_prepare_partition: part: %s", > part) > + @classmethod > + def do_post_partition(cls, part, source_params, creator, > cr_workdir, > + oe_builddir, bootimg_dir, kernel_dir, > rootfs_dir, > + native_sysroot): > + """ > + Called after the partition is created. It is useful to add > post > + operations e.g. security signing the partition. > + """ > + logger.debug("SourcePlugin: do_post_partition: part: %s", > part) diff --git a/scripts/lib/wic/plugins/imager/direct.py > b/scripts/lib/wic/plugins/imager/direct.py index 7d38ab3..2f01999 > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > +++ b/scripts/lib/wic/plugins/imager/direct.py > @@ -1,21 +1,7 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2013, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This implements the 'direct' imager plugin class for 'wic' > @@ -26,17 +12,20 @@ > > import logging > import os > +import random > import shutil > import tempfile > import uuid > > from time import strftime > > +from oe.path import copyhardlinktree > + > from wic import WicError > from wic.filemap import sparse_copy > from wic.ksparser import KickStart, KickStartError > from wic.pluginbase import PluginMgr, ImagerPlugin > -from wic.utils.misc import get_bitbake_var, exec_cmd, exec_native_cmd > +from wic.misc import get_bitbake_var, exec_cmd, exec_native_cmd > > logger = logging.getLogger('wic') > > @@ -68,6 +57,8 @@ class DirectPlugin(ImagerPlugin): > self.outdir = options.outdir > self.compressor = options.compressor > self.bmap = options.bmap > + self.no_fstab_update = options.no_fstab_update > + self.original_fstab = None > > self.name = "%s-%s" % > (os.path.splitext(os.path.basename(wks_file))[0], > strftime("%Y%m%d%H%M")) @@ -113,26 +104,38 @@ class > DirectPlugin(ImagerPlugin): > with open(fstab_path) as fstab: > fstab_lines = fstab.readlines() > + self.original_fstab = fstab_lines.copy() > > if self._update_fstab(fstab_lines, self.parts): > - shutil.copyfile(fstab_path, fstab_path + ".orig") > - > with open(fstab_path, "w") as fstab: > fstab.writelines(fstab_lines) > - > - return fstab_path > + else: > + self.original_fstab = None > > def _update_fstab(self, fstab_lines, parts): > """Assume partition order same as in wks""" > updated = False > for part in parts: > if not part.realnum or not part.mountpoint \ > - or part.mountpoint in ("/", "/boot"): > + or part.mountpoint == "/": > continue > > - # mmc device partitions are named mmcblk0p1, mmcblk0p2.. > - prefix = 'p' if part.disk.startswith('mmcblk') else '' > - device_name = "/dev/%s%s%d" % (part.disk, prefix, > part.realnum) > + if part.use_uuid: > + if part.fsuuid: > + # FAT UUID is different from others > + if len(part.fsuuid) == 10: > + device_name = "UUID=%s-%s" % \ > + (part.fsuuid[2:6], > part.fsuuid[6:]) > + else: > + device_name = "UUID=%s" % part.fsuuid > + else: > + device_name = "PARTUUID=%s" % part.uuid > + elif part.use_label: > + device_name = "LABEL=%s" % part.label > + else: > + # mmc device partitions are named mmcblk0p1, > mmcblk0p2.. > + prefix = 'p' if part.disk.startswith('mmcblk') else > '' > + device_name = "/dev/%s%s%d" % (part.disk, prefix, > part.realnum) > opts = part.fsopts if part.fsopts else "defaults" > line = "\t".join([device_name, part.mountpoint, > part.fstype, @@ -156,7 +159,8 @@ class DirectPlugin(ImagerPlugin): > filesystems from the artifacts directly and combine them into > a partitioned image. > """ > - fstab_path = > self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR")) > + if not self.no_fstab_update: > + self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR")) > > for part in self.parts: > # get rootfs size from bitbake variable if it's not set > in .ks file @@ -173,10 +177,6 @@ class DirectPlugin(ImagerPlugin): > part.size = int(round(float(rsize_bb))) > > self._image.prepare(self) > - > - if fstab_path: > - shutil.move(fstab_path + ".orig", fstab_path) > - > self._image.layout_partitions() > self._image.create() > > @@ -205,8 +205,10 @@ class DirectPlugin(ImagerPlugin): > # Generate .bmap > if self.bmap: > logger.debug("Generating bmap file for %s", disk_name) > - exec_native_cmd("bmaptool create %s -o %s.bmap" % > (full_path, full_path), > - self.native_sysroot) > + python = os.path.join(self.native_sysroot, > 'usr/bin/python3-native/python3') > + bmaptool = os.path.join(self.native_sysroot, > 'usr/bin/bmaptool') > + exec_native_cmd("%s %s create %s -o %s.bmap" % \ > + (python, bmaptool, full_path, > full_path), self.native_sysroot) # Compress the image > if self.compressor: > logger.debug("Compressing disk %s with %s", disk_name, > self.compressor) @@ -233,7 +235,8 @@ class DirectPlugin(ImagerPlugin): > suffix = ':' > else: > suffix = '["%s"]:' % (part.mountpoint or part.label) > - msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20), > part.rootfs_dir) > + rootdir = part.rootfs_dir > + msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20), rootdir) > > msg += ' BOOTIMG_DIR: %s\n' % > self.bootimg_dir msg += ' KERNEL_DIR: %s\n' % > self.kernel_dir @@ -270,6 +273,12 @@ class DirectPlugin(ImagerPlugin): > if os.path.isfile(path): > shutil.move(path, os.path.join(self.outdir, fname)) > > + #Restore original fstab > + if self.original_fstab: > + fstab_path = self.rootfs_dir.get("ROOTFS_DIR") + > "/etc/fstab" > + with open(fstab_path, "w") as fstab: > + fstab.writelines(self.original_fstab) > + > # remove work directory > shutil.rmtree(self.workdir, ignore_errors=True) > > @@ -291,18 +300,23 @@ class PartitionedImage(): > self.path = path # Path to the image file > self.numpart = 0 # Number of allocated partitions > self.realpart = 0 # Number of partitions in the partition > table > + self.primary_part_num = 0 # Number of primary partitions > (msdos) > + self.extendedpart = 0 # Create extended partition > before this logical partition (msdos) > + self.extended_size_sec = 0 # Size of exteded partition > (msdos) > + self.logical_part_cnt = 0 # Number of total logical > paritions (msdos) self.offset = 0 # Offset of next partition (in > sectors) self.min_size = 0 # Minimum required disk size to fit > # all partitions (in bytes) > self.ptable_format = ptable_format # Partition table format > # Disk system identifier > - self.identifier = int.from_bytes(os.urandom(4), 'little') > + self.identifier = random.SystemRandom().randint(1, > 0xffffffff) > self.partitions = partitions > self.partimages = [] > # Size of a sector used in calculations > self.sector_size = SECTOR_SIZE > self.native_sysroot = native_sysroot > + num_real_partitions = len([p for p in self.partitions if not > p.no_table]) > # calculate the real partition number, accounting for > partitions not # in the partition table and logical partitions > @@ -312,18 +326,23 @@ class PartitionedImage(): > part.realnum = 0 > else: > realnum += 1 > - if self.ptable_format == 'msdos' and realnum > 3: > + if self.ptable_format == 'msdos' and realnum > 3 and > num_real_partitions > 4: part.realnum = realnum + 1 > continue > part.realnum = realnum > > - # generate parition UUIDs > + # generate parition and filesystem UUIDs > for part in self.partitions: > if not part.uuid and part.use_uuid: > if self.ptable_format == 'gpt': > part.uuid = str(uuid.uuid4()) > else: # msdos partition table > - part.uuid = '%0x-%02d' % (self.identifier, > part.realnum) > + part.uuid = '%08x-%02d' % (self.identifier, > part.realnum) > + if not part.fsuuid: > + if part.fstype == 'vfat' or part.fstype == 'msdos': > + part.fsuuid = '0x' + > str(uuid.uuid4())[:8].upper() > + else: > + part.fsuuid = str(uuid.uuid4()) > > def prepare(self, imager): > """Prepare an image. Call prepare method of all image > partitions.""" @@ -352,6 +371,10 @@ class PartitionedImage(): > for num in range(len(self.partitions)): > part = self.partitions[num] > > + if self.ptable_format == 'msdos' and part.part_name: > + raise WicError("setting custom partition name is not > " \ > + "implemented for msdos partitions") > + > if self.ptable_format == 'msdos' and part.part_type: > # The --part-type can also be implemented for MBR > partitions, # in which case it would map to the 1-byte "partition > type" @@ -373,12 +396,16 @@ class PartitionedImage(): > # Skip one sector required for the partitioning > scheme overhead self.offset += overhead > > - if self.realpart > 3 and num_real_partitions > 4: > + if self.ptable_format == "msdos": > + if self.primary_part_num > 3 or \ > + (self.extendedpart == 0 and self.primary_part_num > >= 3 and num_real_partitions > 4): > + part.type = 'logical' > # Reserve a sector for EBR for every logical > partition # before alignment is performed. > - if self.ptable_format == "msdos": > - self.offset += 1 > + if part.type == 'logical': > + self.offset += 2 > > + align_sectors = 0 > if part.align: > # If not first partition and we do have alignment > set we need # to align the partition. > @@ -401,21 +428,43 @@ class PartitionedImage(): > # increase the offset so we actually start the > partition on right alignment self.offset += align_sectors > > + if part.offset is not None: > + offset = (part.offset * 1024) // self.sector_size > + > + if offset * self.sector_size != part.offset * 1024: > + raise WicError("Could not place %s%s at offset > %dK with sector size %d" % (part.disk, self.numpart, part.offset, > self.sector_size)) + > + delta = offset - self.offset > + if delta < 0: > + raise WicError("Could not place %s%s at offset > %dK: next free sector is %d (delta: %d)" % (part.disk, self.numpart, > part.offset, self.offset, delta)) + > + logger.debug("Skipping %d sectors to place %s%s at > offset %dK", > + delta, part.disk, self.numpart, > part.offset) + > + self.offset = offset > + > part.start = self.offset > self.offset += part.size_sec > > - part.type = 'primary' > if not part.no_table: > part.num = self.realpart > else: > part.num = 0 > > - if self.ptable_format == "msdos": > - # only count the partitions that are in partition > table > - if num_real_partitions > 4: > - if self.realpart > 3: > - part.type = 'logical' > - part.num = self.realpart + 1 > + if self.ptable_format == "msdos" and not part.no_table: > + if part.type == 'logical': > + self.logical_part_cnt += 1 > + part.num = self.logical_part_cnt + 4 > + if self.extendedpart == 0: > + # Create extended partition as a primary > partition > + self.primary_part_num += 1 > + self.extendedpart = part.num > + else: > + self.extended_size_sec += align_sectors > + self.extended_size_sec += part.size_sec + 2 > + else: > + self.primary_part_num += 1 > + part.num = self.primary_part_num > > logger.debug("Assigned %s to %s%d, sectors range %d-%d > size %d " "sectors (%d bytes).", part.mountpoint, part.disk, > @@ -465,7 +514,7 @@ class PartitionedImage(): > if part.num == 0: > continue > > - if self.ptable_format == "msdos" and part.num == 5: > + if self.ptable_format == "msdos" and part.num == > self.extendedpart: # Create an extended partition (note: extended > # partition is described in MBR and contains all > # logical partitions). The logical partitions save a > @@ -478,8 +527,8 @@ class PartitionedImage(): > # add a sector at the back, so that there is enough > # room for all logical partitions. > self._create_partition(self.path, "extended", > - None, part.start - 1, > - self.offset - part.start + 1) > + None, part.start - 2, > + self.extended_size_sec) > > if part.fstype == "swap": > parted_fs_type = "linux-swap" > @@ -487,8 +536,8 @@ class PartitionedImage(): > parted_fs_type = "fat32" > elif part.fstype == "msdos": > parted_fs_type = "fat16" > - elif part.fstype == "ontrackdm6aux3": > - parted_fs_type = "ontrackdm6aux3" > + if not part.system_id: > + part.system_id = '0x6' # FAT16 > else: > # Type for ext2/ext3/ext4/btrfs > parted_fs_type = "ext2" > @@ -505,6 +554,13 @@ class PartitionedImage(): > self._create_partition(self.path, part.type, > parted_fs_type, part.start, > part.size_sec) > + if part.part_name: > + logger.debug("partition %d: set name to %s", > + part.num, part.part_name) > + exec_native_cmd("sgdisk --change-name=%d:%s %s" % \ > + (part.num, part.part_name, > + self.path), > self.native_sysroot) + > if part.part_type: > logger.debug("partition %d: set type UID to %s", > part.num, part.part_type) > @@ -538,21 +594,8 @@ class PartitionedImage(): > (self.path, part.num, > part.system_id), self.native_sysroot) > > - # Parted defaults to enabling the lba flag for fat16 > partitions, > - # which causes compatibility issues with some firmware > (and really > - # isn't necessary). > - if parted_fs_type == "fat16": > - if self.ptable_format == 'msdos': > - logger.debug("Disable 'lba' flag for partition > '%s' on disk '%s'", > - part.num, self.path) > - exec_native_cmd("parted -s %s set %d lba off" % \ > - (self.path, part.num), > - self.native_sysroot) > - > def cleanup(self): > - # remove partition images > - for image in set(self.partimages): > - os.remove(image) > + pass > > def assemble(self): > logger.debug("Installing partitions") > @@ -561,7 +604,7 @@ class PartitionedImage(): > source = part.source_file > if source: > # install source_file contents into a partition > - sparse_copy(source, self.path, part.start * > self.sector_size) > + sparse_copy(source, self.path, seek=part.start * > self.sector_size) > logger.debug("Installed %s in partition %d, sectors > %d-%d, " "size %d sectors", source, part.num, part.start, > diff --git a/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py > b/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py new file mode > 100644 index 0000000..5bd7390 > --- /dev/null > +++ b/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py > @@ -0,0 +1,213 @@ > +# > +# This program is free software; you can redistribute it and/or > modify +# it under the terms of the GNU General Public License > version 2 as +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > along +# with this program; if not, write to the Free Software > Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# > +# DESCRIPTION > +# This implements the 'bootimg-biosplusefi' source plugin class for > 'wic' +# > +# AUTHORS > +# William Bourque > + > +import types > + > +from wic.pluginbase import SourcePlugin > +from importlib.machinery import SourceFileLoader > + > +class BootimgBiosPlusEFIPlugin(SourcePlugin): > + """ > + Create MBR + EFI boot partition > + > + This plugin creates a boot partition that contains both > + legacy BIOS and EFI content. It will be able to boot from both. > + This is useful when managing PC fleet with some older machines > + without EFI support. > + > + Note it is possible to create an image that can boot from both > + legacy BIOS and EFI by defining two partitions : one with arg > + --source bootimg-efi and another one with --source > bootimg-pcbios. > + However, this method has the obvious downside that it requires > TWO > + partitions to be created on the storage device. > + Both partitions will also be marked as "bootable" which does not > work on > + most BIOS, has BIOS often uses the "bootable" flag to determine > + what to boot. If you have such a BIOS, you need to manually > remove the > + "bootable" flag from the EFI partition for the drive to be > bootable. > + Having two partitions also seems to confuse wic : the content of > + the first partition will be duplicated into the second, even > though it > + will not be used at all. > + > + Also, unlike "isoimage-isohybrid" that also does BIOS and EFI, > this plugin > + allows you to have more than only a single rootfs partitions and > does > + not turn the rootfs into an initramfs RAM image. > + > + This plugin is made to put everything into a single /boot > partition so it > + does not have the limitations listed above. > + > + The plugin is made so it does tries not to reimplement what's > already > + been done in other plugins; as such it imports "bootimg-pcbios" > + and "bootimg-efi". > + Plugin "bootimg-pcbios" is used to generate legacy BIOS boot. > + Plugin "bootimg-efi" is used to generate the UEFI boot. Note > that it > + requires a --sourceparams argument to know which loader to use; > refer > + to "bootimg-efi" code/documentation for the list of loader. > + > + Imports are handled with "SourceFileLoader" from importlib as it > is > + otherwise very difficult to import module that has hyphen "-" in > their > + filename. > + The SourcePlugin() methods used in the plugins (do_install_disk, > + do_configure_partition, do_prepare_partition) are then called on > both, > + beginning by "bootimg-efi". > + > + Plugin options, such as "--sourceparams" can still be passed to a > + plugin, as long they does not cause issue in the other plugin. > + > + Example wic configuration: > + part /boot --source bootimg-biosplusefi > --sourceparams="loader=grub-efi"\\ > + --ondisk sda --label os_boot --active --align 1024 > --use-uuid > + """ > + > + name = 'bootimg-biosplusefi' > + > + __PCBIOS_MODULE_NAME = "bootimg-pcbios" > + __EFI_MODULE_NAME = "bootimg-efi" > + > + __imgEFIObj = None > + __imgBiosObj = None > + > + @classmethod > + def __init__(cls): > + """ > + Constructor (init) > + """ > + > + # XXX > + # For some reasons, __init__ constructor is never called. > + # Something to do with how pluginbase works? > + cls.__instanciateSubClasses() > + > + @classmethod > + def __instanciateSubClasses(cls): > + """ > + > + """ > + > + # Import bootimg-pcbios (class name "BootimgPcbiosPlugin") > + modulePath = > os.path.join(os.path.dirname(os.path.realpath(__file__)), > + cls.__PCBIOS_MODULE_NAME + ".py") > + loader = SourceFileLoader(cls.__PCBIOS_MODULE_NAME, > modulePath) > + mod = types.ModuleType(loader.name) > + loader.exec_module(mod) > + cls.__imgBiosObj = mod.BootimgPcbiosPlugin() > + > + # Import bootimg-efi (class name "BootimgEFIPlugin") > + modulePath = > os.path.join(os.path.dirname(os.path.realpath(__file__)), > + cls.__EFI_MODULE_NAME + ".py") > + loader = SourceFileLoader(cls.__EFI_MODULE_NAME, modulePath) > + mod = types.ModuleType(loader.name) > + loader.exec_module(mod) > + cls.__imgEFIObj = mod.BootimgEFIPlugin() > + > + @classmethod > + def do_install_disk(cls, disk, disk_name, creator, workdir, > oe_builddir, > + bootimg_dir, kernel_dir, native_sysroot): > + """ > + Called after all partitions have been prepared and assembled > into a > + disk image. > + """ > + > + if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ): > + cls.__instanciateSubClasses() > + > + cls.__imgEFIObj.do_install_disk( > + disk, > + disk_name, > + creator, > + workdir, > + oe_builddir, > + bootimg_dir, > + kernel_dir, > + native_sysroot) > + > + cls.__imgBiosObj.do_install_disk( > + disk, > + disk_name, > + creator, > + workdir, > + oe_builddir, > + bootimg_dir, > + kernel_dir, > + native_sysroot) > + > + @classmethod > + def do_configure_partition(cls, part, source_params, creator, > cr_workdir, > + oe_builddir, bootimg_dir, kernel_dir, > + native_sysroot): > + """ > + Called before do_prepare_partition() > + """ > + > + if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ): > + cls.__instanciateSubClasses() > + > + cls.__imgEFIObj.do_configure_partition( > + part, > + source_params, > + creator, > + cr_workdir, > + oe_builddir, > + bootimg_dir, > + kernel_dir, > + native_sysroot) > + > + cls.__imgBiosObj.do_configure_partition( > + part, > + source_params, > + creator, > + cr_workdir, > + oe_builddir, > + bootimg_dir, > + kernel_dir, > + native_sysroot) > + > + @classmethod > + def do_prepare_partition(cls, part, source_params, creator, > cr_workdir, > + oe_builddir, bootimg_dir, kernel_dir, > + rootfs_dir, native_sysroot): > + """ > + Called to do the actual content population for a partition > i.e. it > + 'prepares' the partition to be incorporated into the image. > + """ > + > + if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ): > + cls.__instanciateSubClasses() > + > + cls.__imgEFIObj.do_prepare_partition( > + part, > + source_params, > + creator, > + cr_workdir, > + oe_builddir, > + bootimg_dir, > + kernel_dir, > + rootfs_dir, > + native_sysroot) > + > + cls.__imgBiosObj.do_prepare_partition( > + part, > + source_params, > + creator, > + cr_workdir, > + oe_builddir, > + bootimg_dir, > + kernel_dir, > + rootfs_dir, > + native_sysroot) > diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py > b/scripts/lib/wic/plugins/source/bootimg-efi.py index > 9879cb9..2cfdc10 100644 --- > a/scripts/lib/wic/plugins/source/bootimg-efi.py +++ > b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -1,21 +1,7 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2014, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This implements the 'bootimg-efi' source plugin class for 'wic' > @@ -31,8 +17,8 @@ import shutil > from wic import WicError > from wic.engine import get_custom_config > from wic.pluginbase import SourcePlugin > -from wic.utils.misc import (exec_cmd, exec_native_cmd, > get_bitbake_var, > - BOOTDD_EXTRA_SPACE) > +from wic.misc import (exec_cmd, exec_native_cmd, > + get_bitbake_var, BOOTDD_EXTRA_SPACE) > > logger = logging.getLogger('wic') > > @@ -45,7 +31,7 @@ class BootimgEFIPlugin(SourcePlugin): > name = 'bootimg-efi' > > @classmethod > - def do_configure_grubefi(cls, creator, cr_workdir): > + def do_configure_grubefi(cls, hdddir, creator, cr_workdir, > source_params): """ > Create loader-specific (grub-efi) config > """ > @@ -62,20 +48,52 @@ class BootimgEFIPlugin(SourcePlugin): > raise WicError("configfile is specified but failed > to " "get it from %s." % configfile) > > + initrd = source_params.get('initrd') > + > + if initrd: > + bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > + if not bootimg_dir: > + raise WicError("Couldn't find DEPLOY_DIR_IMAGE, > exiting") + > + initrds = initrd.split(';') > + for rd in initrds: > + cp_cmd = "cp %s/%s %s" % (bootimg_dir, rd, hdddir) > + exec_cmd(cp_cmd, True) > + else: > + logger.debug("Ignoring missing initrd") > + > if not custom_cfg: > # Create grub configuration using parameters from wks > file bootloader = creator.ks.bootloader > + title = source_params.get('title') > > grubefi_conf = "" > grubefi_conf += "serial --unit=0 --speed=115200 --word=8 > --parity=no --stop=1\n" grubefi_conf += "default=boot\n" > grubefi_conf += "timeout=%s\n" % bootloader.timeout > - grubefi_conf += "menuentry 'boot'{\n" > + grubefi_conf += "menuentry '%s'{\n" % (title if title > else "boot") + > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) + > + label = source_params.get('label') > + label_conf = "root=%s" % creator.rootdev > + if label: > + label_conf = "LABEL=%s" % label > > - kernel = "/bzImage" > + grubefi_conf += "linux /%s %s rootwait %s\n" \ > + % (kernel, label_conf, bootloader.append) > + > + if initrd: > + initrds = initrd.split(';') > + grubefi_conf += "initrd" > + for rd in initrds: > + grubefi_conf += " /%s" % rd > + grubefi_conf += "\n" > > - grubefi_conf += "linux %s root=%s rootwait %s\n" \ > - % (kernel, creator.rootdev, bootloader.append) > grubefi_conf += "}\n" > > logger.debug("Writing grubefi config > %s/hdd/boot/EFI/BOOT/grub.cfg", @@ -109,8 +127,10 @@ class > BootimgEFIPlugin(SourcePlugin): if not bootimg_dir: > raise WicError("Couldn't find DEPLOY_DIR_IMAGE, > exiting") > - cp_cmd = "cp %s/%s %s" % (bootimg_dir, initrd, hdddir) > - exec_cmd(cp_cmd, True) > + initrds = initrd.split(';') > + for rd in initrds: > + cp_cmd = "cp %s/%s %s" % (bootimg_dir, rd, hdddir) > + exec_cmd(cp_cmd, True) > else: > logger.debug("Ignoring missing initrd") > > @@ -135,16 +155,30 @@ class BootimgEFIPlugin(SourcePlugin): > > if not custom_cfg: > # Create systemd-boot configuration using parameters > from wks file > - kernel = "/bzImage" > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) + > + title = source_params.get('title') > > boot_conf = "" > - boot_conf += "title boot\n" > - boot_conf += "linux %s\n" % kernel > - boot_conf += "options LABEL=Boot root=%s %s\n" % \ > - (creator.rootdev, bootloader.append) > + boot_conf += "title %s\n" % (title if title else "boot") > + boot_conf += "linux /%s\n" % kernel > + > + label = source_params.get('label') > + label_conf = "LABEL=Boot root=%s" % creator.rootdev > + if label: > + label_conf = "LABEL=%s" % label > + > + boot_conf += "options %s %s\n" % \ > + (label_conf, bootloader.append) > > if initrd: > - boot_conf += "initrd /%s\n" % initrd > + initrds = initrd.split(';') > + for rd in initrds: > + boot_conf += "initrd /%s\n" % rd > > logger.debug("Writing systemd-boot config " > "%s/hdd/boot/loader/entries/boot.conf", > cr_workdir) @@ -167,7 +201,7 @@ class BootimgEFIPlugin(SourcePlugin): > > try: > if source_params['loader'] == 'grub-efi': > - cls.do_configure_grubefi(creator, cr_workdir) > + cls.do_configure_grubefi(hdddir, creator, > cr_workdir, source_params) elif source_params['loader'] == > 'systemd-boot': cls.do_configure_systemdboot(hdddir, creator, > cr_workdir, source_params) else: > @@ -194,8 +228,14 @@ class BootimgEFIPlugin(SourcePlugin): > > hdddir = "%s/hdd/boot" % cr_workdir > > - install_cmd = "install -m 0644 %s/bzImage %s/bzImage" % \ > - (staging_kernel_dir, hdddir) > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) + > + install_cmd = "install -m 0644 %s/%s %s/%s" % \ > + (staging_kernel_dir, kernel, hdddir, kernel) > exec_cmd(install_cmd) > > > @@ -240,7 +280,10 @@ class BootimgEFIPlugin(SourcePlugin): > # dosfs image, created by mkdosfs > bootimg = "%s/boot.img" % cr_workdir > > - dosfs_cmd = "mkdosfs -n efi -C %s %d" % (bootimg, blocks) > + label = part.label if part.label else "ESP" > + > + dosfs_cmd = "mkdosfs -n %s -i %s -C %s %d" % \ > + (label, part.fsuuid, bootimg, blocks) > exec_native_cmd(dosfs_cmd, native_sysroot) > > mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) > diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py > b/scripts/lib/wic/plugins/source/bootimg-partition.py index > 13fddbd..138986a 100644 --- > a/scripts/lib/wic/plugins/source/bootimg-partition.py +++ > b/scripts/lib/wic/plugins/source/bootimg-partition.py @@ -1,18 +1,5 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This implements the 'bootimg-partition' source plugin class for > @@ -30,8 +17,9 @@ import re > from glob import glob > > from wic import WicError > +from wic.engine import get_custom_config > from wic.pluginbase import SourcePlugin > -from wic.utils.misc import exec_cmd, get_bitbake_var > +from wic.misc import exec_cmd, get_bitbake_var > > logger = logging.getLogger('wic') > > @@ -44,17 +32,13 @@ class BootimgPartitionPlugin(SourcePlugin): > name = 'bootimg-partition' > > @classmethod > - def do_prepare_partition(cls, part, source_params, cr, > cr_workdir, > + def do_configure_partition(cls, part, source_params, cr, > cr_workdir, oe_builddir, bootimg_dir, kernel_dir, > - rootfs_dir, native_sysroot): > + native_sysroot): > """ > - Called to do the actual content population for a partition > i.e. it > - 'prepares' the partition to be incorporated into the image. > - In this case, does the following: > - - sets up a vfat partition > - - copies all files listed in IMAGE_BOOT_FILES variable > + Called before do_prepare_partition(), create u-boot specific > boot config """ > - hdddir = "%s/boot" % cr_workdir > + hdddir = "%s/boot.%d" % (cr_workdir, part.lineno) > install_cmd = "install -d %s" % hdddir > exec_cmd(install_cmd) > > @@ -63,12 +47,19 @@ class BootimgPartitionPlugin(SourcePlugin): > if not kernel_dir: > raise WicError("Couldn't find DEPLOY_DIR_IMAGE, > exiting") > - logger.debug('Kernel dir: %s', bootimg_dir) > + boot_files = None > + for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", > part.label), (None, None)): > + if fmt: > + var = fmt % id > + else: > + var = "" > > - boot_files = get_bitbake_var("IMAGE_BOOT_FILES") > + boot_files = get_bitbake_var("IMAGE_BOOT_FILES" + var) > + if boot_files is not None: > + break > > - if not boot_files: > - raise WicError('No boot files defined, IMAGE_BOOT_FILES > unset') > + if boot_files is None: > + raise WicError('No boot files defined, IMAGE_BOOT_FILES > unset for entry #%d' % part.lineno) > logger.debug('Boot files: %s', boot_files) > > @@ -85,9 +76,9 @@ class BootimgPartitionPlugin(SourcePlugin): > logger.debug('Destination entry: %r', dst_entry) > deploy_files.append(dst_entry) > > + cls.install_task = []; > for deploy_entry in deploy_files: > src, dst = deploy_entry > - install_task = [] > if '*' in src: > # by default install files under their basename > entry_name_fn = os.path.basename > @@ -102,22 +93,102 @@ class BootimgPartitionPlugin(SourcePlugin): > > logger.debug('Globbed sources: %s', ', '.join(srcs)) > for entry in srcs: > + src = os.path.relpath(entry, kernel_dir) > entry_dst_name = entry_name_fn(entry) > - install_task.append((entry, > - os.path.join(hdddir, > - > entry_dst_name))) > + cls.install_task.append((src, entry_dst_name)) > else: > - install_task = [(os.path.join(kernel_dir, src), > - os.path.join(hdddir, dst))] > + cls.install_task.append((src, dst)) > + > + if source_params.get('loader') != "u-boot": > + return > + > + configfile = cr.ks.bootloader.configfile > + custom_cfg = None > + if configfile: > + custom_cfg = get_custom_config(configfile) > + if custom_cfg: > + # Use a custom configuration for extlinux.conf > + extlinux_conf = custom_cfg > + logger.debug("Using custom configuration file " > + "%s for extlinux.cfg", configfile) > + else: > + raise WicError("configfile is specified but failed > to " > + "get it from %s." % configfile) > + > + if not custom_cfg: > + # The kernel types supported by the sysboot of u-boot > + kernel_types = ["zImage", "Image", "fitImage", "uImage", > "vmlinux"] > + has_dtb = False > + fdt_dir = '/' > + kernel_name = None > + > + # Find the kernel image name, from the highest > precedence to lowest > + for image in kernel_types: > + for task in cls.install_task: > + src, dst = task > + if re.match(image, src): > + kernel_name = os.path.join('/', dst) > + break > + if kernel_name: > + break > + > + for task in cls.install_task: > + src, dst = task > + # We suppose that all the dtb are in the same > directory > + if re.search(r'\.dtb', src) and fdt_dir == '/': > + has_dtb = True > + fdt_dir = os.path.join(fdt_dir, > os.path.dirname(dst)) > + break > + > + if not kernel_name: > + raise WicError('No kernel file founded') > + > + # Compose the extlinux.conf > + extlinux_conf = "default Yocto\n" > + extlinux_conf += "label Yocto\n" > + extlinux_conf += " kernel %s\n" % kernel_name > + if has_dtb: > + extlinux_conf += " fdtdir %s\n" % fdt_dir > + bootloader = cr.ks.bootloader > + extlinux_conf += "append root=%s rootwait %s\n" \ > + % (cr.rootdev, bootloader.append if > bootloader.append else '') + > + install_cmd = "install -d %s/extlinux/" % hdddir > + exec_cmd(install_cmd) > + cfg = open("%s/extlinux/extlinux.conf" % hdddir, "w") > + cfg.write(extlinux_conf) > + cfg.close() > + > + > + @classmethod > + def do_prepare_partition(cls, part, source_params, cr, > cr_workdir, > + oe_builddir, bootimg_dir, kernel_dir, > + rootfs_dir, native_sysroot): > + """ > + Called to do the actual content population for a partition > i.e. it > + 'prepares' the partition to be incorporated into the image. > + In this case, does the following: > + - sets up a vfat partition > + - copies all files listed in IMAGE_BOOT_FILES variable > + """ > + hdddir = "%s/boot.%d" % (cr_workdir, part.lineno) > + > + if not kernel_dir: > + kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > + if not kernel_dir: > + raise WicError("Couldn't find DEPLOY_DIR_IMAGE, > exiting") + > + logger.debug('Kernel dir: %s', bootimg_dir) > + > > - for task in install_task: > - src_path, dst_path = task > - logger.debug('Install %s as %s', > - os.path.basename(src_path), dst_path) > - install_cmd = "install -m 0644 -D %s %s" \ > - % (src_path, dst_path) > - exec_cmd(install_cmd) > + for task in cls.install_task: > + src_path, dst_path = task > + logger.debug('Install %s as %s', src_path, dst_path) > + install_cmd = "install -m 0644 -D %s %s" \ > + % (os.path.join(kernel_dir, src_path), > + os.path.join(hdddir, dst_path)) > + exec_cmd(install_cmd) > > logger.debug('Prepare boot partition using rootfs in %s', > hdddir) part.prepare_rootfs(cr_workdir, oe_builddir, hdddir, > - native_sysroot) > + native_sysroot, False) > diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py > b/scripts/lib/wic/plugins/source/bootimg-pcbios.py index > 11db304..f2639e7 100644 --- > a/scripts/lib/wic/plugins/source/bootimg-pcbios.py +++ > b/scripts/lib/wic/plugins/source/bootimg-pcbios.py @@ -1,21 +1,7 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2014, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This implements the 'bootimg-pcbios' source plugin class for 'wic' > @@ -26,13 +12,13 @@ > > import logging > import os > +import re > > from wic import WicError > from wic.engine import get_custom_config > -from wic.utils import runner > from wic.pluginbase import SourcePlugin > -from wic.utils.misc import (exec_cmd, exec_native_cmd, > - get_bitbake_var, BOOTDD_EXTRA_SPACE) > +from wic.misc import (exec_cmd, exec_native_cmd, > + get_bitbake_var, BOOTDD_EXTRA_SPACE) > > logger = logging.getLogger('wic') > > @@ -44,19 +30,22 @@ class BootimgPcbiosPlugin(SourcePlugin): > name = 'bootimg-pcbios' > > @classmethod > - def _get_syslinux_dir(cls, bootimg_dir): > + def _get_bootimg_dir(cls, bootimg_dir, dirname): > """ > - Get path to syslinux from either default bootimg_dir > - or wic-tools STAGING_DIR. > + Check if dirname exists in default bootimg_dir or in > STAGING_DIR. """ > - for path in (bootimg_dir, get_bitbake_var("STAGING_DATADIR", > "wic-tools")): > - if not path: > - continue > - syslinux_dir = os.path.join(path, 'syslinux') > - if os.path.exists(syslinux_dir): > - return syslinux_dir > + staging_datadir = get_bitbake_var("STAGING_DATADIR") > + for result in (bootimg_dir, staging_datadir): > + if os.path.exists("%s/%s" % (result, dirname)): > + return result > + > + # STAGING_DATADIR is expanded with MLPREFIX if multilib is > enabled > + # but dependency syslinux is still populated to original > STAGING_DATADIR > + nonarch_datadir = re.sub('/[^/]*recipe-sysroot', > '/recipe-sysroot', staging_datadir) > + if os.path.exists(os.path.join(nonarch_datadir, dirname)): > + return nonarch_datadir > > - raise WicError("Couldn't find syslinux directory, exiting") > + raise WicError("Couldn't find correct bootimg_dir, exiting") > > @classmethod > def do_install_disk(cls, disk, disk_name, creator, workdir, > oe_builddir, @@ -65,11 +54,12 @@ class > BootimgPcbiosPlugin(SourcePlugin): Called after all partitions have > been prepared and assembled into a disk image. In this case, we > install the MBR. """ > - syslinux_dir = cls._get_syslinux_dir(bootimg_dir) > + bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux') > + mbrfile = "%s/syslinux/" % bootimg_dir > if creator.ptable_format == 'msdos': > - mbrfile = os.path.join(syslinux_dir, "mbr.bin") > + mbrfile += "mbr.bin" > elif creator.ptable_format == 'gpt': > - mbrfile = os.path.join(syslinux_dir, "gptmbr.bin") > + mbrfile += "gptmbr.bin" > else: > raise WicError("Unsupported partition table: %s" % > creator.ptable_format) > @@ -83,10 +73,8 @@ class BootimgPcbiosPlugin(SourcePlugin): > logger.debug("Installing MBR on disk %s as %s with size %s > bytes", disk_name, full_path, disk.min_size) > > - rcode = runner.show(['dd', 'if=%s' % mbrfile, > - 'of=%s' % full_path, 'conv=notrunc']) > - if rcode != 0: > - raise WicError("Unable to set MBR to %s" % full_path) > + dd_cmd = "dd if=%s of=%s conv=notrunc" % (mbrfile, full_path) > + exec_cmd(dd_cmd, native_sysroot) > > @classmethod > def do_configure_partition(cls, part, source_params, creator, > cr_workdir, @@ -155,22 +143,28 @@ class > BootimgPcbiosPlugin(SourcePlugin): 'prepares' the partition to be > incorporated into the image. In this case, prepare content for legacy > bios boot partition. """ > - syslinux_dir = cls._get_syslinux_dir(bootimg_dir) > + bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux') > > staging_kernel_dir = kernel_dir > > hdddir = "%s/hdd/boot" % cr_workdir > > - cmds = ("install -m 0644 %s/bzImage %s/vmlinuz" % > - (staging_kernel_dir, hdddir), > - "install -m 444 %s/ldlinux.sys %s/ldlinux.sys" % > - (syslinux_dir, hdddir), > - "install -m 0644 %s/vesamenu.c32 %s/vesamenu.c32" % > - (syslinux_dir, hdddir), > - "install -m 444 %s/libcom32.c32 %s/libcom32.c32" % > - (syslinux_dir, hdddir), > - "install -m 444 %s/libutil.c32 %s/libutil.c32" % > - (syslinux_dir, hdddir)) > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) + > + cmds = ("install -m 0644 %s/%s %s/vmlinuz" % > + (staging_kernel_dir, kernel, hdddir), > + "install -m 444 %s/syslinux/ldlinux.sys > %s/ldlinux.sys" % > + (bootimg_dir, hdddir), > + "install -m 0644 %s/syslinux/vesamenu.c32 > %s/vesamenu.c32" % > + (bootimg_dir, hdddir), > + "install -m 444 %s/syslinux/libcom32.c32 > %s/libcom32.c32" % > + (bootimg_dir, hdddir), > + "install -m 444 %s/syslinux/libutil.c32 > %s/libutil.c32" % > + (bootimg_dir, hdddir)) > > for install_cmd in cmds: > exec_cmd(install_cmd) > @@ -190,9 +184,10 @@ class BootimgPcbiosPlugin(SourcePlugin): > extra_blocks, part.mountpoint, blocks) > > # dosfs image, created by mkdosfs > - bootimg = "%s/boot.img" % cr_workdir > + bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno) > > - dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, > blocks) > + dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \ > + (part.fsuuid, bootimg, blocks) > exec_native_cmd(dosfs_cmd, native_sysroot) > > mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) > diff --git a/scripts/lib/wic/plugins/source/fsimage.py > b/scripts/lib/wic/plugins/source/fsimage.py deleted file mode 100644 > index f781499..0000000 > --- a/scripts/lib/wic/plugins/source/fsimage.py > +++ /dev/null > @@ -1,56 +0,0 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > -# > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. -# > - > -import logging > -import os > - > -from wic import WicError > -from wic.pluginbase import SourcePlugin > -from wic.utils.misc import get_bitbake_var > - > -logger = logging.getLogger('wic') > - > -class FSImagePlugin(SourcePlugin): > - """ > - Add an already existing filesystem image to the partition layout. > - """ > - > - name = 'fsimage' > - > - @classmethod > - def do_prepare_partition(cls, part, source_params, cr, > cr_workdir, > - oe_builddir, bootimg_dir, kernel_dir, > - rootfs_dir, native_sysroot): > - """ > - Called to do the actual content population for a partition > i.e. it > - 'prepares' the partition to be incorporated into the image. > - """ > - if not bootimg_dir: > - bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > - if not bootimg_dir: > - raise WicError("Couldn't find DEPLOY_DIR_IMAGE, > exiting") - > - logger.debug('Bootimg dir: %s', bootimg_dir) > - > - if 'file' not in source_params: > - raise WicError("No file specified") > - > - src = os.path.join(bootimg_dir, source_params['file']) > - > - > - logger.debug('Preparing partition using image %s', src) > - part.prepare_rootfs_from_fs_image(cr_workdir, src, "") > diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py > b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py index > 1ceba62..11326a2 100644 --- > a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py +++ > b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py @@ -1,18 +1,5 > @@ -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > - > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > # > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This implements the 'isoimage-isohybrid' source plugin class for > 'wic' @@ -29,7 +16,7 @@ import shutil > from wic import WicError > from wic.engine import get_custom_config > from wic.pluginbase import SourcePlugin > -from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var > +from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var > > logger = logging.getLogger('wic') > > @@ -47,7 +34,7 @@ class IsoImagePlugin(SourcePlugin): > > Example kickstart file: > part /boot --source isoimage-isohybrid > --sourceparams="loader=grub-efi, \\ > - image_name= IsoImage" --ondisk cd --label LIVECD --fstype=ext2 > + image_name= IsoImage" --ondisk cd --label LIVECD > bootloader --timeout=10 --append=" " > > In --sourceparams "loader" specifies the bootloader used for > booting in EFI @@ -83,8 +70,13 @@ class IsoImagePlugin(SourcePlugin): > syslinux_conf += "DEFAULT boot\n" > syslinux_conf += "LABEL boot\n" > > - kernel = "/bzImage" > - syslinux_conf += "KERNEL " + kernel + "\n" > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) + > + syslinux_conf += "KERNEL /" + kernel + "\n" > syslinux_conf += "APPEND initrd=/initrd LABEL=boot %s\n" \ > % bootloader.append > > @@ -95,7 +87,7 @@ class IsoImagePlugin(SourcePlugin): > cfg.write(syslinux_conf) > > @classmethod > - def do_configure_grubefi(cls, part, creator, cr_workdir): > + def do_configure_grubefi(cls, part, creator, target_dir): > """ > Create loader-specific (grub-efi) config > """ > @@ -109,7 +101,7 @@ class IsoImagePlugin(SourcePlugin): > raise WicError("configfile is specified " > "but failed to get it from %s", > configfile) else: > - splash = os.path.join(cr_workdir, "EFI/boot/splash.jpg") > + splash = os.path.join(target_dir, "splash.jpg") > if os.path.exists(splash): > splashline = "menu background splash.jpg" > else: > @@ -127,9 +119,13 @@ class IsoImagePlugin(SourcePlugin): > grubefi_conf += "\n" > grubefi_conf += "menuentry 'boot'{\n" > > - kernel = "/bzImage" > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) > - grubefi_conf += "linux %s rootwait %s\n" \ > + grubefi_conf += "linux /%s rootwait %s\n" \ > % (kernel, bootloader.append) > grubefi_conf += "initrd /initrd \n" > grubefi_conf += "}\n" > @@ -137,9 +133,10 @@ class IsoImagePlugin(SourcePlugin): > if splashline: > grubefi_conf += "%s\n" % splashline > > - logger.debug("Writing grubefi config %s/EFI/BOOT/grub.cfg", > cr_workdir) > + cfg_path = os.path.join(target_dir, "grub.cfg") > + logger.debug("Writing grubefi config %s", cfg_path) > > - with open("%s/EFI/BOOT/grub.cfg" % cr_workdir, "w") as cfg: > + with open(cfg_path, "w") as cfg: > cfg.write(grubefi_conf) > > @staticmethod > @@ -162,13 +159,14 @@ class IsoImagePlugin(SourcePlugin): > if not image_type: > raise WicError("Couldn't find INITRAMFS_FSTYPES, > exiting.") > - target_arch = get_bitbake_var("TRANSLATED_TARGET_ARCH") > - if not target_arch: > - raise WicError("Couldn't find > TRANSLATED_TARGET_ARCH, exiting.") > + machine = os.path.basename(initrd_dir) > > - initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, > image_name, target_arch, image_type))[0] > + pattern = '%s/%s*%s.%s' % (initrd_dir, image_name, > machine, image_type) > + files = glob.glob(pattern) > + if files: > + initrd = files[0] > > - if not os.path.exists(initrd): > + if not initrd or not os.path.exists(initrd): > # Create initrd from rootfs directory > initrd = "%s/initrd.cpio.gz" % cr_workdir > initrd_dir = "%s/INITRD" % cr_workdir > @@ -189,10 +187,9 @@ class IsoImagePlugin(SourcePlugin): > else: > raise WicError("Couldn't find or build initrd, > exiting.") > - exec_cmd("cd %s && find . | cpio -o -H newc -R +0:+0 > >./initrd.cpio " \ > - % initrd_dir, as_shell=True) > - exec_cmd("gzip -f -9 -c %s/initrd.cpio > %s" \ > - % (cr_workdir, initrd), as_shell=True) > + exec_cmd("cd %s && find . | cpio -o -H newc -R root:root > >%s/initrd.cpio " \ > + % (initrd_dir, cr_workdir), as_shell=True) > + exec_cmd("gzip -f -9 %s/initrd.cpio" % cr_workdir, > as_shell=True) shutil.rmtree(initrd_dir) > > return initrd > @@ -206,8 +203,8 @@ class IsoImagePlugin(SourcePlugin): > """ > isodir = "%s/ISO/" % cr_workdir > > - if os.path.exists(cr_workdir): > - shutil.rmtree(cr_workdir) > + if os.path.exists(isodir): > + shutil.rmtree(isodir) > > install_cmd = "install -d %s " % isodir > exec_cmd(install_cmd) > @@ -251,33 +248,8 @@ class IsoImagePlugin(SourcePlugin): > raise WicError("Couldn't find IMAGE_ROOTFS, exiting.") > > part.rootfs_dir = rootfs_dir > - > - # Prepare rootfs.img > deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > img_iso_dir = get_bitbake_var("ISODIR") > - rootfs_img = "%s/rootfs.img" % img_iso_dir > - if not os.path.isfile(rootfs_img): > - # check if rootfs.img is in deploydir > - deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > - image_name = get_bitbake_var("IMAGE_LINK_NAME") > - rootfs_img = "%s/%s.%s" \ > - % (deploy_dir, image_name, part.fstype) > - > - if not os.path.isfile(rootfs_img): > - # create image file with type specified by --fstype > - # which contains rootfs > - du_cmd = "du -bks %s" % rootfs_dir > - out = exec_cmd(du_cmd) > - part.size = int(out.split()[0]) > - part.extra_space = 0 > - part.overhead_factor = 1.2 > - part.prepare_rootfs(cr_workdir, oe_builddir, rootfs_dir, > \ > - native_sysroot) > - rootfs_img = part.source_file > - > - install_cmd = "install -m 0644 %s %s/rootfs.img" \ > - % (rootfs_img, isodir) > - exec_cmd(install_cmd) > > # Remove the temporary file created by part.prepare_rootfs() > if os.path.isfile(part.source_file): > @@ -305,27 +277,25 @@ class IsoImagePlugin(SourcePlugin): > if os.path.isfile("%s/initrd.cpio.gz" % cr_workdir): > os.remove("%s/initrd.cpio.gz" % cr_workdir) > > - # Install bzImage > - install_cmd = "install -m 0644 %s/bzImage %s/bzImage" % \ > - (kernel_dir, isodir) > + kernel = get_bitbake_var("KERNEL_IMAGETYPE") > + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1": > + if get_bitbake_var("INITRAMFS_IMAGE"): > + kernel = "%s-%s.bin" % \ > + (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) + > + install_cmd = "install -m 0644 %s/%s %s/%s" % \ > + (kernel_dir, kernel, isodir, kernel) > exec_cmd(install_cmd) > > #Create bootloader for efi boot > try: > - if source_params['loader'] == 'grub-efi': > - # Builds grub.cfg if ISODIR didn't exist or > - # didn't contains grub.cfg > - bootimg_dir = img_iso_dir > - if not os.path.exists("%s/EFI/BOOT" % bootimg_dir): > - bootimg_dir = "%s/bootimg" % cr_workdir > - if os.path.exists(bootimg_dir): > - shutil.rmtree(bootimg_dir) > - install_cmd = "install -d %s/EFI/BOOT" % > bootimg_dir > - exec_cmd(install_cmd) > - > - if not os.path.isfile("%s/EFI/BOOT/boot.cfg" % > bootimg_dir): > - cls.do_configure_grubefi(part, creator, > bootimg_dir) > + target_dir = "%s/EFI/BOOT" % isodir > + if os.path.exists(target_dir): > + shutil.rmtree(target_dir) > > + os.makedirs(target_dir) > + > + if source_params['loader'] == 'grub-efi': > # Builds bootx64.efi/bootia32.efi if ISODIR didn't > exist or # didn't contains it > target_arch = get_bitbake_var("TARGET_SYS") > @@ -333,37 +303,25 @@ class IsoImagePlugin(SourcePlugin): > raise WicError("Coludn't find target > architecture") > if re.match("x86_64", target_arch): > - grub_target = 'x86_64-efi' > - grub_image = "bootx64.efi" > + grub_src_image = "grub-efi-bootx64.efi" > + grub_dest_image = "bootx64.efi" > elif re.match('i.86', target_arch): > - grub_target = 'i386-efi' > - grub_image = "bootia32.efi" > + grub_src_image = "grub-efi-bootia32.efi" > + grub_dest_image = "bootia32.efi" > else: > raise WicError("grub-efi is incompatible with > target %s" % target_arch) > > - if not os.path.isfile("%s/EFI/BOOT/%s" \ > - % (bootimg_dir, grub_image)): > - grub_path = get_bitbake_var("STAGING_LIBDIR", > "wic-tools") > - if not grub_path: > - raise WicError("Couldn't find > STAGING_LIBDIR, exiting.") - > - grub_core = "%s/grub/%s" % (grub_path, > grub_target) > - if not os.path.exists(grub_core): > - raise WicError("Please build grub-efi first") > - > - grub_cmd = "grub-mkimage -p '/EFI/BOOT' " > - grub_cmd += "-d %s " % grub_core > - grub_cmd += "-O %s -o %s/EFI/BOOT/%s " \ > - % (grub_target, bootimg_dir, > grub_image) > - grub_cmd += "part_gpt part_msdos ntfs ntfscomp > fat ext2 " > - grub_cmd += "normal chain boot configfile linux > multiboot " > - grub_cmd += "search efi_gop efi_uga font gfxterm > gfxmenu " > - grub_cmd += "terminal minicmd test iorw loadenv > echo help " > - grub_cmd += "reboot serial terminfo iso9660 > loopback tar " > - grub_cmd += "memdisk ls search_fs_uuid udf btrfs > xfs lvm " > - grub_cmd += "reiserfs ata " > - exec_native_cmd(grub_cmd, native_sysroot) > + grub_target = os.path.join(target_dir, > grub_dest_image) > + if not os.path.isfile(grub_target): > + grub_src = os.path.join(deploy_dir, > grub_src_image) > + if not os.path.exists(grub_src): > + raise WicError("Grub loader %s is not found > in %s. " > + "Please build grub-efi first" > % (grub_src_image, deploy_dir)) > + shutil.copy(grub_src, grub_target) > + > + if not os.path.isfile(os.path.join(target_dir, > "boot.cfg")): > + cls.do_configure_grubefi(part, creator, > target_dir) > else: > raise WicError("unrecognized bootimg-efi loader: %s" > % @@ -371,15 +329,6 @@ class IsoImagePlugin(SourcePlugin): > except KeyError: > raise WicError("bootimg-efi requires a loader, none > specified") > - if os.path.exists("%s/EFI/BOOT" % isodir): > - shutil.rmtree("%s/EFI/BOOT" % isodir) > - > - shutil.copytree(bootimg_dir+"/EFI/BOOT", isodir+"/EFI/BOOT") > - > - # If exists, remove cr_workdir/bootimg temporary folder > - if os.path.exists("%s/bootimg" % cr_workdir): > - shutil.rmtree("%s/bootimg" % cr_workdir) > - > # Create efi.img that contains bootloader files for EFI > booting # if ISODIR didn't exist or didn't contains it > if os.path.isfile("%s/efi.img" % img_iso_dir): > @@ -387,19 +336,23 @@ class IsoImagePlugin(SourcePlugin): > (img_iso_dir, isodir) > exec_cmd(install_cmd) > else: > + # Default to 100 blocks of extra space for file system > overhead > + esp_extra_blocks = > int(source_params.get('esp_extra_blocks', '100')) + > du_cmd = "du -bks %s/EFI" % isodir > out = exec_cmd(du_cmd) > blocks = int(out.split()[0]) > - # Add some extra space for file system overhead > - blocks += 100 > + blocks += esp_extra_blocks > logger.debug("Added 100 extra blocks to %s to get to %d " > "total blocks", part.mountpoint, blocks) > > # dosfs image for EFI boot > bootimg = "%s/efi.img" % isodir > > - dosfs_cmd = 'mkfs.vfat -n "EFIimg" -S 512 -C %s %d' \ > - % (bootimg, blocks) > + esp_label = source_params.get('esp_label', 'EFIimg') > + > + dosfs_cmd = 'mkfs.vfat -n \'%s\' -S 512 -C %s %d' \ > + % (esp_label, bootimg, blocks) > exec_native_cmd(dosfs_cmd, native_sysroot) > > mmd_cmd = "mmd -i %s ::/EFI" % bootimg > @@ -413,7 +366,7 @@ class IsoImagePlugin(SourcePlugin): > exec_cmd(chmod_cmd) > > # Prepare files for legacy boot > - syslinux_dir = get_bitbake_var("STAGING_DATADIR", > "wic-tools") > + syslinux_dir = get_bitbake_var("STAGING_DATADIR") > if not syslinux_dir: > raise WicError("Couldn't find STAGING_DATADIR, exiting.") > > diff --git a/scripts/lib/wic/plugins/source/rawcopy.py > b/scripts/lib/wic/plugins/source/rawcopy.py index e1c4f5e..3c4997d > 100644 --- a/scripts/lib/wic/plugins/source/rawcopy.py > +++ b/scripts/lib/wic/plugins/source/rawcopy.py > @@ -1,18 +1,5 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > > import logging > @@ -20,7 +7,7 @@ import os > > from wic import WicError > from wic.pluginbase import SourcePlugin > -from wic.utils.misc import exec_cmd, get_bitbake_var > +from wic.misc import exec_cmd, get_bitbake_var > from wic.filemap import sparse_copy > > logger = logging.getLogger('wic') > @@ -32,6 +19,25 @@ class RawCopyPlugin(SourcePlugin): > > name = 'rawcopy' > > + @staticmethod > + def do_image_label(fstype, dst, label): > + if fstype.startswith('ext'): > + cmd = 'tune2fs -L %s %s' % (label, dst) > + elif fstype in ('msdos', 'vfat'): > + cmd = 'dosfslabel %s %s' % (dst, label) > + elif fstype == 'btrfs': > + cmd = 'btrfs filesystem label %s %s' % (dst, label) > + elif fstype == 'swap': > + cmd = 'mkswap -L %s %s' % (label, dst) > + elif fstype == 'squashfs': > + raise WicError("It's not possible to update a squashfs " > + "filesystem label '%s'" % (label)) > + else: > + raise WicError("Cannot update filesystem label: " > + "Unknown fstype: '%s'" % (fstype)) > + > + exec_cmd(cmd) > + > @classmethod > def do_prepare_partition(cls, part, source_params, cr, > cr_workdir, oe_builddir, bootimg_dir, kernel_dir, > @@ -51,7 +57,10 @@ class RawCopyPlugin(SourcePlugin): > raise WicError("No file specified") > > src = os.path.join(kernel_dir, source_params['file']) > - dst = os.path.join(cr_workdir, "%s.%s" % > (source_params['file'], part.lineno)) > + dst = os.path.join(cr_workdir, "%s.%s" % > (os.path.basename(source_params['file']), part.lineno)) + > + if not os.path.exists(os.path.dirname(dst)): > + os.makedirs(os.path.dirname(dst)) > > if 'skip' in source_params: > sparse_copy(src, dst, skip=int(source_params['skip'])) > @@ -66,4 +75,7 @@ class RawCopyPlugin(SourcePlugin): > if filesize > part.size: > part.size = filesize > > + if part.label: > + RawCopyPlugin.do_image_label(part.fstype, dst, > part.label) + > part.source_file = dst > diff --git a/scripts/lib/wic/plugins/source/rootfs.py > b/scripts/lib/wic/plugins/source/rootfs.py index f2e2ca8..f1db83f > 100644 --- a/scripts/lib/wic/plugins/source/rootfs.py > +++ b/scripts/lib/wic/plugins/source/rootfs.py > @@ -1,21 +1,7 @@ > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2014, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION > # This implements the 'rootfs' source plugin class for 'wic' > @@ -28,12 +14,14 @@ > import logging > import os > import shutil > +import sys > > from oe.path import copyhardlinktree > +from pathlib import Path > > from wic import WicError > from wic.pluginbase import SourcePlugin > -from wic.utils.misc import get_bitbake_var, exec_cmd > +from wic.misc import get_bitbake_var, exec_native_cmd > > logger = logging.getLogger('wic') > > @@ -44,10 +32,26 @@ class RootfsPlugin(SourcePlugin): > > name = 'rootfs' > > + @staticmethod > + def __validate_path(cmd, rootfs_dir, path): > + if os.path.isabs(path): > + logger.error("%s: Must be relative: %s" % (cmd, > orig_path)) > + sys.exit(1) > + > + # Disallow climbing outside of parent directory using '..', > + # because doing so could be quite disastrous (we will delete > the > + # directory, or modify a directory outside OpenEmbedded). > + full_path = os.path.realpath(os.path.join(rootfs_dir, path)) > + if not full_path.startswith(os.path.realpath(rootfs_dir)): > + logger.error("%s: Must point inside the rootfs:" % (cmd, > path)) > + sys.exit(1) > + > + return full_path > + > @staticmethod > def __get_rootfs_dir(rootfs_dir): > if os.path.isdir(rootfs_dir): > - return rootfs_dir > + return os.path.realpath(rootfs_dir) > > image_rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", > rootfs_dir) if not os.path.isdir(image_rootfs_dir): > @@ -55,7 +59,16 @@ class RootfsPlugin(SourcePlugin): > "named %s has been found at %s, exiting." > % (rootfs_dir, image_rootfs_dir)) > > - return image_rootfs_dir > + return os.path.realpath(image_rootfs_dir) > + > + @staticmethod > + def __get_pseudo(native_sysroot, rootfs, pseudo_dir): > + pseudo = "export PSEUDO_PREFIX=%s/usr;" % native_sysroot > + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir > + pseudo += "export PSEUDO_PASSWD=%s;" % rootfs > + pseudo += "export PSEUDO_NOSYMLINKEXP=1;" > + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD") > + return pseudo > > @classmethod > def do_prepare_partition(cls, part, source_params, cr, > cr_workdir, @@ -80,33 +93,111 @@ class RootfsPlugin(SourcePlugin): > raise WicError("Couldn't find --rootfs-dir=%s > connection or " "it is not a valid path, exiting" % part.rootfs_dir) > > - real_rootfs_dir = cls.__get_rootfs_dir(rootfs_dir) > + part.rootfs_dir = cls.__get_rootfs_dir(rootfs_dir) > + pseudo_dir = os.path.join(part.rootfs_dir, "../pseudo") > + if not os.path.lexists(pseudo_dir): > + logger.warn("%s folder does not exist. " > + "Usernames and permissions will be invalid " > % pseudo_dir) > + pseudo_dir = None > > + new_rootfs = None > + new_pseudo = None > # Handle excluded paths. > - if part.exclude_path is not None: > + if part.exclude_path or part.include_path or > part.change_directory: # We need a new rootfs directory we can delete > files from. Copy to # workdir. > - new_rootfs = os.path.realpath(os.path.join(cr_workdir, > "rootfs")) > + new_rootfs = os.path.realpath(os.path.join(cr_workdir, > "rootfs%d" % part.lineno)) > if os.path.lexists(new_rootfs): > shutil.rmtree(os.path.join(new_rootfs)) > > - copyhardlinktree(real_rootfs_dir, new_rootfs) > + if part.change_directory: > + cd = part.change_directory > + if cd[-1] == '/': > + cd = cd[:-1] > + orig_dir = cls.__validate_path("--change-directory", > part.rootfs_dir, cd) > + else: > + orig_dir = part.rootfs_dir > + copyhardlinktree(orig_dir, new_rootfs) > + > + # Convert the pseudo directory to its new location > + if (pseudo_dir): > + new_pseudo = os.path.realpath( > + os.path.join(cr_workdir, "pseudo%d" % > part.lineno)) > + if os.path.lexists(new_pseudo): > + shutil.rmtree(new_pseudo) > + os.mkdir(new_pseudo) > + shutil.copy(os.path.join(pseudo_dir, "files.db"), > + os.path.join(new_pseudo, "files.db")) > + > + pseudo_cmd = "%s -B -m %s -M %s" % > (cls.__get_pseudo(native_sysroot, > + > new_rootfs, > + > new_pseudo), > + orig_dir, > new_rootfs) > + exec_native_cmd(pseudo_cmd, native_sysroot) > + > + for in_path in part.include_path or []: > + #parse arguments > + include_path = in_path[0] > + if len(in_path) > 2: > + logger.error("'Invalid number of arguments for > include-path") > + sys.exit(1) > + if len(in_path) == 2: > + path = in_path[1] > + else: > + path = None > + > + # Pack files to be included into a tar file. > + # We need to create a tar file, because that way we > can keep the > + # permissions from the files even when they belong > to different > + # pseudo enviroments. > + # If we simply copy files using > copyhardlinktree/copytree... the > + # copied files will belong to the user running wic. > + tar_file = os.path.realpath( > + os.path.join(cr_workdir, > "include-path%d.tar" % part.lineno)) > + if os.path.isfile(include_path): > + parent = > os.path.dirname(os.path.realpath(include_path)) > + tar_cmd = "tar c --owner=root --group=root -f %s > -C %s %s" % ( > + tar_file, parent, > os.path.relpath(include_path, parent)) > + exec_native_cmd(tar_cmd, native_sysroot) > + else: > + if include_path in krootfs_dir: > + include_path = krootfs_dir[include_path] > + include_path = cls.__get_rootfs_dir(include_path) > + include_pseudo = os.path.join(include_path, > "../pseudo") > + if os.path.lexists(include_pseudo): > + pseudo = cls.__get_pseudo(native_sysroot, > include_path, > + include_pseudo) > + tar_cmd = "tar cf %s -C %s ." % (tar_file, > include_path) > + else: > + pseudo = None > + tar_cmd = "tar c --owner=root --group=root > -f %s -C %s ." % ( > + tar_file, include_path) > + exec_native_cmd(tar_cmd, native_sysroot, pseudo) > + > + #create destination > + if path: > + destination = > cls.__validate_path("--include-path", new_rootfs, path) > + Path(destination).mkdir(parents=True, > exist_ok=True) > + else: > + destination = new_rootfs > > - real_rootfs_dir = new_rootfs > + #extract destination > + untar_cmd = "tar xf %s -C %s" % (tar_file, > destination) > + if new_pseudo: > + pseudo = cls.__get_pseudo(native_sysroot, > new_rootfs, new_pseudo) > + else: > + pseudo = None > + exec_native_cmd(untar_cmd, native_sysroot, pseudo) > + os.remove(tar_file) > > - for orig_path in part.exclude_path: > + for orig_path in part.exclude_path or []: > path = orig_path > - if os.path.isabs(path): > - msger.error("Must be relative: > --exclude-path=%s" % orig_path) > - full_path = > os.path.realpath(os.path.join(new_rootfs, path)) > + full_path = cls.__validate_path("--exclude-path", > new_rootfs, path) > - # Disallow climbing outside of parent directory > using '..', > - # because doing so could be quite disastrous (we > will delete the > - # directory). > - if not full_path.startswith(new_rootfs): > - msger.error("'%s' points to a path outside the > rootfs" % orig_path) > + if not os.path.lexists(full_path): > + continue > > if path.endswith(os.sep): > # Delete content only. > @@ -120,6 +211,6 @@ class RootfsPlugin(SourcePlugin): > # Delete whole directory. > shutil.rmtree(full_path) > > - part.rootfs_dir = real_rootfs_dir > part.prepare_rootfs(cr_workdir, oe_builddir, > - real_rootfs_dir, native_sysroot) > + new_rootfs or part.rootfs_dir, > native_sysroot, > + pseudo_dir = new_pseudo or pseudo_dir) > diff --git a/scripts/lib/wic/utils/__init__.py > b/scripts/lib/wic/utils/__init__.py deleted file mode 100644 > index e69de29..0000000 > diff --git a/scripts/lib/wic/utils/runner.py > b/scripts/lib/wic/utils/runner.py deleted file mode 100644 > index 56d7ea3..0000000 > --- a/scripts/lib/wic/utils/runner.py > +++ /dev/null > @@ -1,114 +0,0 @@ > -#!/usr/bin/env python -tt > -# > -# Copyright (c) 2011 Intel, Inc. > -# > -# This program is free software; you can redistribute it and/or > modify it -# under the terms of the GNU General Public License as > published by the Free -# Software Foundation; version 2 of the License > -# > -# This program is distributed in the hope that it will be useful, but > -# WITHOUT ANY WARRANTY; without even the implied warranty of > MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > General Public License -# for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA > 02111-1307, USA. - > -import logging > -import os > -import subprocess > - > -from wic import WicError > - > -logger = logging.getLogger('wic') > - > -def runtool(cmdln_or_args, catch=1): > - """ wrapper for most of the subprocess calls > - input: > - cmdln_or_args: can be both args and cmdln str (shell=True) > - catch: 0, quitely run > - 1, only STDOUT > - 2, only STDERR > - 3, both STDOUT and STDERR > - return: > - (rc, output) > - if catch==0: the output will always None > - """ > - > - if catch not in (0, 1, 2, 3): > - # invalid catch selection, will cause exception, that's good > - return None > - > - if isinstance(cmdln_or_args, list): > - cmd = cmdln_or_args[0] > - shell = False > - else: > - import shlex > - cmd = shlex.split(cmdln_or_args)[0] > - shell = True > - > - if catch != 3: > - dev_null = os.open("/dev/null", os.O_WRONLY) > - > - if catch == 0: > - sout = dev_null > - serr = dev_null > - elif catch == 1: > - sout = subprocess.PIPE > - serr = dev_null > - elif catch == 2: > - sout = dev_null > - serr = subprocess.PIPE > - elif catch == 3: > - sout = subprocess.PIPE > - serr = subprocess.STDOUT > - > - try: > - process = subprocess.Popen(cmdln_or_args, stdout=sout, > - stderr=serr, shell=shell) > - (sout, serr) = process.communicate() > - # combine stdout and stderr, filter None out and decode > - out = ''.join([out.decode('utf-8') for out in [sout, serr] > if out]) > - except OSError as err: > - if err.errno == 2: > - # [Errno 2] No such file or directory > - raise WicError('Cannot run command: %s, lost > dependency?' % cmd) > - else: > - raise # relay > - finally: > - if catch != 3: > - os.close(dev_null) > - > - return (process.returncode, out) > - > -def show(cmdln_or_args): > - """Show all messages using logger.debug.""" > - > - rcode, out = runtool(cmdln_or_args, catch=3) > - > - if isinstance(cmdln_or_args, list): > - cmd = ' '.join(cmdln_or_args) > - else: > - cmd = cmdln_or_args > - > - msg = 'running command: "%s"' % cmd > - if out: > - out = out.strip() > - if out: > - msg += ', with output::' > - msg += '\n +----------------' > - for line in out.splitlines(): > - msg += '\n | %s' % line > - msg += '\n +----------------' > - > - logger.debug(msg) > - > - return rcode > - > -def outs(cmdln_or_args, catch=1): > - # get the outputs of tools > - return runtool(cmdln_or_args, catch)[1].strip() > - > -def quiet(cmdln_or_args): > - return runtool(cmdln_or_args, catch=0)[0] > diff --git a/scripts/wic b/scripts/wic > index a5f2dbf..24700f3 100755 > --- a/scripts/wic > +++ b/scripts/wic > @@ -1,22 +1,8 @@ > #!/usr/bin/env python3 > -# ex:ts=4:sw=4:sts=4:et > -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > # > # Copyright (c) 2013, Intel Corporation. > -# All rights reserved. > # > -# This program is free software; you can redistribute it and/or > modify -# it under the terms of the GNU General Public License > version 2 as -# published by the Free Software Foundation. > -# > -# This program is distributed in the hope that it will be useful, > -# but WITHOUT ANY WARRANTY; without even the implied warranty of > -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > -# GNU General Public License for more details. > -# > -# You should have received a copy of the GNU General Public License > along -# with this program; if not, write to the Free Software > Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA > 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only > # > # DESCRIPTION 'wic' is the OpenEmbedded Image Creator that users can > # use to generate bootable images. Invoking it without any arguments > @@ -33,28 +19,41 @@ __version__ = "0.2.0" > # Python Standard Library modules > import os > import sys > -import optparse > +import argparse > import logging > +import subprocess > + > +from collections import namedtuple > from distutils import spawn > > # External modules > -scripts_path = os.path.abspath(os.path.dirname(__file__)) > +scripts_path = os.path.dirname(os.path.realpath(__file__)) > lib_path = scripts_path + '/lib' > sys.path.insert(0, lib_path) > -oe_lib_path = os.path.join(os.path.dirname(scripts_path), 'meta', > 'lib') -sys.path.insert(0, oe_lib_path) > +import scriptpath > +scriptpath.add_oe_lib_path() > + > +# Check whether wic is running within eSDK environment > +sdkroot = scripts_path > +if os.environ.get('SDKTARGETSYSROOT'): > + while sdkroot != '' and sdkroot != os.sep: > + if os.path.exists(os.path.join(sdkroot, '.devtoolbase')): > + # Set BUILDDIR for wic to work within eSDK > + os.environ['BUILDDIR'] = sdkroot > + # .devtoolbase only exists within eSDK > + # If found, initialize bitbake path for eSDK environment > and append to PATH > + sdkroot = os.path.join(os.path.dirname(scripts_path), > 'bitbake', 'bin') > + os.environ['PATH'] += ":" + sdkroot > + break > + sdkroot = os.path.dirname(sdkroot) > > bitbake_exe = spawn.find_executable('bitbake') > if bitbake_exe: > - bitbake_path = os.path.join(os.path.dirname(bitbake_exe), > '../lib') > - sys.path.insert(0, bitbake_path) > - from bb import cookerdata > - from bb.main import bitbake_main, BitBakeConfigParameters > -else: > - bitbake_main = None > + bitbake_path = scriptpath.add_bitbake_lib_path() > + import bb > > from wic import WicError > -from wic.utils.misc import get_bitbake_var, BB_VARS > +from wic.misc import get_bitbake_var, BB_VARS > from wic import engine > from wic import help as hlp > > @@ -85,67 +84,31 @@ def rootfs_dir_to_args(krootfs_dir): > rootfs_dir += '='.join([key, val]) > return rootfs_dir.strip() > > -def callback_rootfs_dir(option, opt, value, parser): > - """ > - Build a dict using --rootfs_dir connection=dir > - """ > - if not type(parser.values.rootfs_dir) is dict: > - parser.values.rootfs_dir = dict() > > - if '=' in value: > - (key, rootfs_dir) = value.split('=') > - else: > - key = 'ROOTFS_DIR' > - rootfs_dir = value > +class RootfsArgAction(argparse.Action): > + def __init__(self, **kwargs): > + super().__init__(**kwargs) > + > + def __call__(self, parser, namespace, value, option_string=None): > + if not "rootfs_dir" in vars(namespace) or \ > + not type(namespace.__dict__['rootfs_dir']) is dict: > + namespace.__dict__['rootfs_dir'] = {} > > - parser.values.rootfs_dir[key] = rootfs_dir > + if '=' in value: > + (key, rootfs_dir) = value.split('=') > + else: > + key = 'ROOTFS_DIR' > + rootfs_dir = value > > -def wic_create_subcommand(args, usage_str): > + namespace.__dict__['rootfs_dir'][key] = rootfs_dir > + > + > +def wic_create_subcommand(options, usage_str): > """ > Command-line handling for image creation. The real work is done > by image.engine.wic_create() > """ > - parser = optparse.OptionParser(usage=usage_str) > - > - parser.add_option("-o", "--outdir", dest="outdir", default='.', > - help="name of directory to create image in") > - parser.add_option("-e", "--image-name", dest="image_name", > - help="name of the image to use the artifacts > from " > - "e.g. core-image-sato") > - parser.add_option("-r", "--rootfs-dir", dest="rootfs_dir", > type="string", > - action="callback", > callback=callback_rootfs_dir, > - help="path to the /rootfs dir to use as the " > - ".wks rootfs source") > - parser.add_option("-b", "--bootimg-dir", dest="bootimg_dir", > - help="path to the dir containing the boot > artifacts " > - "(e.g. /EFI or /syslinux dirs) to use as > the " > - ".wks bootimg source") > - parser.add_option("-k", "--kernel-dir", dest="kernel_dir", > - help="path to the dir containing the kernel to > use " > - "in the .wks bootimg") > - parser.add_option("-n", "--native-sysroot", > dest="native_sysroot", > - help="path to the native sysroot containing > the tools " > - "to use to build the image") > - parser.add_option("-s", "--skip-build-check", dest="build_check", > - action="store_false", default=True, help="skip > the build check") > - parser.add_option("-f", "--build-rootfs", action="store_true", > help="build rootfs") > - parser.add_option("-c", "--compress-with", choices=("gzip", > "bzip2", "xz"), > - dest='compressor', > - help="compress image with specified > compressor") > - parser.add_option("-m", "--bmap", action="store_true", > help="generate .bmap") > - parser.add_option("-v", "--vars", dest='vars_dir', > - help="directory with .env files that > store " > - "bitbake variables") > - parser.add_option("-D", "--debug", dest="debug", > action="store_true", > - default=False, help="output debug information") > - > - (options, args) = parser.parse_args(args) > - > - if len(args) != 1: > - parser.print_help() > - raise WicError("Wrong number of arguments, exiting") > - > - if options.build_rootfs and not bitbake_main: > + if options.build_rootfs and not bitbake_exe: > raise WicError("Can't build rootfs as bitbake is not in the > $PATH") > if not options.image_name: > @@ -181,39 +144,37 @@ def wic_create_subcommand(args, usage_str): > argv.append("--debug") > > logger.info("Building rootfs...\n") > - if bitbake_main(BitBakeConfigParameters(argv), > - cookerdata.CookerConfiguration()): > - raise WicError("bitbake exited with error") > + subprocess.check_call(argv) > > rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", > options.image_name) kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE", > options.image_name) bootimg_dir = get_bitbake_var("STAGING_DATADIR", > options.image_name) > - native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", > - options.image_name) #, > cache=False) + > + native_sysroot = options.native_sysroot > + if options.vars_dir and not native_sysroot: > + native_sysroot = > get_bitbake_var("RECIPE_SYSROOT_NATIVE", options.image_name) else: > if options.build_rootfs: > raise WicError("Image name is not specified, exiting. " > "(Use -e/--image-name to specify it)") > native_sysroot = options.native_sysroot > > - if not native_sysroot or not os.path.isdir(native_sysroot): > + if not options.vars_dir and (not native_sysroot or not > os.path.isdir(native_sysroot)): logger.info("Building wic-tools...\n") > - if bitbake_main(BitBakeConfigParameters("bitbake > wic-tools".split()), > - cookerdata.CookerConfiguration()): > - raise WicError("bitbake wic-tools failed") > + subprocess.check_call(["bitbake", "wic-tools"]) > native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", > "wic-tools") > - if not native_sysroot: > - raise WicError("Unable to find the location of the > native " > - "tools sysroot to use") > > - wks_file = args[0] > + if not native_sysroot: > + raise WicError("Unable to find the location of the native > tools sysroot") + > + wks_file = options.wks_file > > if not wks_file.endswith(".wks"): > wks_file = engine.find_canned_image(scripts_path, wks_file) > if not wks_file: > raise WicError("No image named %s found, exiting. (Use > 'wic list images' " "to list available images, or specify a > fully-qualified OE " > - "kickstart (.wks) filename)" % args[0]) > + "kickstart (.wks) filename)" % > options.wks_file) > if not options.image_name: > rootfs_dir = '' > @@ -264,59 +225,312 @@ def wic_list_subcommand(args, usage_str): > Command-line handling for listing available images. > The real work is done by image.engine.wic_list() > """ > - parser = optparse.OptionParser(usage=usage_str) > - args = parser.parse_args(args)[1] > - > if not engine.wic_list(args, scripts_path): > - parser.print_help() > raise WicError("Bad list arguments, exiting") > > > -def wic_help_topic_subcommand(args, usage_str): > +def wic_ls_subcommand(args, usage_str): > + """ > + Command-line handling for list content of images. > + The real work is done by engine.wic_ls() > + """ > + engine.wic_ls(args, args.native_sysroot) > + > +def wic_cp_subcommand(args, usage_str): > + """ > + Command-line handling for copying files/dirs to images. > + The real work is done by engine.wic_cp() > + """ > + engine.wic_cp(args, args.native_sysroot) > + > +def wic_rm_subcommand(args, usage_str): > """ > - Command-line handling for help-only 'subcommands'. This is > - essentially a dummy command that doesn nothing but allow users to > - use the existing subcommand infrastructure to display help on a > - particular topic not attached to any particular subcommand. > + Command-line handling for removing files/dirs from images. > + The real work is done by engine.wic_rm() > + """ > + engine.wic_rm(args, args.native_sysroot) > + > +def wic_write_subcommand(args, usage_str): > + """ > + Command-line handling for writing images. > + The real work is done by engine.wic_write() > + """ > + engine.wic_write(args, args.native_sysroot) > + > +def wic_help_subcommand(args, usage_str): > + """ > + Command-line handling for help subcommand to keep the current > + structure of the function definitions. > """ > pass > > > +def wic_help_topic_subcommand(usage_str, help_str): > + """ > + Display function for help 'sub-subcommands'. > + """ > + print(help_str) > + return > + > + > wic_help_topic_usage = """ > """ > > -subcommands = { > - "create": [wic_create_subcommand, > - hlp.wic_create_usage, > - hlp.wic_create_help], > - "list": [wic_list_subcommand, > - hlp.wic_list_usage, > - hlp.wic_list_help], > +helptopics = { > "plugins": [wic_help_topic_subcommand, > wic_help_topic_usage, > - hlp.get_wic_plugins_help], > + hlp.wic_plugins_help], > "overview": [wic_help_topic_subcommand, > wic_help_topic_usage, > hlp.wic_overview_help], > "kickstart": [wic_help_topic_subcommand, > wic_help_topic_usage, > hlp.wic_kickstart_help], > + "create": [wic_help_topic_subcommand, > + wic_help_topic_usage, > + hlp.wic_create_help], > + "ls": [wic_help_topic_subcommand, > + wic_help_topic_usage, > + hlp.wic_ls_help], > + "cp": [wic_help_topic_subcommand, > + wic_help_topic_usage, > + hlp.wic_cp_help], > + "rm": [wic_help_topic_subcommand, > + wic_help_topic_usage, > + hlp.wic_rm_help], > + "write": [wic_help_topic_subcommand, > + wic_help_topic_usage, > + hlp.wic_write_help], > + "list": [wic_help_topic_subcommand, > + wic_help_topic_usage, > + hlp.wic_list_help] > } > > > +def wic_init_parser_create(subparser): > + subparser.add_argument("wks_file") > + > + subparser.add_argument("-o", "--outdir", dest="outdir", > default='.', > + help="name of directory to create image in") > + subparser.add_argument("-e", "--image-name", dest="image_name", > + help="name of the image to use the artifacts > from " > + "e.g. core-image-sato") > + subparser.add_argument("-r", "--rootfs-dir", > action=RootfsArgAction, > + help="path to the /rootfs dir to use as the " > + ".wks rootfs source") > + subparser.add_argument("-b", "--bootimg-dir", dest="bootimg_dir", > + help="path to the dir containing the boot > artifacts " > + "(e.g. /EFI or /syslinux dirs) to use as > the " > + ".wks bootimg source") > + subparser.add_argument("-k", "--kernel-dir", dest="kernel_dir", > + help="path to the dir containing the kernel to > use " > + "in the .wks bootimg") > + subparser.add_argument("-n", "--native-sysroot", > dest="native_sysroot", > + help="path to the native sysroot containing > the tools " > + "to use to build the image") > + subparser.add_argument("-s", "--skip-build-check", > dest="build_check", > + action="store_false", default=True, help="skip > the build check") > + subparser.add_argument("-f", "--build-rootfs", > action="store_true", help="build rootfs") > + subparser.add_argument("-c", "--compress-with", choices=("gzip", > "bzip2", "xz"), > + dest='compressor', > + help="compress image with specified > compressor") > + subparser.add_argument("-m", "--bmap", action="store_true", > help="generate .bmap") > + subparser.add_argument("--no-fstab-update" ,action="store_true", > + help="Do not change fstab file.") > + subparser.add_argument("-v", "--vars", dest='vars_dir', > + help="directory with .env files that > store " > + "bitbake variables") > + subparser.add_argument("-D", "--debug", dest="debug", > action="store_true", > + default=False, help="output debug information") > + subparser.add_argument("-i", "--imager", dest="imager", > + default="direct", help="the wic imager plugin") > + return > + > + > +def wic_init_parser_list(subparser): > + subparser.add_argument("list_type", > + help="can be 'images' or 'source-plugins' " > + "to obtain a list. " > + "If value is a valid .wks image file") > + subparser.add_argument("help_for", default=[], nargs='*', > + help="If 'list_type' is a valid .wks image > file " > + "this value can be 'help' to show the > help information " > + "defined inside the .wks file") > + return > + > +def imgtype(arg): > + """ > + Custom type for ArgumentParser > + Converts path spec to named tuple: (image, partition, path) > + """ > + image = arg > + part = path = None > + if ':' in image: > + image, part = image.split(':') > + if '/' in part: > + part, path = part.split('/', 1) > + if not path: > + path = '/' > + > + if not os.path.isfile(image): > + err = "%s is not a regular file or symlink" % image > + raise argparse.ArgumentTypeError(err) > + > + return namedtuple('ImgType', 'image part path')(image, part, > path) + > +def wic_init_parser_ls(subparser): > + subparser.add_argument("path", type=imgtype, > + help="image spec: [: partition>[]]") > + subparser.add_argument("-n", "--native-sysroot", > + help="path to the native sysroot containing > the tools") + > +def imgpathtype(arg): > + img = imgtype(arg) > + if img.part is None: > + raise argparse.ArgumentTypeError("partition number is not > specified") > + return img > + > +def wic_init_parser_cp(subparser): > + subparser.add_argument("src", > + help="image spec: : partition>[] or ") > + subparser.add_argument("dest", > + help="image spec: : partition>[] or ") > + subparser.add_argument("-n", "--native-sysroot", > + help="path to the native sysroot containing > the tools") + > +def wic_init_parser_rm(subparser): > + subparser.add_argument("path", type=imgpathtype, > + help="path: :") > + subparser.add_argument("-n", "--native-sysroot", > + help="path to the native sysroot containing > the tools") > + subparser.add_argument("-r", dest="recursive_delete", > action="store_true", default=False, > + help="remove directories and their contents > recursively, " > + " this only applies to ext* partition") > + > +def expandtype(rules): > + """ > + Custom type for ArgumentParser > + Converts expand rules to the dictionary {: size} > + """ > + if rules == 'auto': > + return {} > + result = {} > + for rule in rules.split(','): > + try: > + part, size = rule.split(':') > + except ValueError: > + raise argparse.ArgumentTypeError("Incorrect rule format: > %s" % rule) + > + if not part.isdigit(): > + raise argparse.ArgumentTypeError("Rule '%s': partition > number must be integer" % rule) + > + # validate size > + multiplier = 1 > + for suffix, mult in [('K', 1024), ('M', 1024 * 1024), ('G', > 1024 * 1024 * 1024)]: > + if size.upper().endswith(suffix): > + multiplier = mult > + size = size[:-1] > + break > + if not size.isdigit(): > + raise argparse.ArgumentTypeError("Rule '%s': size must > be integer" % rule) + > + result[int(part)] = int(size) * multiplier > + > + return result > + > +def wic_init_parser_write(subparser): > + subparser.add_argument("image", > + help="path to the wic image") > + subparser.add_argument("target", > + help="target file or device") > + subparser.add_argument("-e", "--expand", type=expandtype, > + help="expand rules: auto or > :[,:]") > + subparser.add_argument("-n", "--native-sysroot", > + help="path to the native sysroot containing > the tools") + > +def wic_init_parser_help(subparser): > + helpparsers = subparser.add_subparsers(dest='help_topic', > help=hlp.wic_usage) > + for helptopic in helptopics: > + helpparsers.add_parser(helptopic, > help=helptopics[helptopic][2]) > + return > + > + > +subcommands = { > + "create": [wic_create_subcommand, > + hlp.wic_create_usage, > + hlp.wic_create_help, > + wic_init_parser_create], > + "list": [wic_list_subcommand, > + hlp.wic_list_usage, > + hlp.wic_list_help, > + wic_init_parser_list], > + "ls": [wic_ls_subcommand, > + hlp.wic_ls_usage, > + hlp.wic_ls_help, > + wic_init_parser_ls], > + "cp": [wic_cp_subcommand, > + hlp.wic_cp_usage, > + hlp.wic_cp_help, > + wic_init_parser_cp], > + "rm": [wic_rm_subcommand, > + hlp.wic_rm_usage, > + hlp.wic_rm_help, > + wic_init_parser_rm], > + "write": [wic_write_subcommand, > + hlp.wic_write_usage, > + hlp.wic_write_help, > + wic_init_parser_write], > + "help": [wic_help_subcommand, > + wic_help_topic_usage, > + hlp.wic_help_help, > + wic_init_parser_help] > +} > + > + > +def init_parser(parser): > + parser.add_argument("--version", action="version", > + version="%(prog)s {version}".format(version=__version__)) > + parser.add_argument("-D", "--debug", dest="debug", > action="store_true", > + default=False, help="output debug information") > + > + subparsers = parser.add_subparsers(dest='command', > help=hlp.wic_usage) > + for subcmd in subcommands: > + subparser = subparsers.add_parser(subcmd, > help=subcommands[subcmd][2]) > + subcommands[subcmd][3](subparser) > + > +class WicArgumentParser(argparse.ArgumentParser): > + def format_help(self): > + return hlp.wic_help > + > def main(argv): > - parser = optparse.OptionParser(version="wic version %s" % > __version__, > - usage=hlp.wic_usage) > + parser = WicArgumentParser( > + description="wic version %s" % __version__) > + > + init_parser(parser) > > - parser.disable_interspersed_args() > + args = parser.parse_args(argv) > > - args = parser.parse_args(argv)[1] > + if args.debug: > + logger.setLevel(logging.DEBUG) > > - if len(args): > - if args[0] == "help": > - if len(args) == 1: > + if "command" in vars(args): > + if args.command == "help": > + if args.help_topic is None: > parser.print_help() > - raise WicError("help command requires parameter") > + elif args.help_topic in helptopics: > + hlpt = helptopics[args.help_topic] > + hlpt[0](hlpt[1], hlpt[2]) > + return 0 > + > + # validate wic cp src and dest parameter to identify which one > of it is > + # image and cast it into imgtype > + if args.command == "cp": > + if ":" in args.dest: > + args.dest = imgtype(args.dest) > + elif ":" in args.src: > + args.src = imgtype(args.src) > + else: > + raise argparse.ArgumentTypeError("no image or partition > number specified.") > return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, > subcommands)