From: Henning Schild <henning.schild@siemens.com>
To: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
Cc: <isar-users@googlegroups.com>
Subject: Re: [PATCH v2 01/10] wic: Update to the latest wic from openembedded core
Date: Sat, 5 Sep 2020 11:00:55 +0200 [thread overview]
Message-ID: <20200905110055.6d9ce1f1@md1za8fc.ad001.siemens.net> (raw)
In-Reply-To: <20200902185624.15044-2-Vijaikumar_Kanagarajan@mentor.com>
I assume if i took that commit and copied it over i would end up with
100% the same patch, right?
On Thu, 3 Sep 2020 00:26:15 +0530
Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> wrote:
> Update to the latest wic from OE-core.
>
> OE-core Revision: 532ae127c52c9f7b1d2e4ca5cbca91881d23a2ac
>
> Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> ---
> scripts/lib/scriptpath.py | 32 ++
> scripts/lib/wic/__init__.py | 14 +-
> scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> .../directdisk-bootloader-config.cfg | 8 +-
> .../lib/wic/canned-wks/efi-bootdisk.wks.in | 3 +
> scripts/lib/wic/canned-wks/mkhybridiso.wks | 2 +-
> scripts/lib/wic/canned-wks/qemuriscv.wks | 3 +
> .../lib/wic/canned-wks/qemux86-directdisk.wks | 2 +-
> .../lib/wic/canned-wks/sdimage-bootpart.wks | 4 +-
> .../lib/wic/canned-wks/systemd-bootdisk.wks | 4 +-
> scripts/lib/wic/engine.py | 421 +++++++++++++++-
> scripts/lib/wic/filemap.py | 170 ++++---
> scripts/lib/wic/help.py | 401 ++++++++++++++--
> scripts/lib/wic/ksparser.py | 115 +++--
> scripts/lib/wic/{utils => }/misc.py | 99 ++--
> scripts/lib/wic/partition.py | 234 ++++-----
> scripts/lib/wic/pluginbase.py | 36 +-
> scripts/lib/wic/plugins/imager/direct.py | 175 ++++---
> .../wic/plugins/source/bootimg-biosplusefi.py | 213 +++++++++
> scripts/lib/wic/plugins/source/bootimg-efi.py | 111 +++--
> .../wic/plugins/source/bootimg-partition.py | 153 ++++--
> .../lib/wic/plugins/source/bootimg-pcbios.py | 91 ++--
> scripts/lib/wic/plugins/source/fsimage.py | 56 ---
> .../wic/plugins/source/isoimage-isohybrid.py | 185 +++----
> scripts/lib/wic/plugins/source/rawcopy.py | 44 +-
> scripts/lib/wic/plugins/source/rootfs.py | 159 ++++--
> scripts/lib/wic/utils/__init__.py | 0
> scripts/lib/wic/utils/runner.py | 114 -----
> scripts/wic | 452
> +++++++++++++----- 29 files changed, 2283 insertions(+), 1020
> deletions(-) create mode 100644 scripts/lib/scriptpath.py
> create mode 100644 scripts/lib/wic/canned-wks/efi-bootdisk.wks.in
> create mode 100644 scripts/lib/wic/canned-wks/qemuriscv.wks
> rename scripts/lib/wic/{utils => }/misc.py (70%)
> create mode 100644
> scripts/lib/wic/plugins/source/bootimg-biosplusefi.py delete mode
> 100644 scripts/lib/wic/plugins/source/fsimage.py delete mode 100644
> scripts/lib/wic/utils/__init__.py delete mode 100644
> scripts/lib/wic/utils/runner.py
>
> diff --git a/scripts/lib/scriptpath.py b/scripts/lib/scriptpath.py
> new file mode 100644
> index 0000000..f32326d
> --- /dev/null
> +++ b/scripts/lib/scriptpath.py
> @@ -0,0 +1,32 @@
> +# Path utility functions for OE python scripts
> +#
> +# Copyright (C) 2012-2014 Intel Corporation
> +# Copyright (C) 2011 Mentor Graphics Corporation
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +
> +import sys
> +import os
> +import os.path
> +
> +def add_oe_lib_path():
> + basepath = os.path.abspath(os.path.dirname(__file__) + '/../..')
> + newpath = basepath + '/meta/lib'
> + sys.path.insert(0, newpath)
> +
> +def add_bitbake_lib_path():
> + basepath = os.path.abspath(os.path.dirname(__file__) + '/../..')
> + bitbakepath = None
> + if os.path.exists(basepath + '/bitbake/lib/bb'):
> + bitbakepath = basepath + '/bitbake'
> + else:
> + # look for bitbake/bin dir in PATH
> + for pth in os.environ['PATH'].split(':'):
> + if os.path.exists(os.path.join(pth, '../lib/bb')):
> + bitbakepath = os.path.abspath(os.path.join(pth,
> '..'))
> + break
> +
> + if bitbakepath:
> + sys.path.insert(0, bitbakepath + '/lib')
> + return bitbakepath
> diff --git a/scripts/lib/wic/__init__.py b/scripts/lib/wic/__init__.py
> index 85876b1..8556793 100644
> --- a/scripts/lib/wic/__init__.py
> +++ b/scripts/lib/wic/__init__.py
> @@ -1,20 +1,10 @@
> -#!/usr/bin/env python -tt
> +#!/usr/bin/env python3
> #
> # Copyright (c) 2007 Red Hat, Inc.
> # Copyright (c) 2011 Intel, Inc.
> #
> -# This program is free software; you can redistribute it and/or
> modify it -# under the terms of the GNU General Public License as
> published by the Free -# Software Foundation; version 2 of the License
> +# SPDX-License-Identifier: GPL-2.0-only
> #
> -# This program is distributed in the hope that it will be useful, but
> -# WITHOUT ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> General Public License -# for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA
> 02111-1307, USA.
> class WicError(Exception):
> pass
> diff --git a/scripts/lib/wic/canned-wks/common.wks.inc
> b/scripts/lib/wic/canned-wks/common.wks.inc index 5cf2fd1..89880b4
> 100644 --- a/scripts/lib/wic/canned-wks/common.wks.inc
> +++ b/scripts/lib/wic/canned-wks/common.wks.inc
> @@ -1,3 +1,3 @@
> # This file is included into 3 canned wks files from this directory
> part /boot --source bootimg-pcbios --ondisk sda --label boot
> --active --align 1024 -part / --source rootfs --ondisk sda
> --fstype=ext4 --label platform --align 1024 +part / --source rootfs
> --use-uuid --fstype=ext4 --label platform --align 1024 diff --git
> a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg
> b/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg index
> d5a07d2..c58e74a 100644 ---
> a/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg +++
> b/scripts/lib/wic/canned-wks/directdisk-bootloader-config.cfg @@
> -12,16 +12,16 @@ DEFAULT Graphics console boot LABEL Graphics console
> boot KERNEL /vmlinuz
> -APPEND label=boot root=/dev/sda2 rootwait
> +APPEND label=boot rootwait
>
> LABEL Serial console boot
> KERNEL /vmlinuz
> -APPEND label=boot root=/dev/sda2 rootwait console=ttyS0,115200
> +APPEND label=boot rootwait console=ttyS0,115200
>
> LABEL Graphics console install
> KERNEL /vmlinuz
> -APPEND label=install root=/dev/sda2 rootwait
> +APPEND label=install rootwait
>
> LABEL Serial console install
> KERNEL /vmlinuz
> -APPEND label=install root=/dev/sda2 rootwait console=ttyS0,115200
> +APPEND label=install rootwait console=ttyS0,115200
> diff --git a/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in
> b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in new file mode 100644
> index 0000000..7300e65
> --- /dev/null
> +++ b/scripts/lib/wic/canned-wks/efi-bootdisk.wks.in
> @@ -0,0 +1,3 @@
> +bootloader --ptable gpt
> +part /boot --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/boot
> --fstype=vfat --label boot --active --align 1024 --use-uuid
> --overhead-factor 1.0 +part / --source rootfs --fstype=ext4 --label
> root --align 1024 --exclude-path boot/ diff --git
> a/scripts/lib/wic/canned-wks/mkhybridiso.wks
> b/scripts/lib/wic/canned-wks/mkhybridiso.wks index 9d34e9b..48c5ac4
> 100644 --- a/scripts/lib/wic/canned-wks/mkhybridiso.wks +++
> b/scripts/lib/wic/canned-wks/mkhybridiso.wks @@ -2,6 +2,6 @@ #
> long-description: Creates an EFI and legacy bootable hybrid ISO image
> # which can be used on optical media as well as USB media.
> -part /boot --source isoimage-isohybrid
> --sourceparams="loader=grub-efi,image_name=HYBRID_ISO_IMG" --ondisk
> cd --label HYBRIDISO --fstype=ext4 +part /boot --source
> isoimage-isohybrid
> --sourceparams="loader=grub-efi,image_name=HYBRID_ISO_IMG" --ondisk
> cd --label HYBRIDISO bootloader --timeout=15 --append="" diff --git
> a/scripts/lib/wic/canned-wks/qemuriscv.wks
> b/scripts/lib/wic/canned-wks/qemuriscv.wks new file mode 100644 index
> 0000000..12c68b7 --- /dev/null +++
> b/scripts/lib/wic/canned-wks/qemuriscv.wks @@ -0,0 +1,3 @@
> +# short-description: Create qcow2 image for RISC-V QEMU machines
> +
> +part / --source rootfs --fstype=ext4 --label root --align 4096
> --size 5G diff --git
> a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
> b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks index
> a6518a0..22b4521 100644 ---
> a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks +++
> b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks @@ -4,5 +4,5 @@
> include common.wks.inc
>
> -bootloader --timeout=0 --append="vga=0
> uvesafb.mode_option=640x480-32 root=/dev/vda2 rw mem=256M
> ip=192.168.7.2::192.168.7.1:255.255.255.0 oprofile.timer=1
> rootfstype=ext4 " +bootloader --timeout=0 --append="rw
> oprofile.timer=1 rootfstype=ext4 " diff --git
> a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks
> b/scripts/lib/wic/canned-wks/sdimage-bootpart.wks index
> 7ffd632..63bc4da 100644 ---
> a/scripts/lib/wic/canned-wks/sdimage-bootpart.wks +++
> b/scripts/lib/wic/canned-wks/sdimage-bootpart.wks @@ -2,5 +2,5 @@ #
> long-description: Creates a partitioned SD card image. Boot files #
> are located in the first vfat partition. -part /boot --source
> bootimg-partition --ondisk mmcblk --fstype=vfat --label boot --active
> --align 4 --size 16 -part / --source rootfs --ondisk mmcblk
> --fstype=ext4 --label root --align 4 +part /boot --source
> bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot
> --active --align 4 --size 16 +part / --source rootfs --ondisk mmcblk0
> --fstype=ext4 --label root --align 4 diff --git
> a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks
> b/scripts/lib/wic/canned-wks/systemd-bootdisk.wks index
> 4bd9d6a..95d7b97 100644 ---
> a/scripts/lib/wic/canned-wks/systemd-bootdisk.wks +++
> b/scripts/lib/wic/canned-wks/systemd-bootdisk.wks @@ -2,10 +2,10 @@ #
> long-description: Creates a partitioned EFI disk image that the user
> # can directly dd to boot media. The selected bootloader is
> systemd-boot. -part /boot --source bootimg-efi
> --sourceparams="loader=systemd-boot" --ondisk sda --label msdos
> --active --align 1024 +part /boot --source bootimg-efi
> --sourceparams="loader=systemd-boot" --ondisk sda --label msdos
> --active --align 1024 --use-uuid part / --source rootfs --ondisk sda
> --fstype=ext4 --label platform --align 1024 --use-uuid -part swap
> --ondisk sda --size 44 --label swap1 --fstype=swap +part swap
> --ondisk sda --size 44 --label swap1 --fstype=swap --use-uuid
> bootloader --ptable gpt --timeout=5 --append="rootwait
> rootfstype=ext4 console=ttyS0,115200 console=tty0" diff --git
> a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py index
> f59821f..018815b 100644 --- a/scripts/lib/wic/engine.py +++
> b/scripts/lib/wic/engine.py @@ -1,21 +1,7 @@ -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2013, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
>
> @@ -30,10 +16,18 @@
>
> import logging
> import os
> +import tempfile
> +import json
> +import subprocess
> +import re
> +
> +from collections import namedtuple, OrderedDict
> +from distutils.spawn import find_executable
>
> from wic import WicError
> +from wic.filemap import sparse_copy
> from wic.pluginbase import PluginMgr
> -from wic.utils.misc import get_bitbake_var
> +from wic.misc import get_bitbake_var, exec_cmd
>
> logger = logging.getLogger('wic')
>
> @@ -82,7 +76,8 @@ def find_canned_image(scripts_path, wks_file):
> for fname in files:
> if fname.endswith("~") or fname.endswith("#"):
> continue
> - if fname.endswith(".wks") and wks_file + ".wks" ==
> fname:
> + if ((fname.endswith(".wks") and wks_file + ".wks" ==
> fname) or \
> + (fname.endswith(".wks.in") and wks_file +
> ".wks.in" == fname)): fullpath = os.path.join(canned_wks_dir, fname)
> return fullpath
> return None
> @@ -99,7 +94,7 @@ def list_canned_images(scripts_path):
> for fname in files:
> if fname.endswith("~") or fname.endswith("#"):
> continue
> - if fname.endswith(".wks"):
> + if fname.endswith(".wks") or
> fname.endswith(".wks.in"): fullpath = os.path.join(canned_wks_dir,
> fname) with open(fullpath) as wks:
> for line in wks:
> @@ -108,7 +103,7 @@ def list_canned_images(scripts_path):
> if idx != -1:
> desc = line[idx +
> len("short-description:"):].strip() break
> - basename = os.path.splitext(fname)[0]
> + basename = fname.split('.')[0]
> print(" %s\t\t%s" % (basename.ljust(30), desc))
>
>
> @@ -184,7 +179,7 @@ def wic_create(wks_file, rootfs_dir, bootimg_dir,
> kernel_dir, if not os.path.exists(options.outdir):
> os.makedirs(options.outdir)
>
> - pname = 'direct'
> + pname = options.imager
> plugin_class = PluginMgr.get_plugins('imager').get(pname)
> if not plugin_class:
> raise WicError('Unknown plugin: %s' % pname)
> @@ -201,17 +196,18 @@ def wic_list(args, scripts_path):
> """
> Print the list of images or source plugins.
> """
> - if len(args) < 1:
> + if args.list_type is None:
> return False
>
> - if args == ["images"]:
> + if args.list_type == "images":
> +
> list_canned_images(scripts_path)
> return True
> - elif args == ["source-plugins"]:
> + elif args.list_type == "source-plugins":
> list_source_plugins()
> return True
> - elif len(args) == 2 and args[1] == "help":
> - wks_file = args[0]
> + elif len(args.help_for) == 1 and args.help_for[0] == 'help':
> + wks_file = args.list_type
> fullpath = find_canned_image(scripts_path, wks_file)
> if not fullpath:
> raise WicError("No image named %s found, exiting. "
> @@ -224,6 +220,381 @@ def wic_list(args, scripts_path):
>
> return False
>
> +
> +class Disk:
> + def __init__(self, imagepath, native_sysroot, fstypes=('fat',
> 'ext')):
> + self.imagepath = imagepath
> + self.native_sysroot = native_sysroot
> + self.fstypes = fstypes
> + self._partitions = None
> + self._partimages = {}
> + self._lsector_size = None
> + self._psector_size = None
> + self._ptable_format = None
> +
> + # find parted
> + # read paths from $PATH environment variable
> + # if it fails, use hardcoded paths
> + pathlist = "/bin:/usr/bin:/usr/sbin:/sbin/"
> + try:
> + self.paths = os.environ['PATH'] + ":" + pathlist
> + except KeyError:
> + self.paths = pathlist
> +
> + if native_sysroot:
> + for path in pathlist.split(':'):
> + self.paths = "%s%s:%s" % (native_sysroot, path,
> self.paths) +
> + self.parted = find_executable("parted", self.paths)
> + if not self.parted:
> + raise WicError("Can't find executable parted")
> +
> + self.partitions = self.get_partitions()
> +
> + def __del__(self):
> + for path in self._partimages.values():
> + os.unlink(path)
> +
> + def get_partitions(self):
> + if self._partitions is None:
> + self._partitions = OrderedDict()
> + out = exec_cmd("%s -sm %s unit B print" % (self.parted,
> self.imagepath))
> + parttype = namedtuple("Part", "pnum start end size
> fstype")
> + splitted = out.splitlines()
> + # skip over possible errors in exec_cmd output
> + try:
> + idx =splitted.index("BYT;")
> + except ValueError:
> + raise WicError("Error getting partition information
> from %s" % (self.parted))
> + lsector_size, psector_size, self._ptable_format =
> splitted[idx + 1].split(":")[3:6]
> + self._lsector_size = int(lsector_size)
> + self._psector_size = int(psector_size)
> + for line in splitted[idx + 2:]:
> + pnum, start, end, size, fstype = line.split(':')[:5]
> + partition = parttype(int(pnum), int(start[:-1]),
> int(end[:-1]),
> + int(size[:-1]), fstype)
> + self._partitions[pnum] = partition
> +
> + return self._partitions
> +
> + def __getattr__(self, name):
> + """Get path to the executable in a lazy way."""
> + if name in ("mdir", "mcopy", "mdel", "mdeltree", "sfdisk",
> "e2fsck",
> + "resize2fs", "mkswap", "mkdosfs",
> "debugfs","blkid"):
> + aname = "_%s" % name
> + if aname not in self.__dict__:
> + setattr(self, aname, find_executable(name,
> self.paths))
> + if aname not in self.__dict__ or
> self.__dict__[aname] is None:
> + raise WicError("Can't find executable
> '{}'".format(name))
> + return self.__dict__[aname]
> + return self.__dict__[name]
> +
> + def _get_part_image(self, pnum):
> + if pnum not in self.partitions:
> + raise WicError("Partition %s is not in the image" % pnum)
> + part = self.partitions[pnum]
> + # check if fstype is supported
> + for fstype in self.fstypes:
> + if part.fstype.startswith(fstype):
> + break
> + else:
> + raise WicError("Not supported fstype:
> {}".format(part.fstype))
> + if pnum not in self._partimages:
> + tmpf = tempfile.NamedTemporaryFile(prefix="wic-part")
> + dst_fname = tmpf.name
> + tmpf.close()
> + sparse_copy(self.imagepath, dst_fname, skip=part.start,
> length=part.size)
> + self._partimages[pnum] = dst_fname
> +
> + return self._partimages[pnum]
> +
> + def _put_part_image(self, pnum):
> + """Put partition image into partitioned image."""
> + sparse_copy(self._partimages[pnum], self.imagepath,
> + seek=self.partitions[pnum].start)
> +
> + def dir(self, pnum, path):
> + if pnum not in self.partitions:
> + raise WicError("Partition %s is not in the image" % pnum)
> +
> + if self.partitions[pnum].fstype.startswith('ext'):
> + return exec_cmd("{} {} -R 'ls -l
> {}'".format(self.debugfs,
> +
> self._get_part_image(pnum),
> + path),
> as_shell=True)
> + else: # fat
> + return exec_cmd("{} -i {} ::{}".format(self.mdir,
> +
> self._get_part_image(pnum),
> + path))
> +
> + def copy(self, src, dest):
> + """Copy partition image into wic image."""
> + pnum = dest.part if isinstance(src, str) else src.part
> +
> + if self.partitions[pnum].fstype.startswith('ext'):
> + if isinstance(src, str):
> + cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\
> + format(os.path.dirname(dest.path), src,
> os.path.basename(src),
> + self.debugfs,
> self._get_part_image(pnum))
> + else: # copy from wic
> + # run both dump and rdump to support both files and
> directory
> + cmd = "printf 'cd {}\ndump /{} {}\nrdump /{} {}\n' |
> {} {}".\
> + format(os.path.dirname(src.path), src.path,
> + dest, src.path, dest, self.debugfs,
> + self._get_part_image(pnum))
> + else: # fat
> + if isinstance(src, str):
> + cmd = "{} -i {} -snop {} ::{}".format(self.mcopy,
> +
> self._get_part_image(pnum),
> + src, dest.path)
> + else:
> + cmd = "{} -i {} -snop ::{} {}".format(self.mcopy,
> +
> self._get_part_image(pnum),
> + src.path, dest)
> +
> + exec_cmd(cmd, as_shell=True)
> + self._put_part_image(pnum)
> +
> + def remove_ext(self, pnum, path, recursive):
> + """
> + Remove files/dirs and their contents from the partition.
> + This only applies to ext* partition.
> + """
> + abs_path = re.sub('\/\/+', '/', path)
> + cmd = "{} {} -wR 'rm \"{}\"'".format(self.debugfs,
> +
> self._get_part_image(pnum),
> + abs_path)
> + out = exec_cmd(cmd , as_shell=True)
> + for line in out.splitlines():
> + if line.startswith("rm:"):
> + if "file is a directory" in line:
> + if recursive:
> + # loop through content and delete them one
> by one if
> + # flaged with -r
> + subdirs = iter(self.dir(pnum,
> abs_path).splitlines())
> + next(subdirs)
> + for subdir in subdirs:
> + dir = subdir.split(':')[1].split(" ",
> 1)[1]
> + if not dir == "." and not dir == "..":
> + self.remove_ext(pnum, "%s/%s" %
> (abs_path, dir), recursive) +
> + rmdir_out = exec_cmd("{} {} -wR 'rmdir
> \"{}\"'".format(self.debugfs,
> +
> self._get_part_image(pnum),
> +
> abs_path.rstrip('/'))
> + , as_shell=True)
> +
> + for rmdir_line in rmdir_out.splitlines():
> + if "directory not empty" in rmdir_line:
> + raise WicError("Could not complete
> operation: \n%s \n"
> + "use -r to remove
> non-empty directory" % rmdir_line)
> + if rmdir_line.startswith("rmdir:"):
> + raise WicError("Could not complete
> operation: \n%s "
> + "\n%s" % (str(line),
> rmdir_line)) +
> + else:
> + raise WicError("Could not complete operation:
> \n%s "
> + "\nUnable to remove %s" %
> (str(line), abs_path)) +
> + def remove(self, pnum, path, recursive):
> + """Remove files/dirs from the partition."""
> + partimg = self._get_part_image(pnum)
> + if self.partitions[pnum].fstype.startswith('ext'):
> + self.remove_ext(pnum, path, recursive)
> +
> + else: # fat
> + cmd = "{} -i {} ::{}".format(self.mdel, partimg, path)
> + try:
> + exec_cmd(cmd)
> + except WicError as err:
> + if "not found" in str(err) or "non empty" in
> str(err):
> + # mdel outputs 'File ... not found' or
> 'directory .. non empty"
> + # try to use mdeltree as path could be a
> directory
> + cmd = "{} -i {} ::{}".format(self.mdeltree,
> + partimg, path)
> + exec_cmd(cmd)
> + else:
> + raise err
> + self._put_part_image(pnum)
> +
> + def write(self, target, expand):
> + """Write disk image to the media or file."""
> + def write_sfdisk_script(outf, parts):
> + for key, val in parts['partitiontable'].items():
> + if key in ("partitions", "device", "firstlba",
> "lastlba"):
> + continue
> + if key == "id":
> + key = "label-id"
> + outf.write("{}: {}\n".format(key, val))
> + outf.write("\n")
> + for part in parts['partitiontable']['partitions']:
> + line = ''
> + for name in ('attrs', 'name', 'size', 'type',
> 'uuid'):
> + if name == 'size' and part['type'] == 'f':
> + # don't write size for extended partition
> + continue
> + val = part.get(name)
> + if val:
> + line += '{}={}, '.format(name, val)
> + if line:
> + line = line[:-2] # strip ', '
> + if part.get('bootable'):
> + line += ' ,bootable'
> + outf.write("{}\n".format(line))
> + outf.flush()
> +
> + def read_ptable(path):
> + out = exec_cmd("{} -J {}".format(self.sfdisk, path))
> + return json.loads(out)
> +
> + def write_ptable(parts, target):
> + with tempfile.NamedTemporaryFile(prefix="wic-sfdisk-",
> mode='w') as outf:
> + write_sfdisk_script(outf, parts)
> + cmd = "{} --no-reread {} < {} ".format(self.sfdisk,
> target, outf.name)
> + exec_cmd(cmd, as_shell=True)
> +
> + if expand is None:
> + sparse_copy(self.imagepath, target)
> + else:
> + # copy first sectors that may contain bootloader
> + sparse_copy(self.imagepath, target, length=2048 *
> self._lsector_size) +
> + # copy source partition table to the target
> + parts = read_ptable(self.imagepath)
> + write_ptable(parts, target)
> +
> + # get size of unpartitioned space
> + free = None
> + for line in exec_cmd("{} -F {}".format(self.sfdisk,
> target)).splitlines():
> + if line.startswith("Unpartitioned space ") and
> line.endswith("sectors"):
> + free = int(line.split()[-2])
> + # Align free space to a 2048 sector boundary.
> YOCTO #12840.
> + free = free - (free % 2048)
> + if free is None:
> + raise WicError("Can't get size of unpartitioned
> space") +
> + # calculate expanded partitions sizes
> + sizes = {}
> + num_auto_resize = 0
> + for num, part in
> enumerate(parts['partitiontable']['partitions'], 1):
> + if num in expand:
> + if expand[num] != 0: # don't resize partition if
> size is set to 0
> + sectors = expand[num] // self._lsector_size
> + free -= sectors - part['size']
> + part['size'] = sectors
> + sizes[num] = sectors
> + elif part['type'] != 'f':
> + sizes[num] = -1
> + num_auto_resize += 1
> +
> + for num, part in
> enumerate(parts['partitiontable']['partitions'], 1):
> + if sizes.get(num) == -1:
> + part['size'] += free // num_auto_resize
> +
> + # write resized partition table to the target
> + write_ptable(parts, target)
> +
> + # read resized partition table
> + parts = read_ptable(target)
> +
> + # copy partitions content
> + for num, part in
> enumerate(parts['partitiontable']['partitions'], 1):
> + pnum = str(num)
> + fstype = self.partitions[pnum].fstype
> +
> + # copy unchanged partition
> + if part['size'] == self.partitions[pnum].size //
> self._lsector_size:
> + logger.info("copying unchanged partition
> {}".format(pnum))
> + sparse_copy(self._get_part_image(pnum), target,
> seek=part['start'] * self._lsector_size)
> + continue
> +
> + # resize or re-create partitions
> + if fstype.startswith('ext') or
> fstype.startswith('fat') or \
> + fstype.startswith('linux-swap'):
> +
> + partfname = None
> + with
> tempfile.NamedTemporaryFile(prefix="wic-part{}-".format(pnum)) as
> partf:
> + partfname = partf.name
> +
> + if fstype.startswith('ext'):
> + logger.info("resizing ext partition
> {}".format(pnum))
> + partimg = self._get_part_image(pnum)
> + sparse_copy(partimg, partfname)
> + exec_cmd("{} -pf {}".format(self.e2fsck,
> partfname))
> + exec_cmd("{} {} {}s".format(\
> + self.resize2fs, partfname,
> part['size']))
> + elif fstype.startswith('fat'):
> + logger.info("copying content of the fat
> partition {}".format(pnum))
> + with
> tempfile.TemporaryDirectory(prefix='wic-fatdir-') as tmpdir:
> + # copy content to the temporary directory
> + cmd = "{} -snompi {} ::
> {}".format(self.mcopy,
> +
> self._get_part_image(pnum),
> +
> tmpdir)
> + exec_cmd(cmd)
> + # create new msdos partition
> + label = part.get("name")
> + label_str = "-n {}".format(label) if
> label else '' +
> + cmd = "{} {} -C {}
> {}".format(self.mkdosfs, label_str, partfname,
> +
> part['size'])
> + exec_cmd(cmd)
> + # copy content from the temporary
> directory to the new partition
> + cmd = "{} -snompi {} {}/*
> ::".format(self.mcopy, partfname, tmpdir)
> + exec_cmd(cmd, as_shell=True)
> + elif fstype.startswith('linux-swap'):
> + logger.info("creating swap partition
> {}".format(pnum))
> + label = part.get("name")
> + label_str = "-L {}".format(label) if label
> else ''
> + out = exec_cmd("{} --probe
> {}".format(self.blkid, self._get_part_image(pnum)))
> + uuid =
> out[out.index("UUID=\"")+6:out.index("UUID=\"")+42]
> + uuid_str = "-U {}".format(uuid) if uuid else
> ''
> + with open(partfname, 'w') as sparse:
> + os.ftruncate(sparse.fileno(),
> part['size'] * self._lsector_size)
> + exec_cmd("{} {} {} {}".format(self.mkswap,
> label_str, uuid_str, partfname))
> + sparse_copy(partfname, target,
> seek=part['start'] * self._lsector_size)
> + os.unlink(partfname)
> + elif part['type'] != 'f':
> + logger.warning("skipping partition {}:
> unsupported fstype {}".format(pnum, fstype)) +
> +def wic_ls(args, native_sysroot):
> + """List contents of partitioned image or vfat partition."""
> + disk = Disk(args.path.image, native_sysroot)
> + if not args.path.part:
> + if disk.partitions:
> + print('Num Start End Size
> Fstype')
> + for part in disk.partitions.values():
> + print("{:2d} {:12d} {:12d} {:12d} {}".format(\
> + part.pnum, part.start, part.end,
> + part.size, part.fstype))
> + else:
> + path = args.path.path or '/'
> + print(disk.dir(args.path.part, path))
> +
> +def wic_cp(args, native_sysroot):
> + """
> + Copy file or directory to/from the vfat/ext partition of
> + partitioned image.
> + """
> + if isinstance(args.dest, str):
> + disk = Disk(args.src.image, native_sysroot)
> + else:
> + disk = Disk(args.dest.image, native_sysroot)
> + disk.copy(args.src, args.dest)
> +
> +
> +def wic_rm(args, native_sysroot):
> + """
> + Remove files or directories from the vfat partition of
> + partitioned image.
> + """
> + disk = Disk(args.path.image, native_sysroot)
> + disk.remove(args.path.part, args.path.path,
> args.recursive_delete) +
> +def wic_write(args, native_sysroot):
> + """
> + Write image to a target device.
> + """
> + disk = Disk(args.image, native_sysroot, ('fat', 'ext',
> 'linux-swap'))
> + disk.write(args.target, args.expand)
> +
> def find_canned(scripts_path, file_name):
> """
> Find a file either by its path or by name in the canned files
> dir. diff --git a/scripts/lib/wic/filemap.py
> b/scripts/lib/wic/filemap.py index 080668e..4d9da28 100644
> --- a/scripts/lib/wic/filemap.py
> +++ b/scripts/lib/wic/filemap.py
> @@ -1,13 +1,8 @@
> +#
> # Copyright (c) 2012 Intel, Inc.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License,
> version 2, -# as published by the Free Software Foundation.
> +# SPDX-License-Identifier: GPL-2.0-only
> #
> -# This program is distributed in the hope that it will be useful, but
> -# WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> -# General Public License for more details.
>
> """
> This module implements python implements a way to get file block.
> Two methods @@ -22,6 +17,7 @@ and returns an instance of the class.
> # * Too many instance attributes (R0902)
> # pylint: disable=R0902
>
> +import errno
> import os
> import struct
> import array
> @@ -34,14 +30,23 @@ def get_block_size(file_obj):
> Returns block size for file object 'file_obj'. Errors are
> indicated by the 'IOError' exception.
> """
> -
> - from fcntl import ioctl
> - import struct
> -
> # Get the block size of the host file-system for the image file
> by calling # the FIGETBSZ ioctl (number 2).
> - binary_data = ioctl(file_obj, 2, struct.pack('I', 0))
> - return struct.unpack('I', binary_data)[0]
> + try:
> + binary_data = fcntl.ioctl(file_obj, 2, struct.pack('I', 0))
> + bsize = struct.unpack('I', binary_data)[0]
> + except OSError:
> + bsize = None
> +
> + # If ioctl causes OSError or give bsize to zero failback to
> os.fstat
> + if not bsize:
> + import os
> + stat = os.fstat(file_obj.fileno())
> + if hasattr(stat, 'st_blksize'):
> + bsize = stat.st_blksize
> + else:
> + raise IOError("Unable to determine block size")
> + return bsize
>
> class ErrorNotSupp(Exception):
> """
> @@ -137,15 +142,6 @@ class _FilemapBase(object):
>
> raise Error("the method is not implemented")
>
> - def block_is_unmapped(self, block): # pylint: disable=W0613,R0201
> - """
> - This method has has to be implemented by child classes. It
> returns
> - 'True' if block number 'block' of the image file is not
> mapped (hole)
> - and 'False' otherwise.
> - """
> -
> - raise Error("the method is not implemented")
> -
> def get_mapped_ranges(self, start, count): # pylint:
> disable=W0613,R0201 """
> This method has has to be implemented by child classes. This
> is a @@ -159,15 +155,6 @@ class _FilemapBase(object):
>
> raise Error("the method is not implemented")
>
> - def get_unmapped_ranges(self, start, count): # pylint:
> disable=W0613,R0201
> - """
> - This method has has to be implemented by child classes. Just
> like
> - 'get_mapped_ranges()', but yields unmapped block ranges
> instead
> - (holes).
> - """
> -
> - raise Error("the method is not implemented")
> -
>
> # The 'SEEK_HOLE' and 'SEEK_DATA' options of the file seek system
> call _SEEK_DATA = 3
> @@ -185,9 +172,9 @@ def _lseek(file_obj, offset, whence):
> except OSError as err:
> # The 'lseek' system call returns the ENXIO if there is no
> data or # hole starting from the specified offset.
> - if err.errno == os.errno.ENXIO:
> + if err.errno == errno.ENXIO:
> return -1
> - elif err.errno == os.errno.EINVAL:
> + elif err.errno == errno.EINVAL:
> raise ErrorNotSupp("the kernel or file-system does not
> support " "\"SEEK_HOLE\" and \"SEEK_DATA\"")
> else:
> @@ -228,7 +215,7 @@ class FilemapSeek(_FilemapBase):
> try:
> tmp_obj = tempfile.TemporaryFile("w+", dir=directory)
> except IOError as err:
> - raise ErrorNotSupp("cannot create a temporary in \"%s\":
> %s"
> + raise ErrorNotSupp("cannot create a temporary in \"%s\":
> %s" \ % (directory, err))
>
> try:
> @@ -260,15 +247,10 @@ class FilemapSeek(_FilemapBase):
> % (block, result))
> return result
>
> - def block_is_unmapped(self, block):
> - """Refer the '_FilemapBase' class for the documentation."""
> - return not self.block_is_mapped(block)
> -
> def _get_ranges(self, start, count, whence1, whence2):
> """
> - This function implements 'get_mapped_ranges()' and
> - 'get_unmapped_ranges()' depending on what is passed in the
> 'whence1'
> - and 'whence2' arguments.
> + This function implements 'get_mapped_ranges()' depending
> + on what is passed in the 'whence1' and 'whence2' arguments.
> """
>
> assert whence1 != whence2
> @@ -298,12 +280,6 @@ class FilemapSeek(_FilemapBase):
> % (start, count, start + count - 1))
> return self._get_ranges(start, count, _SEEK_DATA, _SEEK_HOLE)
>
> - def get_unmapped_ranges(self, start, count):
> - """Refer the '_FilemapBase' class for the documentation."""
> - self._log.debug("FilemapSeek: get_unmapped_ranges(%d,
> %d(%d))"
> - % (start, count, start + count - 1))
> - return self._get_ranges(start, count, _SEEK_HOLE, _SEEK_DATA)
> -
>
> # Below goes the FIEMAP ioctl implementation, which is not very
> readable # because it deals with the rather complex FIEMAP ioctl. To
> understand the @@ -390,12 +366,12 @@ class
> FilemapFiemap(_FilemapBase): except IOError as err:
> # Note, the FIEMAP ioctl is supported by the Linux
> kernel starting # from version 2.6.28 (year 2008).
> - if err.errno == os.errno.EOPNOTSUPP:
> + if err.errno == errno.EOPNOTSUPP:
> errstr = "FilemapFiemap: the FIEMAP ioctl is not
> supported " \ "by the file-system"
> self._log.debug(errstr)
> raise ErrorNotSupp(errstr)
> - if err.errno == os.errno.ENOTTY:
> + if err.errno == errno.ENOTTY:
> errstr = "FilemapFiemap: the FIEMAP ioctl is not
> supported " \ "by the kernel"
> self._log.debug(errstr)
> @@ -417,10 +393,6 @@ class FilemapFiemap(_FilemapBase):
> % (block, result))
> return result
>
> - def block_is_unmapped(self, block):
> - """Refer the '_FilemapBase' class for the documentation."""
> - return not self.block_is_mapped(block)
> -
> def _unpack_fiemap_extent(self, index):
> """
> Unpack a 'struct fiemap_extent' structure object number
> 'index' from @@ -497,23 +469,28 @@ class FilemapFiemap(_FilemapBase):
> % (first_prev, last_prev))
> yield (first_prev, last_prev)
>
> - def get_unmapped_ranges(self, start, count):
> +class FilemapNobmap(_FilemapBase):
> + """
> + This class is used when both the 'SEEK_DATA/HOLE' and FIEMAP are
> not
> + supported by the filesystem or kernel.
> + """
> +
> + def __init__(self, image, log=None):
> """Refer the '_FilemapBase' class for the documentation."""
> - self._log.debug("FilemapFiemap: get_unmapped_ranges(%d,
> %d(%d))"
> - % (start, count, start + count - 1))
> - hole_first = start
> - for first, last in self._do_get_mapped_ranges(start, count):
> - if first > hole_first:
> - self._log.debug("FilemapFiemap: yielding range (%d,
> %d)"
> - % (hole_first, first - 1))
> - yield (hole_first, first - 1)
>
> - hole_first = last + 1
> + # Call the base class constructor first
> + _FilemapBase.__init__(self, image, log)
> + self._log.debug("FilemapNobmap: initializing")
>
> - if hole_first < start + count:
> - self._log.debug("FilemapFiemap: yielding range (%d, %d)"
> - % (hole_first, start + count - 1))
> - yield (hole_first, start + count - 1)
> + def block_is_mapped(self, block):
> + """Refer the '_FilemapBase' class for the documentation."""
> + return True
> +
> + def get_mapped_ranges(self, start, count):
> + """Refer the '_FilemapBase' class for the documentation."""
> + self._log.debug("FilemapNobmap: get_mapped_ranges(%d,
> %d(%d))"
> + % (start, count, start + count - 1))
> + yield (start, start + count -1)
>
> def filemap(image, log=None):
> """
> @@ -528,26 +505,56 @@ def filemap(image, log=None):
> try:
> return FilemapFiemap(image, log)
> except ErrorNotSupp:
> - return FilemapSeek(image, log)
> + try:
> + return FilemapSeek(image, log)
> + except ErrorNotSupp:
> + return FilemapNobmap(image, log)
>
> -def sparse_copy(src_fname, dst_fname, offset=0, skip=0):
> - """Efficiently copy sparse file to or into another file."""
> - fmap = filemap(src_fname)
> +def sparse_copy(src_fname, dst_fname, skip=0, seek=0,
> + length=0, api=None):
> + """
> + Efficiently copy sparse file to or into another file.
> +
> + src_fname: path to source file
> + dst_fname: path to destination file
> + skip: skip N bytes at thestart of src
> + seek: seek N bytes from the start of dst
> + length: read N bytes from src and write them to dst
> + api: FilemapFiemap or FilemapSeek object
> + """
> + if not api:
> + api = filemap
> + fmap = api(src_fname)
> try:
> dst_file = open(dst_fname, 'r+b')
> except IOError:
> dst_file = open(dst_fname, 'wb')
> - dst_file.truncate(os.path.getsize(src_fname))
> + if length:
> + dst_size = length + seek
> + else:
> + dst_size = os.path.getsize(src_fname) + seek - skip
> + dst_file.truncate(dst_size)
>
> + written = 0
> for first, last in fmap.get_mapped_ranges(0, fmap.blocks_cnt):
> start = first * fmap.block_size
> end = (last + 1) * fmap.block_size
>
> + if skip >= end:
> + continue
> +
> if start < skip < end:
> - fmap._f_image.seek(skip, os.SEEK_SET)
> - else:
> - fmap._f_image.seek(start, os.SEEK_SET)
> - dst_file.seek(offset + start, os.SEEK_SET)
> + start = skip
> +
> + fmap._f_image.seek(start, os.SEEK_SET)
> +
> + written += start - skip - written
> + if length and written >= length:
> + dst_file.seek(seek + length, os.SEEK_SET)
> + dst_file.close()
> + return
> +
> + dst_file.seek(seek + start - skip, os.SEEK_SET)
>
> chunk_size = 1024 * 1024
> to_read = end - start
> @@ -556,7 +563,14 @@ def sparse_copy(src_fname, dst_fname, offset=0,
> skip=0): while read < to_read:
> if read + chunk_size > to_read:
> chunk_size = to_read - read
> - chunk = fmap._f_image.read(chunk_size)
> + size = chunk_size
> + if length and written + size > length:
> + size = length - written
> + chunk = fmap._f_image.read(size)
> dst_file.write(chunk)
> - read += chunk_size
> + read += size
> + written += size
> + if written == length:
> + dst_file.close()
> + return
> dst_file.close()
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index 148da89..bd3a2b9 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -1,21 +1,6 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> -#
> # Copyright (c) 2013, Intel Corporation.
> -# All rights reserved.
> -#
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> #
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This module implements some basic help invocation functions along
> @@ -56,7 +41,7 @@ def wic_help(args, usage_str, subcommands):
> """
> Subcommand help dispatcher.
> """
> - if len(args) == 1 or not display_help(args[1], subcommands):
> + if args.help_topic == None or not display_help(args.help_topic,
> subcommands): print(usage_str)
>
>
> @@ -82,19 +67,20 @@ def invoke_subcommand(args, parser,
> main_command_usage, subcommands): Dispatch to subcommand handler
> borrowed from combo-layer. Should use argparse, but has to work in
> 2.6. """
> - if not args:
> + if not args.command:
> logger.error("No subcommand specified, exiting")
> parser.print_help()
> return 1
> - elif args[0] == "help":
> + elif args.command == "help":
> wic_help(args, main_command_usage, subcommands)
> - elif args[0] not in subcommands:
> - logger.error("Unsupported subcommand %s, exiting\n", args[0])
> + elif args.command not in subcommands:
> + logger.error("Unsupported subcommand %s, exiting\n",
> args.command) parser.print_help()
> return 1
> else:
> - usage = subcommands.get(args[0], subcommand_error)[1]
> - subcommands.get(args[0], subcommand_error)[0](args[1:],
> usage)
> + subcmd = subcommands.get(args.command, subcommand_error)
> + usage = subcmd[1]
> + subcmd[0](args, usage)
>
>
> ##
> @@ -130,10 +116,10 @@ wic_create_usage = """
> Create a new OpenEmbedded image
>
> usage: wic create <wks file or image name> [-o <DIRNAME> | --outdir
> <DIRNAME>]
> - [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
> [-e | --image-name] [-s, --skip-build-check] [-D,
> --debug] [-r, --rootfs-dir] [-b, --bootimg-dir]
> [-k, --kernel-dir] [-n, --native-sysroot] [-f,
> --build-rootfs]
> + [-c, --compress-with] [-m, --bmap]
>
> This command creates an OpenEmbedded image based on the 'OE
> kickstart commands' found in the <wks file>.
> @@ -154,7 +140,7 @@ SYNOPSIS
> [-e | --image-name] [-s, --skip-build-check] [-D, --debug]
> [-r, --rootfs-dir] [-b, --bootimg-dir]
> [-k, --kernel-dir] [-n, --native-sysroot] [-f,
> --build-rootfs]
> - [-c, --compress-with] [-m, --bmap]
> + [-c, --compress-with] [-m, --bmap] [--no-fstab-update]
>
> DESCRIPTION
> This command creates an OpenEmbedded image based on the 'OE
> @@ -226,6 +212,11 @@ DESCRIPTION
>
> The -m option is used to produce .bmap file for the image. This
> file can be used to flash image using bmaptool utility.
> +
> + The --no-fstab-update option is used to doesn't change fstab
> file. When
> + using this option the final fstab file will be same that in
> rootfs and
> + wic doesn't update file, e.g adding a new mount point. User can
> control
> + the fstab file content in base-files recipe.
> """
>
> wic_list_usage = """
> @@ -283,6 +274,243 @@ DESCRIPTION
> details.
> """
>
> +wic_ls_usage = """
> +
> + List content of a partitioned image
> +
> + usage: wic ls <image>[:<partition>[<path>]] [--native-sysroot
> <path>] +
> + This command outputs either list of image partitions or directory
> contents
> + of vfat and ext* partitions.
> +
> + See 'wic help ls' for more detailed instructions.
> +
> +"""
> +
> +wic_ls_help = """
> +
> +NAME
> + wic ls - List contents of partitioned image or partition
> +
> +SYNOPSIS
> + wic ls <image>
> + wic ls <image>:<vfat or ext* partition>
> + wic ls <image>:<vfat or ext* partition><path>
> + wic ls <image>:<vfat or ext* partition><path> --native-sysroot
> <path> +
> +DESCRIPTION
> + This command lists either partitions of the image or directory
> contents
> + of vfat or ext* partitions.
> +
> + The first form it lists partitions of the image.
> + For example:
> + $ wic ls
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic
> + Num Start End Size Fstype
> + 1 1048576 24438783 23390208 fat16
> + 2 25165824 50315263 25149440 ext4
> +
> + Second and third form list directory content of the partition:
> + $ wic ls
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
> + Volume in drive : is boot
> + Volume Serial Number is 2DF2-5F02
> + Directory for ::/
> +
> + efi <DIR> 2017-05-11 10:54
> + startup nsh 26 2017-05-11 10:54
> + vmlinuz 6922288 2017-05-11 10:54
> + 3 files 6 922 314 bytes
> + 15 818 752 bytes free
> +
> +
> + $ wic ls
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/EFI/boot/
> + Volume in drive : is boot
> + Volume Serial Number is 2DF2-5F02
> + Directory for ::/EFI/boot
> +
> + . <DIR> 2017-05-11 10:54
> + .. <DIR> 2017-05-11 10:54
> + grub cfg 679 2017-05-11 10:54
> + bootx64 efi 571392 2017-05-11 10:54
> + 4 files 572 071 bytes
> + 15 818 752 bytes free
> +
> + The -n option is used to specify the path to the native sysroot
> + containing the tools(parted and mtools) to use.
> +
> +"""
> +
> +wic_cp_usage = """
> +
> + Copy files and directories to/from the vfat or ext* partition
> +
> + usage: wic cp <src> <dest> [--native-sysroot <path>]
> +
> + source/destination image in format <image>:<partition>[<path>]
> +
> + This command copies files or directories either
> + - from local to vfat or ext* partitions of partitioned image
> + - from vfat or ext* partitions of partitioned image to local
> +
> + See 'wic help cp' for more detailed instructions.
> +
> +"""
> +
> +wic_cp_help = """
> +
> +NAME
> + wic cp - copy files and directories to/from the vfat or ext*
> partitions +
> +SYNOPSIS
> + wic cp <src> <dest>:<partition>
> + wic cp <src>:<partition> <dest>
> + wic cp <src> <dest-image>:<partition><path>
> + wic cp <src> <dest-image>:<partition><path> --native-sysroot
> <path> +
> +DESCRIPTION
> + This command copies files or directories either
> + - from local to vfat or ext* partitions of partitioned image
> + - from vfat or ext* partitions of partitioned image to local
> +
> + The first form of it copies file or directory to the root
> directory of
> + the partition:
> + $ wic cp test.wks
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
> + $ wic ls
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
> + Volume in drive : is boot
> + Volume Serial Number is DB4C-FD4C
> + Directory for ::/
> +
> + efi <DIR> 2017-05-24 18:15
> + loader <DIR> 2017-05-24 18:15
> + startup nsh 26 2017-05-24 18:15
> + vmlinuz 6926384 2017-05-24 18:15
> + test wks 628 2017-05-24 21:22
> + 5 files 6 927 038 bytes
> + 15 677 440 bytes free
> +
> + The second form of the command copies file or directory to the
> specified directory
> + on the partition:
> + $ wic cp test
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/
> + $ wic ls
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/efi/
> + Volume in drive : is boot
> + Volume Serial Number is DB4C-FD4C
> + Directory for ::/efi
> +
> + . <DIR> 2017-05-24 18:15
> + .. <DIR> 2017-05-24 18:15
> + boot <DIR> 2017-05-24 18:15
> + test <DIR> 2017-05-24 21:27
> + 4 files 0 bytes
> + 15 675 392 bytes free
> +
> + The third form of the command copies file or directory from the
> specified directory
> + on the partition to local:
> + $ wic cp
> tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/vmlinuz
> test +
> + The -n option is used to specify the path to the native sysroot
> + containing the tools(parted and mtools) to use.
> +"""
> +
> +wic_rm_usage = """
> +
> + Remove files or directories from the vfat or ext* partitions
> +
> + usage: wic rm <image>:<partition><path> [--native-sysroot <path>]
> +
> + This command removes files or directories from the vfat or ext*
> partitions of
> + the partitioned image.
> +
> + See 'wic help rm' for more detailed instructions.
> +
> +"""
> +
> +wic_rm_help = """
> +
> +NAME
> + wic rm - remove files or directories from the vfat or ext*
> partitions +
> +SYNOPSIS
> + wic rm <src> <image>:<partition><path>
> + wic rm <src> <image>:<partition><path> --native-sysroot <path>
> + wic rm -r <image>:<partition><path>
> +
> +DESCRIPTION
> + This command removes files or directories from the vfat or ext*
> partition of the
> + partitioned image:
> +
> + $ wic ls
> ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
> + Volume in drive : is boot
> + Volume Serial Number is 11D0-DE21
> + Directory for ::/
> +
> + libcom32 c32 186500 2017-06-02 15:15
> + libutil c32 24148 2017-06-02 15:15
> + syslinux cfg 209 2017-06-02 15:15
> + vesamenu c32 27104 2017-06-02 15:15
> + vmlinuz 6926384 2017-06-02 15:15
> + 5 files 7 164 345 bytes
> + 16 582 656 bytes free
> +
> + $ wic rm
> ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1/libutil.c32
> +
> + $ wic ls
> ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic:1
> + Volume in drive : is boot
> + Volume Serial Number is 11D0-DE21
> + Directory for ::/
> +
> + libcom32 c32 186500 2017-06-02 15:15
> + syslinux cfg 209 2017-06-02 15:15
> + vesamenu c32 27104 2017-06-02 15:15
> + vmlinuz 6926384 2017-06-02 15:15
> + 4 files 7 140 197 bytes
> + 16 607 232 bytes free
> +
> + The -n option is used to specify the path to the native sysroot
> + containing the tools(parted and mtools) to use.
> +
> + The -r option is used to remove directories and their contents
> + recursively,this only applies to ext* partition.
> +"""
> +
> +wic_write_usage = """
> +
> + Write image to a device
> +
> + usage: wic write <image> <target device> [--expand [rules]]
> [--native-sysroot <path>] +
> + This command writes partitioned image to a target device (USB
> stick, SD card etc). +
> + See 'wic help write' for more detailed instructions.
> +
> +"""
> +
> +wic_write_help = """
> +
> +NAME
> + wic write - write an image to a device
> +
> +SYNOPSIS
> + wic write <image> <target>
> + wic write <image> <target> --expand auto
> + wic write <image> <target> --expand 1:100M,2:300M
> + wic write <image> <target> --native-sysroot <path>
> +
> +DESCRIPTION
> + This command writes an image to a target device (USB stick, SD
> card etc) +
> + $ wic write
> ./tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic
> /dev/sdb +
> + The --expand option is used to resize image partitions.
> + --expand auto expands partitions to occupy all free space
> available on the target device.
> + It's also possible to specify expansion rules in a format
> + <partition>:<size>[,<partition>:<size>...] for one or more
> partitions.
> + Specifying size 0 will keep partition unmodified.
> + Note: Resizing boot partition can result in non-bootable image
> for non-EFI images. It is
> + recommended to use size 0 for boot partition to keep image
> bootable. +
> + The --native-sysroot option is used to specify the path to the
> native sysroot
> + containing the tools(parted, resize2fs) to use.
> +"""
> +
> wic_plugins_help = """
>
> NAME
> @@ -308,7 +536,8 @@ DESCRIPTION
>
> Source plugins can also be implemented and added by external
> layers - any plugins found in a scripts/lib/wic/plugins/source/
> - directory in an external layer will also be made available.
> + or lib/wic/plugins/source/ directory in an external layer will
> + also be made available.
>
> When the wic implementation needs to invoke a partition-specific
> implementation, it looks for the plugin that has the same name as
> @@ -346,6 +575,10 @@ DESCRIPTION
> partition. In other words, it 'prepares' the final
> partition image which will be incorporated into the disk image.
>
> + do_post_partition()
> + Called after the partition is created. It is useful to add
> post
> + operations e.g. signing the partition.
> +
> do_configure_partition()
> Called before do_prepare_partition(), typically used to
> create custom configuration files for a partition, for
> @@ -632,8 +865,11 @@ DESCRIPTION
> Partitions with a <mountpoint> specified will be
> automatically mounted. This is achieved by wic adding entries to the
> fstab during image generation. In order for a valid fstab to be
> generated one of the
> - --ondrive, --ondisk or --use-uuid partition options must be
> used for
> - each partition that specifies a mountpoint.
> + --ondrive, --ondisk, --use-uuid or --use-label partition
> options must
> + be used for each partition that specifies a mountpoint. Note
> that with
> + --use-{uuid,label} and non-root <mountpoint>, including swap,
> the mount
> + program must understand the PARTUUID or LABEL syntax. This
> currently
> + excludes the busybox versions of these applications.
>
>
> The following are supported 'part' options:
> @@ -687,6 +923,8 @@ DESCRIPTION
> apply to partitions created using '--source rootfs' (see
> --source above). Valid values are:
>
> + vfat
> + msdos
> ext2
> ext3
> ext4
> @@ -706,6 +944,14 @@ DESCRIPTION
> label is already in use by another
> filesystem, a new label is created for the partition.
>
> + --use-label: This option is specific to wic. It makes wic
> to use the
> + label in /etc/fstab to specify a partition. If
> the
> + --use-label and --use-uuid are used at the
> same time,
> + we prefer the uuid because it is less likely
> to cause
> + name confliction. We don't support using this
> parameter
> + on the root partition since it requires an
> initramfs to
> + parse this value and we do not currently
> support that. +
> --active: Marks the partition as active.
>
> --align (in KBytes): This option is specific to wic and says
> @@ -719,11 +965,31 @@ DESCRIPTION
> bootloaders.
>
> --exclude-path: This option is specific to wic. It excludes
> the given
> - absolute path from the resulting image. If
> the path
> + relative path from the resulting image. If
> the path ends with a slash, only the content of the directory
> is omitted, not the directory itself. This
> option only has an effect with the rootfs source plugin.
>
> + --include-path: This option is specific to wic. It adds the
> contents
> + of the given path or a rootfs to the
> resulting image.
> + The option contains two fields, the origin
> and the
> + destination. When the origin is a rootfs,
> it follows
> + the same logic as the rootfs-dir argument
> and the
> + permissions and owners are kept. When the
> origin is a
> + path, it is relative to the directory in
> which wic is
> + running not the rootfs itself so use of an
> absolute
> + path is recommended, and the owner and
> group is set to
> + root:root. If no destination is given it is
> + automatically set to the root of the
> rootfs. This
> + option only has an effect with the rootfs
> source
> + plugin.
> +
> + --change-directory: This option is specific to wic. It
> changes to the
> + given directory before copying the
> files. This
> + option is useful when we want to split
> a rootfs in
> + multiple partitions and we want to keep
> the right
> + permissions and usernames in all the
> partitions. +
> --extra-space: This option is specific to wic. It adds extra
> space after the space filled by the content
> of the partition. The final size can go
> @@ -738,6 +1004,8 @@ DESCRIPTION
> This option cannot be used with
> --fixed-size option.
>
> + --part-name: This option is specific to wic. It specifies
> name for GPT partitions. +
> --part-type: This option is specific to wic. It specifies
> partition type GUID for GPT partitions.
> List of partition type GUIDS can be found here:
> @@ -752,10 +1020,21 @@ DESCRIPTION
> in bootloader configuration before running wic. In
> this case .wks file can be generated or modified to set preconfigured
> parition UUID using this option.
> + --fsuuid: This option is specific to wic. It specifies
> filesystem UUID.
> + It's useful if preconfigured filesystem UUID is
> added to kernel command line
> + in bootloader configuration before running wic.
> In this case .wks file can
> + be generated or modified to set preconfigured
> filesystem UUID using this option. +
> --system-id: This option is specific to wic. It specifies
> partition system id. It's useful for the harware that requires
> non-default partition system ids. The parameter in one byte long hex
> number either with 0x prefix or without it.
> + --mkfs-extraopts: This option specifies extra options to
> pass to mkfs utility.
> + NOTE, that wic uses default options for
> some filesystems, for example
> + '-S 512' for mkfs.fat or '-F -i 8192' for
> mkfs.ext. Those options will
> + not take effect when --mkfs-extraopts is
> used. This should be taken into
> + account when using --mkfs-extraopts.
> +
> * bootloader
>
> This command allows the user to specify various bootloader
> @@ -793,3 +1072,67 @@ DESCRIPTION
> .wks files.
>
> """
> +
> +wic_help_help = """
> +NAME
> + wic help - display a help topic
> +
> +DESCRIPTION
> + Specify a help topic to display it. Topics are shown above.
> +"""
> +
> +
> +wic_help = """
> +Creates a customized OpenEmbedded image.
> +
> +Usage: wic [--version]
> + wic help [COMMAND or TOPIC]
> + wic COMMAND [ARGS]
> +
> + usage 1: Returns the current version of Wic
> + usage 2: Returns detailed help for a COMMAND or TOPIC
> + usage 3: Executes COMMAND
> +
> +
> +COMMAND:
> +
> + list - List available canned images and source plugins
> + ls - List contents of partitioned image or partition
> + rm - Remove files or directories from the vfat or ext*
> partitions
> + help - Show help for a wic COMMAND or TOPIC
> + write - Write an image to a device
> + cp - Copy files and directories to the vfat or ext*
> partitions
> + create - Create a new OpenEmbedded image
> +
> +
> +TOPIC:
> + overview - Presents an overall overview of Wic
> + plugins - Presents an overview and API for Wic plugins
> + kickstart - Presents a Wic kicstart file reference
> +
> +
> +Examples:
> +
> + $ wic --version
> +
> + Returns the current version of Wic
> +
> +
> + $ wic help cp
> +
> + Returns the SYNOPSIS and DESCRIPTION for the Wic "cp" command.
> +
> +
> + $ wic list images
> +
> + Returns the list of canned images (i.e. *.wks files located in
> + the /scripts/lib/wic/canned-wks directory.
> +
> +
> + $ wic create mkefidisk -e core-image-minimal
> +
> + Creates an EFI disk image from artifacts used in a previous
> + core-image-minimal build in standard BitBake locations
> + (e.g. Cooked Mode).
> +
> +"""
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index a039300..3453d9c 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -1,21 +1,8 @@
> -#!/usr/bin/env python -tt
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> +#!/usr/bin/env python3
> #
> # Copyright (c) 2016 Intel, Inc.
> #
> -# This program is free software; you can redistribute it and/or
> modify it -# under the terms of the GNU General Public License as
> published by the Free -# Software Foundation; version 2 of the License
> -#
> -# This program is distributed in the hope that it will be useful, but
> -# WITHOUT ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> General Public License -# for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA
> 02111-1307, USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This module provides parser for kickstart format
> @@ -28,14 +15,30 @@
> import os
> import shlex
> import logging
> +import re
>
> from argparse import ArgumentParser, ArgumentError, ArgumentTypeError
>
> from wic.engine import find_canned
> from wic.partition import Partition
> +from wic.misc import get_bitbake_var
>
> logger = logging.getLogger('wic')
>
> +__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}")
> +
> +def expand_line(line):
> + while True:
> + m = __expand_var_regexp__.search(line)
> + if not m:
> + return line
> + key = m.group()[2:-1]
> + val = get_bitbake_var(key)
> + if val is None:
> + logger.warning("cannot expand variable %s" % key)
> + return line
> + line = line[:m.start()] + val + line[m.end():]
> +
> class KickStartError(Exception):
> """Custom exception."""
> pass
> @@ -48,26 +51,31 @@ class KickStartParser(ArgumentParser):
> def error(self, message):
> raise ArgumentError(None, message)
>
> -def sizetype(arg):
> - """
> - Custom type for ArgumentParser
> - Converts size string in <num>[K|k|M|G] format into the integer
> value
> - """
> - if arg.isdigit():
> - return int(arg) * 1024
> -
> - if not arg[:-1].isdigit():
> - raise ArgumentTypeError("Invalid size: %r" % arg)
> +def sizetype(default):
> + def f(arg):
> + """
> + Custom type for ArgumentParser
> + Converts size string in <num>[K|k|M|G] format into the
> integer value
> + """
> + try:
> + suffix = default
> + size = int(arg)
> + except ValueError:
> + try:
> + suffix = arg[-1:]
> + size = int(arg[:-1])
> + except ValueError:
> + raise ArgumentTypeError("Invalid size: %r" % arg)
>
> - size = int(arg[:-1])
> - if arg.endswith("k") or arg.endswith("K"):
> - return size
> - if arg.endswith("M"):
> - return size * 1024
> - if arg.endswith("G"):
> - return size * 1024 * 1024
> + if suffix == "k" or suffix == "K":
> + return size
> + if suffix == "M":
> + return size * 1024
> + if suffix == "G":
> + return size * 1024 * 1024
>
> - raise ArgumentTypeError("Invalid size: %r" % arg)
> + raise ArgumentTypeError("Invalid size: %r" % arg)
> + return f
>
> def overheadtype(arg):
> """
> @@ -114,7 +122,7 @@ def systemidtype(arg):
> return arg
>
> class KickStart():
> - """"Kickstart parser implementation."""
> + """Kickstart parser implementation."""
>
> DEFAULT_EXTRA_SPACE = 10*1024
> DEFAULT_OVERHEAD_FACTOR = 1.3
> @@ -133,30 +141,41 @@ class KickStart():
> part.add_argument('mountpoint', nargs='?')
> part.add_argument('--active', action='store_true')
> part.add_argument('--align', type=int)
> + part.add_argument('--offset', type=sizetype("K"))
> part.add_argument('--exclude-path', nargs='+')
> - part.add_argument("--extra-space", type=sizetype)
> + part.add_argument('--include-path', nargs='+',
> action='append')
> + part.add_argument('--change-directory')
> + part.add_argument("--extra-space", type=sizetype("M"))
> part.add_argument('--fsoptions', dest='fsopts')
> - part.add_argument('--fstype')
> + part.add_argument('--fstype', default='vfat',
> + choices=('ext2', 'ext3', 'ext4', 'btrfs',
> + 'squashfs', 'vfat', 'msdos',
> 'swap'))
> + part.add_argument('--mkfs-extraopts', default='')
> part.add_argument('--label')
> + part.add_argument('--use-label', action='store_true')
> part.add_argument('--no-table', action='store_true')
> part.add_argument('--ondisk', '--ondrive', dest='disk',
> default='sda') part.add_argument("--overhead-factor",
> type=overheadtype)
> + part.add_argument('--part-name')
> part.add_argument('--part-type')
> part.add_argument('--rootfs-dir')
> + part.add_argument('--type', default='primary',
> + choices = ('primary', 'logical'))
>
> # --size and --fixed-size cannot be specified together;
> options # ----extra-space and --overhead-factor should also raise a
> parser # --error, but since nesting mutually exclusive groups does
> not work, # ----extra-space/--overhead-factor are handled later
> sizeexcl = part.add_mutually_exclusive_group()
> - sizeexcl.add_argument('--size', type=sizetype, default=0)
> - sizeexcl.add_argument('--fixed-size', type=sizetype,
> default=0)
> + sizeexcl.add_argument('--size', type=sizetype("M"),
> default=0)
> + sizeexcl.add_argument('--fixed-size', type=sizetype("M"),
> default=0)
> part.add_argument('--source')
> part.add_argument('--sourceparams')
> part.add_argument('--system-id', type=systemidtype)
> part.add_argument('--use-uuid', action='store_true')
> part.add_argument('--uuid')
> + part.add_argument('--fsuuid')
>
> bootloader = subparsers.add_parser('bootloader')
> bootloader.add_argument('--append')
> @@ -184,6 +203,7 @@ class KickStart():
> line = line.strip()
> lineno += 1
> if line and line[0] != '#':
> + line = expand_line(line)
> try:
> line_args = shlex.split(line)
> parsed = parser.parse_args(line_args)
> @@ -191,6 +211,20 @@ class KickStart():
> raise KickStartError('%s:%d: %s' % \
> (confpath, lineno, err))
> if line.startswith('part'):
> + # SquashFS does not support filesystem UUID
> + if parsed.fstype == 'squashfs':
> + if parsed.fsuuid:
> + err = "%s:%d: SquashFS does not
> support UUID" \
> + % (confpath, lineno)
> + raise KickStartError(err)
> + if parsed.label:
> + err = "%s:%d: SquashFS does not
> support LABEL" \
> + % (confpath, lineno)
> + raise KickStartError(err)
> + if parsed.use_label and not parsed.label:
> + err = "%s:%d: Must set the label with
> --label" \
> + % (confpath, lineno)
> + raise KickStartError(err)
> # using ArgumentParser one cannot easily
> tell if option # was passed as argument, if said option has a default
> # value; --overhead-factor/--extra-space
> cannot be used @@ -219,6 +253,11 @@ class KickStart():
> elif line.startswith('bootloader'):
> if not self.bootloader:
> self.bootloader = parsed
> + # Concatenate the strings set in APPEND
> + append_var = get_bitbake_var("APPEND")
> + if append_var:
> + self.bootloader.append = '
> '.join(filter(None, \
> +
> (self.bootloader.append, append_var))) else:
> err = "%s:%d: more than one bootloader
> specified" \ % (confpath, lineno)
> diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/misc.py
> similarity index 70%
> rename from scripts/lib/wic/utils/misc.py
> rename to scripts/lib/wic/misc.py
> index c941112..91975ba 100644
> --- a/scripts/lib/wic/utils/misc.py
> +++ b/scripts/lib/wic/misc.py
> @@ -1,21 +1,7 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2013, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This module provides a place to collect various wic-related utils
> @@ -29,12 +15,12 @@
> import logging
> import os
> import re
> +import subprocess
>
> from collections import defaultdict
> from distutils import spawn
>
> from wic import WicError
> -from wic.utils import runner
>
> logger = logging.getLogger('wic')
>
> @@ -43,6 +29,9 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> "grub-mkimage": "grub-efi",
> "isohybrid": "syslinux",
> "mcopy": "mtools",
> + "mdel" : "mtools",
> + "mdeltree" : "mtools",
> + "mdir" : "mtools",
> "mkdosfs": "dosfstools",
> "mkisofs": "cdrtools",
> "mkfs.btrfs": "btrfs-tools",
> @@ -52,14 +41,48 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> "mkfs.vfat": "dosfstools",
> "mksquashfs": "squashfs-tools",
> "mkswap": "util-linux",
> - "mmd": "syslinux",
> + "mmd": "mtools",
> "parted": "parted",
> "sfdisk": "util-linux",
> "sgdisk": "gptfdisk",
> - "syslinux": "syslinux"
> + "syslinux": "syslinux",
> + "tar": "tar"
> }
>
> -def _exec_cmd(cmd_and_args, as_shell=False, catch=3):
> +def runtool(cmdln_or_args):
> + """ wrapper for most of the subprocess calls
> + input:
> + cmdln_or_args: can be both args and cmdln str (shell=True)
> + return:
> + rc, output
> + """
> + if isinstance(cmdln_or_args, list):
> + cmd = cmdln_or_args[0]
> + shell = False
> + else:
> + import shlex
> + cmd = shlex.split(cmdln_or_args)[0]
> + shell = True
> +
> + sout = subprocess.PIPE
> + serr = subprocess.STDOUT
> +
> + try:
> + process = subprocess.Popen(cmdln_or_args, stdout=sout,
> + stderr=serr, shell=shell)
> + sout, serr = process.communicate()
> + # combine stdout and stderr, filter None out and decode
> + out = ''.join([out.decode('utf-8') for out in [sout, serr]
> if out])
> + except OSError as err:
> + if err.errno == 2:
> + # [Errno 2] No such file or directory
> + raise WicError('Cannot run command: %s, lost
> dependency?' % cmd)
> + else:
> + raise # relay
> +
> + return process.returncode, out
> +
> +def _exec_cmd(cmd_and_args, as_shell=False):
> """
> Execute command, catching stderr, stdout
>
> @@ -70,9 +93,9 @@ def _exec_cmd(cmd_and_args, as_shell=False,
> catch=3): logger.debug(args)
>
> if as_shell:
> - ret, out = runner.runtool(cmd_and_args, catch)
> + ret, out = runtool(cmd_and_args)
> else:
> - ret, out = runner.runtool(args, catch)
> + ret, out = runtool(args)
> out = out.strip()
> if ret != 0:
> raise WicError("_exec_cmd: %s returned '%s' instead of
> 0\noutput: %s" % \ @@ -84,14 +107,23 @@ def _exec_cmd(cmd_and_args,
> as_shell=False, catch=3): return ret, out
>
>
> -def exec_cmd(cmd_and_args, as_shell=False, catch=3):
> +def exec_cmd(cmd_and_args, as_shell=False):
> """
> Execute command, return output
> """
> - return _exec_cmd(cmd_and_args, as_shell, catch)[1]
> + return _exec_cmd(cmd_and_args, as_shell)[1]
> +
> +def find_executable(cmd, paths):
> + recipe = cmd
> + if recipe in NATIVE_RECIPES:
> + recipe = NATIVE_RECIPES[recipe]
> + provided = get_bitbake_var("ASSUME_PROVIDED")
> + if provided and "%s-native" % recipe in provided:
> + return True
>
> + return spawn.find_executable(cmd, paths)
>
> -def exec_native_cmd(cmd_and_args, native_sysroot, catch=3,
> pseudo=""): +def exec_native_cmd(cmd_and_args, native_sysroot,
> pseudo=""): """
> Execute native command, catching stderr, stdout
>
> @@ -106,19 +138,16 @@ def exec_native_cmd(cmd_and_args,
> native_sysroot, catch=3, pseudo=""): if pseudo:
> cmd_and_args = pseudo + cmd_and_args
>
> - wtools_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE",
> "wic-tools")
> + native_paths = "%s/sbin:%s/usr/sbin:%s/usr/bin" % \
> + (native_sysroot, native_sysroot, native_sysroot)
>
> - native_paths = \
> -
> "%s/sbin:%s/usr/sbin:%s/usr/bin:%s/sbin:%s/usr/sbin:%s/usr/bin" % \
> - (wtools_sysroot, wtools_sysroot, wtools_sysroot,
> - native_sysroot, native_sysroot, native_sysroot)
> native_cmd_and_args = "export PATH=%s:$PATH;%s" % \
> - (native_paths, cmd_and_args)
> + (native_paths, cmd_and_args)
> logger.debug("exec_native_cmd: %s", native_cmd_and_args)
>
> # If the command isn't in the native sysroot say we failed.
> - if spawn.find_executable(args[0], native_paths):
> - ret, out = _exec_cmd(native_cmd_and_args, True, catch)
> + if find_executable(args[0], native_paths):
> + ret, out = _exec_cmd(native_cmd_and_args, True)
> else:
> ret = 127
> out = "can't find native executable %s in %s" % (args[0],
> native_paths) @@ -131,8 +160,8 @@ def exec_native_cmd(cmd_and_args,
> native_sysroot, catch=3, pseudo=""): "was not found (see details
> above).\n\n" % prog recipe = NATIVE_RECIPES.get(prog)
> if recipe:
> - msg += "Please bake it with 'bitbake %s-native' "\
> - "and try again.\n" % recipe
> + msg += "Please make sure wic-tools have %s-native in its
> DEPENDS, "\
> + "build it with 'bitbake wic-tools' and try
> again.\n" % recipe else:
> msg += "Wic failed to find a recipe to build native %s.
> Please "\ "file a bug against wic.\n" % prog
> @@ -153,7 +182,7 @@ class BitbakeVars(defaultdict):
> self.default_image = None
> self.vars_dir = None
>
> - def _parse_line(self, line, image,
> matcher=re.compile(r"^(\w+)=(.+)")):
> + def _parse_line(self, line, image,
> matcher=re.compile(r"^([a-zA-Z0-9\-_+./~]+)=(.*)")): """
> Parse one line from bitbake -e output or from .env file.
> Put result key-value pair into the storage.
> diff --git a/scripts/lib/wic/partition.py
> b/scripts/lib/wic/partition.py index 8e32afc..85eb15c 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -1,21 +1,7 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2013-2016 Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This module provides the OpenEmbedded partition object definitions.
> @@ -26,10 +12,10 @@
>
> import logging
> import os
> -import tempfile
> +import uuid
>
> from wic import WicError
> -from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var
> +from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var
> from wic.pluginbase import PluginMgr
>
> logger = logging.getLogger('wic')
> @@ -44,13 +30,19 @@ class Partition():
> self.device = None
> self.extra_space = args.extra_space
> self.exclude_path = args.exclude_path
> + self.include_path = args.include_path
> + self.change_directory = args.change_directory
> self.fsopts = args.fsopts
> self.fstype = args.fstype
> self.label = args.label
> + self.use_label = args.use_label
> + self.mkfs_extraopts = args.mkfs_extraopts
> self.mountpoint = args.mountpoint
> self.no_table = args.no_table
> self.num = None
> + self.offset = args.offset
> self.overhead_factor = args.overhead_factor
> + self.part_name = args.part_name
> self.part_type = args.part_type
> self.rootfs_dir = args.rootfs_dir
> self.size = args.size
> @@ -60,10 +52,11 @@ class Partition():
> self.system_id = args.system_id
> self.use_uuid = args.use_uuid
> self.uuid = args.uuid
> + self.fsuuid = args.fsuuid
> + self.type = args.type
>
> self.lineno = lineno
> self.source_file = ""
> - self.sourceparams_dict = {}
>
> def get_extra_block_count(self, current_blocks):
> """
> @@ -136,22 +129,24 @@ class Partition():
> "specify a non-zero
> --size/--fixed-size for that " "partition." % self.mountpoint)
>
> - if self.fstype and self.fstype == "swap":
> + if self.fstype == "swap":
> self.prepare_swap_partition(cr_workdir, oe_builddir,
> native_sysroot)
> self.source_file = "%s/fs.%s" % (cr_workdir,
> self.fstype)
> - elif self.fstype:
> + else:
> + if self.fstype == 'squashfs':
> + raise WicError("It's not possible to create
> empty squashfs "
> + "partition '%s'" %
> (self.mountpoint)) +
> rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label,
> self.lineno,
> self.fstype) if os.path.isfile(rootfs):
> os.remove(rootfs)
> - for prefix in ("ext", "btrfs", "vfat", "squashfs"):
> - if self.fstype.startswith(prefix):
> - method = getattr(self,
> - "prepare_empty_partition_"
> + prefix)
> - method(rootfs, oe_builddir, native_sysroot)
> - self.source_file = rootfs
> - break
> +
> + prefix = "ext" if self.fstype.startswith("ext") else
> self.fstype
> + method = getattr(self, "prepare_empty_partition_" +
> prefix)
> + method(rootfs, oe_builddir, native_sysroot)
> + self.source_file = rootfs
> return
>
> plugins = PluginMgr.get_plugins('source')
> @@ -168,7 +163,7 @@ class Partition():
> # Split sourceparams string of the form
> key1=val1[,key2=val2,...] # into a dict. Also accepts valueless keys
> i.e. without = splitted = self.sourceparams.split(',')
> - srcparams_dict = dict(par.split('=') for par in splitted
> if par)
> + srcparams_dict = dict(par.split('=', 1) for par in
> splitted if par)
> plugin = PluginMgr.get_plugins('source')[self.source]
> plugin.do_configure_partition(self, srcparams_dict, creator,
> @@ -180,6 +175,9 @@ class Partition():
> plugin.do_prepare_partition(self, srcparams_dict, creator,
> cr_workdir, oe_builddir,
> bootimg_dir, kernel_dir, rootfs_dir, native_sysroot)
> + plugin.do_post_partition(self, srcparams_dict, creator,
> + cr_workdir, oe_builddir,
> bootimg_dir,
> + kernel_dir, rootfs_dir,
> native_sysroot)
> # further processing required Partition.size to be an
> integer, make # sure that it is one
> @@ -193,74 +191,57 @@ class Partition():
> "larger (%d kB) than its allowed size %d
> kB" % (self.mountpoint, self.size, self.fixed_size))
>
> - def prepare_rootfs_from_fs_image(self, cr_workdir, oe_builddir,
> - rootfs_dir):
> - """
> - Handle an already-created partition e.g. xxx.ext3
> - """
> - rootfs = oe_builddir
> - du_cmd = "du -Lbks %s" % rootfs
> - out = exec_cmd(du_cmd)
> - rootfs_size = out.split()[0]
> -
> - self.size = int(rootfs_size)
> - self.source_file = rootfs
> -
> def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
> - native_sysroot):
> + native_sysroot, real_rootfs = True,
> pseudo_dir = None): """
> Prepare content for a rootfs partition i.e. create a
> partition and fill it from a /rootfs dir.
>
> - Currently handles ext2/3/4, btrfs and vfat.
> + Currently handles ext2/3/4, btrfs, vfat and squashfs.
> """
> p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" %
> native_sysroot)
> - p_localstatedir = os.environ.get("PSEUDO_LOCALSTATEDIR",
> - "%s/../pseudo" % rootfs_dir)
> - p_passwd = os.environ.get("PSEUDO_PASSWD", rootfs_dir)
> - p_nosymlinkexp = os.environ.get("PSEUDO_NOSYMLINKEXP", "1")
> - pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix
> - pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % p_localstatedir
> - pseudo += "export PSEUDO_PASSWD=%s;" % p_passwd
> - pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
> - pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
> + if (pseudo_dir):
> + pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix
> + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir
> + pseudo += "export PSEUDO_PASSWD=%s;" % rootfs_dir
> + pseudo += "export PSEUDO_NOSYMLINKEXP=1;"
> + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
> + else:
> + pseudo = None
>
> rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> self.lineno, self.fstype)
> if os.path.isfile(rootfs):
> os.remove(rootfs)
>
> - if not self.fstype:
> - raise WicError("File system for partition %s not
> specified in "
> - "kickstart, use --fstype option" %
> self.mountpoint) -
> - # Get rootfs size from bitbake variable if it's not set in
> .ks file
> - if not self.size:
> - # Bitbake variable ROOTFS_SIZE is calculated in
> - # Image._get_rootfs_size method from meta/lib/oe/image.py
> - # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
> - # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
> + if not self.size and real_rootfs:
> + # The rootfs size is not set in .ks file so try to get it
> + # from bitbake variable
> rsize_bb = get_bitbake_var('ROOTFS_SIZE')
> - if rsize_bb:
> - logger.warning('overhead-factor was specified, but
> size was not,'
> - ' so bitbake variables will be used
> for the size.'
> - ' In this case both
> IMAGE_OVERHEAD_FACTOR and '
> - '--overhead-factor will be applied')
> + rdir = get_bitbake_var('IMAGE_ROOTFS')
> + if rsize_bb and rdir == rootfs_dir:
> + # Bitbake variable ROOTFS_SIZE is calculated in
> + # Image._get_rootfs_size method from
> meta/lib/oe/image.py
> + # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT,
> + # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE
> self.size = int(round(float(rsize_bb)))
> -
> - for prefix in ("ext", "btrfs", "vfat", "squashfs"):
> - if self.fstype.startswith(prefix):
> - method = getattr(self, "prepare_rootfs_" + prefix)
> - method(rootfs, oe_builddir, rootfs_dir,
> native_sysroot, pseudo) -
> - self.source_file = rootfs
> -
> - # get the rootfs size in the right units for
> kickstart (kB)
> - du_cmd = "du -Lbks %s" % rootfs
> + else:
> + # Bitbake variable ROOTFS_SIZE is not defined so
> compute it
> + # from the rootfs_dir size using the same logic
> found in
> + # get_rootfs_size() from meta/classes/image.bbclass
> + du_cmd = "du -ks %s" % rootfs_dir
> out = exec_cmd(du_cmd)
> self.size = int(out.split()[0])
>
> - break
> + prefix = "ext" if self.fstype.startswith("ext") else
> self.fstype
> + method = getattr(self, "prepare_rootfs_" + prefix)
> + method(rootfs, oe_builddir, rootfs_dir, native_sysroot,
> pseudo)
> + self.source_file = rootfs
> +
> + # get the rootfs size in the right units for kickstart (kB)
> + du_cmd = "du -Lbks %s" % rootfs
> + out = exec_cmd(du_cmd)
> + self.size = int(out.split()[0])
>
> def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
> native_sysroot, pseudo):
> @@ -276,25 +257,23 @@ class Partition():
> with open(rootfs, 'w') as sparse:
> os.ftruncate(sparse.fileno(), rootfs_size * 1024)
>
> - extra_imagecmd = "-i 8192"
> + extraopts = self.mkfs_extraopts or "-F -i 8192"
>
> label_str = ""
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
> - (self.fstype, extra_imagecmd, rootfs, label_str,
> rootfs_dir)
> + mkfs_cmd = "mkfs.%s %s %s %s -U %s -d %s" % \
> + (self.fstype, extraopts, rootfs, label_str, self.fsuuid,
> rootfs_dir) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> - mkfs_cmd = "fsck.%s -fy %s" % (self.fstype, rootfs)
> + mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
> native_sysroot, pseudo):
> """
> Prepare content for a btrfs rootfs partition.
> -
> - Currently handles ext2/3/4 and btrfs.
> """
> du_cmd = "du -ks %s" % rootfs_dir
> out = exec_cmd(du_cmd)
> @@ -309,14 +288,15 @@ class Partition():
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
> - (self.fstype, rootfs_size * 1024, rootfs_dir, label_str,
> rootfs)
> + mkfs_cmd = "mkfs.%s -b %d -r %s %s %s -U %s %s" % \
> + (self.fstype, rootfs_size * 1024, rootfs_dir, label_str,
> + self.mkfs_extraopts, self.fsuuid, rootfs)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> - def prepare_rootfs_vfat(self, rootfs, oe_builddir, rootfs_dir,
> - native_sysroot, pseudo):
> + def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir,
> + native_sysroot, pseudo):
> """
> - Prepare content for a vfat rootfs partition.
> + Prepare content for a msdos/vfat rootfs partition.
> """
> du_cmd = "du -bks %s" % rootfs_dir
> out = exec_cmd(du_cmd)
> @@ -328,7 +308,15 @@ class Partition():
> if self.label:
> label_str = "-n %s" % self.label
>
> - dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str,
> rootfs, rootfs_size)
> + size_str = ""
> + if self.fstype == 'msdos':
> + size_str = "-F 16" # FAT 16
> +
> + extraopts = self.mkfs_extraopts or '-S 512'
> +
> + dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \
> + (label_str, self.fsuuid, size_str, extraopts,
> rootfs,
> + rootfs_size)
> exec_native_cmd(dosfs_cmd, native_sysroot)
>
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
> @@ -337,13 +325,16 @@ class Partition():
> chmod_cmd = "chmod 644 %s" % rootfs
> exec_cmd(chmod_cmd)
>
> + prepare_rootfs_vfat = prepare_rootfs_msdos
> +
> def prepare_rootfs_squashfs(self, rootfs, oe_builddir,
> rootfs_dir, native_sysroot, pseudo):
> """
> Prepare content for a squashfs rootfs partition.
> """
> - squashfs_cmd = "mksquashfs %s %s -noappend" % \
> - (rootfs_dir, rootfs)
> + extraopts = self.mkfs_extraopts or '-noappend'
> + squashfs_cmd = "mksquashfs %s %s %s" % \
> + (rootfs_dir, rootfs, extraopts)
> exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo)
>
> def prepare_empty_partition_ext(self, rootfs, oe_builddir,
> @@ -355,14 +346,14 @@ class Partition():
> with open(rootfs, 'w') as sparse:
> os.ftruncate(sparse.fileno(), size * 1024)
>
> - extra_imagecmd = "-i 8192"
> + extraopts = self.mkfs_extraopts or "-i 8192"
>
> label_str = ""
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "mkfs.%s -F %s %s %s" % \
> - (self.fstype, extra_imagecmd, label_str, rootfs)
> + mkfs_cmd = "mkfs.%s -F %s %s -U %s %s" % \
> + (self.fstype, extraopts, label_str, self.fsuuid, rootfs)
> exec_native_cmd(mkfs_cmd, native_sysroot)
>
> def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
> @@ -378,12 +369,13 @@ class Partition():
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "mkfs.%s -b %d %s %s" % \
> - (self.fstype, self.size * 1024, label_str, rootfs)
> + mkfs_cmd = "mkfs.%s -b %d %s -U %s %s %s" % \
> + (self.fstype, self.size * 1024, label_str,
> self.fsuuid,
> + self.mkfs_extraopts, rootfs)
> exec_native_cmd(mkfs_cmd, native_sysroot)
>
> - def prepare_empty_partition_vfat(self, rootfs, oe_builddir,
> - native_sysroot):
> + def prepare_empty_partition_msdos(self, rootfs, oe_builddir,
> + native_sysroot):
> """
> Prepare an empty vfat partition.
> """
> @@ -393,40 +385,22 @@ class Partition():
> if self.label:
> label_str = "-n %s" % self.label
>
> - dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str,
> rootfs, blocks)
> - exec_native_cmd(dosfs_cmd, native_sysroot)
> + size_str = ""
> + if self.fstype == 'msdos':
> + size_str = "-F 16" # FAT 16
>
> - chmod_cmd = "chmod 644 %s" % rootfs
> - exec_cmd(chmod_cmd)
> + extraopts = self.mkfs_extraopts or '-S 512'
>
> - def prepare_empty_partition_squashfs(self, cr_workdir,
> oe_builddir,
> - native_sysroot):
> - """
> - Prepare an empty squashfs partition.
> - """
> - logger.warning("Creating of an empty squashfs %s partition
> was attempted. "
> - "Proceeding as requested.", self.mountpoint)
> -
> - path = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
> - if os.path.isfile(path):
> - os.remove(path)
> + dosfs_cmd = "mkdosfs %s -i %s %s %s -C %s %d" % \
> + (label_str, self.fsuuid, extraopts, size_str,
> rootfs,
> + blocks)
>
> - # it is not possible to create a squashfs without source
> data,
> - # thus prepare an empty temp dir that is used as source
> - tmpdir = tempfile.mkdtemp()
> -
> - squashfs_cmd = "mksquashfs %s %s -noappend" % \
> - (tmpdir, path)
> - exec_native_cmd(squashfs_cmd, native_sysroot)
> -
> - os.rmdir(tmpdir)
> + exec_native_cmd(dosfs_cmd, native_sysroot)
>
> - # get the rootfs size in the right units for kickstart (kB)
> - du_cmd = "du -Lbks %s" % path
> - out = exec_cmd(du_cmd)
> - fs_size = out.split()[0]
> + chmod_cmd = "chmod 644 %s" % rootfs
> + exec_cmd(chmod_cmd)
>
> - self.size = int(fs_size)
> + prepare_empty_partition_vfat = prepare_empty_partition_msdos
>
> def prepare_swap_partition(self, cr_workdir, oe_builddir,
> native_sysroot): """
> @@ -437,9 +411,9 @@ class Partition():
> with open(path, 'w') as sparse:
> os.ftruncate(sparse.fileno(), self.size * 1024)
>
> - import uuid
> label_str = ""
> if self.label:
> label_str = "-L %s" % self.label
> - mkswap_cmd = "mkswap %s -U %s %s" % (label_str,
> str(uuid.uuid1()), path) +
> + mkswap_cmd = "mkswap %s -U %s %s" % (label_str, self.fsuuid,
> path) exec_native_cmd(mkswap_cmd, native_sysroot)
> diff --git a/scripts/lib/wic/pluginbase.py
> b/scripts/lib/wic/pluginbase.py index fb3d179..d9b4e57 100644
> --- a/scripts/lib/wic/pluginbase.py
> +++ b/scripts/lib/wic/pluginbase.py
> @@ -1,19 +1,9 @@
> -#!/usr/bin/env python -tt
> +#!/usr/bin/env python3
> #
> # Copyright (c) 2011 Intel, Inc.
> #
> -# This program is free software; you can redistribute it and/or
> modify it -# under the terms of the GNU General Public License as
> published by the Free -# Software Foundation; version 2 of the License
> +# SPDX-License-Identifier: GPL-2.0-only
> #
> -# This program is distributed in the hope that it will be useful, but
> -# WITHOUT ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> General Public License -# for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA
> 02111-1307, USA.
> __all__ = ['ImagerPlugin', 'SourcePlugin']
>
> @@ -24,11 +14,11 @@ from collections import defaultdict
> from importlib.machinery import SourceFileLoader
>
> from wic import WicError
> -from wic.utils.misc import get_bitbake_var
> +from wic.misc import get_bitbake_var
>
> PLUGIN_TYPES = ["imager", "source"]
>
> -SCRIPTS_PLUGIN_DIR = "scripts/lib/wic/plugins"
> +SCRIPTS_PLUGIN_DIR = ["scripts/lib/wic/plugins", "lib/wic/plugins"]
>
> logger = logging.getLogger('wic')
>
> @@ -48,10 +38,11 @@ class PluginMgr:
> cls._plugin_dirs =
> [os.path.join(os.path.dirname(__file__), 'plugins')] layers =
> get_bitbake_var("BBLAYERS") or '' for layer_path in layers.split():
> - path = os.path.join(layer_path, SCRIPTS_PLUGIN_DIR)
> - path = os.path.abspath(os.path.expanduser(path))
> - if path not in cls._plugin_dirs and
> os.path.isdir(path):
> - cls._plugin_dirs.insert(0, path)
> + for script_plugin_dir in SCRIPTS_PLUGIN_DIR:
> + path = os.path.join(layer_path,
> script_plugin_dir)
> + path = os.path.abspath(os.path.expanduser(path))
> + if path not in cls._plugin_dirs and
> os.path.isdir(path):
> + cls._plugin_dirs.insert(0, path)
>
> if ptype not in PLUGINS:
> # load all ptype plugins
> @@ -138,3 +129,12 @@ class SourcePlugin(metaclass=PluginMeta):
> """
> logger.debug("SourcePlugin: do_prepare_partition: part: %s",
> part)
> + @classmethod
> + def do_post_partition(cls, part, source_params, creator,
> cr_workdir,
> + oe_builddir, bootimg_dir, kernel_dir,
> rootfs_dir,
> + native_sysroot):
> + """
> + Called after the partition is created. It is useful to add
> post
> + operations e.g. security signing the partition.
> + """
> + logger.debug("SourcePlugin: do_post_partition: part: %s",
> part) diff --git a/scripts/lib/wic/plugins/imager/direct.py
> b/scripts/lib/wic/plugins/imager/direct.py index 7d38ab3..2f01999
> 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> +++ b/scripts/lib/wic/plugins/imager/direct.py
> @@ -1,21 +1,7 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2013, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This implements the 'direct' imager plugin class for 'wic'
> @@ -26,17 +12,20 @@
>
> import logging
> import os
> +import random
> import shutil
> import tempfile
> import uuid
>
> from time import strftime
>
> +from oe.path import copyhardlinktree
> +
> from wic import WicError
> from wic.filemap import sparse_copy
> from wic.ksparser import KickStart, KickStartError
> from wic.pluginbase import PluginMgr, ImagerPlugin
> -from wic.utils.misc import get_bitbake_var, exec_cmd, exec_native_cmd
> +from wic.misc import get_bitbake_var, exec_cmd, exec_native_cmd
>
> logger = logging.getLogger('wic')
>
> @@ -68,6 +57,8 @@ class DirectPlugin(ImagerPlugin):
> self.outdir = options.outdir
> self.compressor = options.compressor
> self.bmap = options.bmap
> + self.no_fstab_update = options.no_fstab_update
> + self.original_fstab = None
>
> self.name = "%s-%s" %
> (os.path.splitext(os.path.basename(wks_file))[0],
> strftime("%Y%m%d%H%M")) @@ -113,26 +104,38 @@ class
> DirectPlugin(ImagerPlugin):
> with open(fstab_path) as fstab:
> fstab_lines = fstab.readlines()
> + self.original_fstab = fstab_lines.copy()
>
> if self._update_fstab(fstab_lines, self.parts):
> - shutil.copyfile(fstab_path, fstab_path + ".orig")
> -
> with open(fstab_path, "w") as fstab:
> fstab.writelines(fstab_lines)
> -
> - return fstab_path
> + else:
> + self.original_fstab = None
>
> def _update_fstab(self, fstab_lines, parts):
> """Assume partition order same as in wks"""
> updated = False
> for part in parts:
> if not part.realnum or not part.mountpoint \
> - or part.mountpoint in ("/", "/boot"):
> + or part.mountpoint == "/":
> continue
>
> - # mmc device partitions are named mmcblk0p1, mmcblk0p2..
> - prefix = 'p' if part.disk.startswith('mmcblk') else ''
> - device_name = "/dev/%s%s%d" % (part.disk, prefix,
> part.realnum)
> + if part.use_uuid:
> + if part.fsuuid:
> + # FAT UUID is different from others
> + if len(part.fsuuid) == 10:
> + device_name = "UUID=%s-%s" % \
> + (part.fsuuid[2:6],
> part.fsuuid[6:])
> + else:
> + device_name = "UUID=%s" % part.fsuuid
> + else:
> + device_name = "PARTUUID=%s" % part.uuid
> + elif part.use_label:
> + device_name = "LABEL=%s" % part.label
> + else:
> + # mmc device partitions are named mmcblk0p1,
> mmcblk0p2..
> + prefix = 'p' if part.disk.startswith('mmcblk') else
> ''
> + device_name = "/dev/%s%s%d" % (part.disk, prefix,
> part.realnum)
> opts = part.fsopts if part.fsopts else "defaults"
> line = "\t".join([device_name, part.mountpoint,
> part.fstype, @@ -156,7 +159,8 @@ class DirectPlugin(ImagerPlugin):
> filesystems from the artifacts directly and combine them into
> a partitioned image.
> """
> - fstab_path =
> self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
> + if not self.no_fstab_update:
> + self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR"))
>
> for part in self.parts:
> # get rootfs size from bitbake variable if it's not set
> in .ks file @@ -173,10 +177,6 @@ class DirectPlugin(ImagerPlugin):
> part.size = int(round(float(rsize_bb)))
>
> self._image.prepare(self)
> -
> - if fstab_path:
> - shutil.move(fstab_path + ".orig", fstab_path)
> -
> self._image.layout_partitions()
> self._image.create()
>
> @@ -205,8 +205,10 @@ class DirectPlugin(ImagerPlugin):
> # Generate .bmap
> if self.bmap:
> logger.debug("Generating bmap file for %s", disk_name)
> - exec_native_cmd("bmaptool create %s -o %s.bmap" %
> (full_path, full_path),
> - self.native_sysroot)
> + python = os.path.join(self.native_sysroot,
> 'usr/bin/python3-native/python3')
> + bmaptool = os.path.join(self.native_sysroot,
> 'usr/bin/bmaptool')
> + exec_native_cmd("%s %s create %s -o %s.bmap" % \
> + (python, bmaptool, full_path,
> full_path), self.native_sysroot) # Compress the image
> if self.compressor:
> logger.debug("Compressing disk %s with %s", disk_name,
> self.compressor) @@ -233,7 +235,8 @@ class DirectPlugin(ImagerPlugin):
> suffix = ':'
> else:
> suffix = '["%s"]:' % (part.mountpoint or part.label)
> - msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20),
> part.rootfs_dir)
> + rootdir = part.rootfs_dir
> + msg += ' ROOTFS_DIR%s%s\n' % (suffix.ljust(20), rootdir)
>
> msg += ' BOOTIMG_DIR: %s\n' %
> self.bootimg_dir msg += ' KERNEL_DIR: %s\n' %
> self.kernel_dir @@ -270,6 +273,12 @@ class DirectPlugin(ImagerPlugin):
> if os.path.isfile(path):
> shutil.move(path, os.path.join(self.outdir, fname))
>
> + #Restore original fstab
> + if self.original_fstab:
> + fstab_path = self.rootfs_dir.get("ROOTFS_DIR") +
> "/etc/fstab"
> + with open(fstab_path, "w") as fstab:
> + fstab.writelines(self.original_fstab)
> +
> # remove work directory
> shutil.rmtree(self.workdir, ignore_errors=True)
>
> @@ -291,18 +300,23 @@ class PartitionedImage():
> self.path = path # Path to the image file
> self.numpart = 0 # Number of allocated partitions
> self.realpart = 0 # Number of partitions in the partition
> table
> + self.primary_part_num = 0 # Number of primary partitions
> (msdos)
> + self.extendedpart = 0 # Create extended partition
> before this logical partition (msdos)
> + self.extended_size_sec = 0 # Size of exteded partition
> (msdos)
> + self.logical_part_cnt = 0 # Number of total logical
> paritions (msdos) self.offset = 0 # Offset of next partition (in
> sectors) self.min_size = 0 # Minimum required disk size to fit
> # all partitions (in bytes)
> self.ptable_format = ptable_format # Partition table format
> # Disk system identifier
> - self.identifier = int.from_bytes(os.urandom(4), 'little')
> + self.identifier = random.SystemRandom().randint(1,
> 0xffffffff)
> self.partitions = partitions
> self.partimages = []
> # Size of a sector used in calculations
> self.sector_size = SECTOR_SIZE
> self.native_sysroot = native_sysroot
> + num_real_partitions = len([p for p in self.partitions if not
> p.no_table])
> # calculate the real partition number, accounting for
> partitions not # in the partition table and logical partitions
> @@ -312,18 +326,23 @@ class PartitionedImage():
> part.realnum = 0
> else:
> realnum += 1
> - if self.ptable_format == 'msdos' and realnum > 3:
> + if self.ptable_format == 'msdos' and realnum > 3 and
> num_real_partitions > 4: part.realnum = realnum + 1
> continue
> part.realnum = realnum
>
> - # generate parition UUIDs
> + # generate parition and filesystem UUIDs
> for part in self.partitions:
> if not part.uuid and part.use_uuid:
> if self.ptable_format == 'gpt':
> part.uuid = str(uuid.uuid4())
> else: # msdos partition table
> - part.uuid = '%0x-%02d' % (self.identifier,
> part.realnum)
> + part.uuid = '%08x-%02d' % (self.identifier,
> part.realnum)
> + if not part.fsuuid:
> + if part.fstype == 'vfat' or part.fstype == 'msdos':
> + part.fsuuid = '0x' +
> str(uuid.uuid4())[:8].upper()
> + else:
> + part.fsuuid = str(uuid.uuid4())
>
> def prepare(self, imager):
> """Prepare an image. Call prepare method of all image
> partitions.""" @@ -352,6 +371,10 @@ class PartitionedImage():
> for num in range(len(self.partitions)):
> part = self.partitions[num]
>
> + if self.ptable_format == 'msdos' and part.part_name:
> + raise WicError("setting custom partition name is not
> " \
> + "implemented for msdos partitions")
> +
> if self.ptable_format == 'msdos' and part.part_type:
> # The --part-type can also be implemented for MBR
> partitions, # in which case it would map to the 1-byte "partition
> type" @@ -373,12 +396,16 @@ class PartitionedImage():
> # Skip one sector required for the partitioning
> scheme overhead self.offset += overhead
>
> - if self.realpart > 3 and num_real_partitions > 4:
> + if self.ptable_format == "msdos":
> + if self.primary_part_num > 3 or \
> + (self.extendedpart == 0 and self.primary_part_num
> >= 3 and num_real_partitions > 4):
> + part.type = 'logical'
> # Reserve a sector for EBR for every logical
> partition # before alignment is performed.
> - if self.ptable_format == "msdos":
> - self.offset += 1
> + if part.type == 'logical':
> + self.offset += 2
>
> + align_sectors = 0
> if part.align:
> # If not first partition and we do have alignment
> set we need # to align the partition.
> @@ -401,21 +428,43 @@ class PartitionedImage():
> # increase the offset so we actually start the
> partition on right alignment self.offset += align_sectors
>
> + if part.offset is not None:
> + offset = (part.offset * 1024) // self.sector_size
> +
> + if offset * self.sector_size != part.offset * 1024:
> + raise WicError("Could not place %s%s at offset
> %dK with sector size %d" % (part.disk, self.numpart, part.offset,
> self.sector_size)) +
> + delta = offset - self.offset
> + if delta < 0:
> + raise WicError("Could not place %s%s at offset
> %dK: next free sector is %d (delta: %d)" % (part.disk, self.numpart,
> part.offset, self.offset, delta)) +
> + logger.debug("Skipping %d sectors to place %s%s at
> offset %dK",
> + delta, part.disk, self.numpart,
> part.offset) +
> + self.offset = offset
> +
> part.start = self.offset
> self.offset += part.size_sec
>
> - part.type = 'primary'
> if not part.no_table:
> part.num = self.realpart
> else:
> part.num = 0
>
> - if self.ptable_format == "msdos":
> - # only count the partitions that are in partition
> table
> - if num_real_partitions > 4:
> - if self.realpart > 3:
> - part.type = 'logical'
> - part.num = self.realpart + 1
> + if self.ptable_format == "msdos" and not part.no_table:
> + if part.type == 'logical':
> + self.logical_part_cnt += 1
> + part.num = self.logical_part_cnt + 4
> + if self.extendedpart == 0:
> + # Create extended partition as a primary
> partition
> + self.primary_part_num += 1
> + self.extendedpart = part.num
> + else:
> + self.extended_size_sec += align_sectors
> + self.extended_size_sec += part.size_sec + 2
> + else:
> + self.primary_part_num += 1
> + part.num = self.primary_part_num
>
> logger.debug("Assigned %s to %s%d, sectors range %d-%d
> size %d " "sectors (%d bytes).", part.mountpoint, part.disk,
> @@ -465,7 +514,7 @@ class PartitionedImage():
> if part.num == 0:
> continue
>
> - if self.ptable_format == "msdos" and part.num == 5:
> + if self.ptable_format == "msdos" and part.num ==
> self.extendedpart: # Create an extended partition (note: extended
> # partition is described in MBR and contains all
> # logical partitions). The logical partitions save a
> @@ -478,8 +527,8 @@ class PartitionedImage():
> # add a sector at the back, so that there is enough
> # room for all logical partitions.
> self._create_partition(self.path, "extended",
> - None, part.start - 1,
> - self.offset - part.start + 1)
> + None, part.start - 2,
> + self.extended_size_sec)
>
> if part.fstype == "swap":
> parted_fs_type = "linux-swap"
> @@ -487,8 +536,8 @@ class PartitionedImage():
> parted_fs_type = "fat32"
> elif part.fstype == "msdos":
> parted_fs_type = "fat16"
> - elif part.fstype == "ontrackdm6aux3":
> - parted_fs_type = "ontrackdm6aux3"
> + if not part.system_id:
> + part.system_id = '0x6' # FAT16
> else:
> # Type for ext2/ext3/ext4/btrfs
> parted_fs_type = "ext2"
> @@ -505,6 +554,13 @@ class PartitionedImage():
> self._create_partition(self.path, part.type,
> parted_fs_type, part.start,
> part.size_sec)
> + if part.part_name:
> + logger.debug("partition %d: set name to %s",
> + part.num, part.part_name)
> + exec_native_cmd("sgdisk --change-name=%d:%s %s" % \
> + (part.num, part.part_name,
> + self.path),
> self.native_sysroot) +
> if part.part_type:
> logger.debug("partition %d: set type UID to %s",
> part.num, part.part_type)
> @@ -538,21 +594,8 @@ class PartitionedImage():
> (self.path, part.num,
> part.system_id), self.native_sysroot)
>
> - # Parted defaults to enabling the lba flag for fat16
> partitions,
> - # which causes compatibility issues with some firmware
> (and really
> - # isn't necessary).
> - if parted_fs_type == "fat16":
> - if self.ptable_format == 'msdos':
> - logger.debug("Disable 'lba' flag for partition
> '%s' on disk '%s'",
> - part.num, self.path)
> - exec_native_cmd("parted -s %s set %d lba off" % \
> - (self.path, part.num),
> - self.native_sysroot)
> -
> def cleanup(self):
> - # remove partition images
> - for image in set(self.partimages):
> - os.remove(image)
> + pass
>
> def assemble(self):
> logger.debug("Installing partitions")
> @@ -561,7 +604,7 @@ class PartitionedImage():
> source = part.source_file
> if source:
> # install source_file contents into a partition
> - sparse_copy(source, self.path, part.start *
> self.sector_size)
> + sparse_copy(source, self.path, seek=part.start *
> self.sector_size)
> logger.debug("Installed %s in partition %d, sectors
> %d-%d, " "size %d sectors", source, part.num, part.start,
> diff --git a/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py
> b/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py new file mode
> 100644 index 0000000..5bd7390
> --- /dev/null
> +++ b/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py
> @@ -0,0 +1,213 @@
> +#
> +# This program is free software; you can redistribute it and/or
> modify +# it under the terms of the GNU General Public License
> version 2 as +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> along +# with this program; if not, write to the Free Software
> Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +#
> +# DESCRIPTION
> +# This implements the 'bootimg-biosplusefi' source plugin class for
> 'wic' +#
> +# AUTHORS
> +# William Bourque <wbourque [at) gmail.com>
> +
> +import types
> +
> +from wic.pluginbase import SourcePlugin
> +from importlib.machinery import SourceFileLoader
> +
> +class BootimgBiosPlusEFIPlugin(SourcePlugin):
> + """
> + Create MBR + EFI boot partition
> +
> + This plugin creates a boot partition that contains both
> + legacy BIOS and EFI content. It will be able to boot from both.
> + This is useful when managing PC fleet with some older machines
> + without EFI support.
> +
> + Note it is possible to create an image that can boot from both
> + legacy BIOS and EFI by defining two partitions : one with arg
> + --source bootimg-efi and another one with --source
> bootimg-pcbios.
> + However, this method has the obvious downside that it requires
> TWO
> + partitions to be created on the storage device.
> + Both partitions will also be marked as "bootable" which does not
> work on
> + most BIOS, has BIOS often uses the "bootable" flag to determine
> + what to boot. If you have such a BIOS, you need to manually
> remove the
> + "bootable" flag from the EFI partition for the drive to be
> bootable.
> + Having two partitions also seems to confuse wic : the content of
> + the first partition will be duplicated into the second, even
> though it
> + will not be used at all.
> +
> + Also, unlike "isoimage-isohybrid" that also does BIOS and EFI,
> this plugin
> + allows you to have more than only a single rootfs partitions and
> does
> + not turn the rootfs into an initramfs RAM image.
> +
> + This plugin is made to put everything into a single /boot
> partition so it
> + does not have the limitations listed above.
> +
> + The plugin is made so it does tries not to reimplement what's
> already
> + been done in other plugins; as such it imports "bootimg-pcbios"
> + and "bootimg-efi".
> + Plugin "bootimg-pcbios" is used to generate legacy BIOS boot.
> + Plugin "bootimg-efi" is used to generate the UEFI boot. Note
> that it
> + requires a --sourceparams argument to know which loader to use;
> refer
> + to "bootimg-efi" code/documentation for the list of loader.
> +
> + Imports are handled with "SourceFileLoader" from importlib as it
> is
> + otherwise very difficult to import module that has hyphen "-" in
> their
> + filename.
> + The SourcePlugin() methods used in the plugins (do_install_disk,
> + do_configure_partition, do_prepare_partition) are then called on
> both,
> + beginning by "bootimg-efi".
> +
> + Plugin options, such as "--sourceparams" can still be passed to a
> + plugin, as long they does not cause issue in the other plugin.
> +
> + Example wic configuration:
> + part /boot --source bootimg-biosplusefi
> --sourceparams="loader=grub-efi"\\
> + --ondisk sda --label os_boot --active --align 1024
> --use-uuid
> + """
> +
> + name = 'bootimg-biosplusefi'
> +
> + __PCBIOS_MODULE_NAME = "bootimg-pcbios"
> + __EFI_MODULE_NAME = "bootimg-efi"
> +
> + __imgEFIObj = None
> + __imgBiosObj = None
> +
> + @classmethod
> + def __init__(cls):
> + """
> + Constructor (init)
> + """
> +
> + # XXX
> + # For some reasons, __init__ constructor is never called.
> + # Something to do with how pluginbase works?
> + cls.__instanciateSubClasses()
> +
> + @classmethod
> + def __instanciateSubClasses(cls):
> + """
> +
> + """
> +
> + # Import bootimg-pcbios (class name "BootimgPcbiosPlugin")
> + modulePath =
> os.path.join(os.path.dirname(os.path.realpath(__file__)),
> + cls.__PCBIOS_MODULE_NAME + ".py")
> + loader = SourceFileLoader(cls.__PCBIOS_MODULE_NAME,
> modulePath)
> + mod = types.ModuleType(loader.name)
> + loader.exec_module(mod)
> + cls.__imgBiosObj = mod.BootimgPcbiosPlugin()
> +
> + # Import bootimg-efi (class name "BootimgEFIPlugin")
> + modulePath =
> os.path.join(os.path.dirname(os.path.realpath(__file__)),
> + cls.__EFI_MODULE_NAME + ".py")
> + loader = SourceFileLoader(cls.__EFI_MODULE_NAME, modulePath)
> + mod = types.ModuleType(loader.name)
> + loader.exec_module(mod)
> + cls.__imgEFIObj = mod.BootimgEFIPlugin()
> +
> + @classmethod
> + def do_install_disk(cls, disk, disk_name, creator, workdir,
> oe_builddir,
> + bootimg_dir, kernel_dir, native_sysroot):
> + """
> + Called after all partitions have been prepared and assembled
> into a
> + disk image.
> + """
> +
> + if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ):
> + cls.__instanciateSubClasses()
> +
> + cls.__imgEFIObj.do_install_disk(
> + disk,
> + disk_name,
> + creator,
> + workdir,
> + oe_builddir,
> + bootimg_dir,
> + kernel_dir,
> + native_sysroot)
> +
> + cls.__imgBiosObj.do_install_disk(
> + disk,
> + disk_name,
> + creator,
> + workdir,
> + oe_builddir,
> + bootimg_dir,
> + kernel_dir,
> + native_sysroot)
> +
> + @classmethod
> + def do_configure_partition(cls, part, source_params, creator,
> cr_workdir,
> + oe_builddir, bootimg_dir, kernel_dir,
> + native_sysroot):
> + """
> + Called before do_prepare_partition()
> + """
> +
> + if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ):
> + cls.__instanciateSubClasses()
> +
> + cls.__imgEFIObj.do_configure_partition(
> + part,
> + source_params,
> + creator,
> + cr_workdir,
> + oe_builddir,
> + bootimg_dir,
> + kernel_dir,
> + native_sysroot)
> +
> + cls.__imgBiosObj.do_configure_partition(
> + part,
> + source_params,
> + creator,
> + cr_workdir,
> + oe_builddir,
> + bootimg_dir,
> + kernel_dir,
> + native_sysroot)
> +
> + @classmethod
> + def do_prepare_partition(cls, part, source_params, creator,
> cr_workdir,
> + oe_builddir, bootimg_dir, kernel_dir,
> + rootfs_dir, native_sysroot):
> + """
> + Called to do the actual content population for a partition
> i.e. it
> + 'prepares' the partition to be incorporated into the image.
> + """
> +
> + if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ):
> + cls.__instanciateSubClasses()
> +
> + cls.__imgEFIObj.do_prepare_partition(
> + part,
> + source_params,
> + creator,
> + cr_workdir,
> + oe_builddir,
> + bootimg_dir,
> + kernel_dir,
> + rootfs_dir,
> + native_sysroot)
> +
> + cls.__imgBiosObj.do_prepare_partition(
> + part,
> + source_params,
> + creator,
> + cr_workdir,
> + oe_builddir,
> + bootimg_dir,
> + kernel_dir,
> + rootfs_dir,
> + native_sysroot)
> diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py
> b/scripts/lib/wic/plugins/source/bootimg-efi.py index
> 9879cb9..2cfdc10 100644 ---
> a/scripts/lib/wic/plugins/source/bootimg-efi.py +++
> b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -1,21 +1,7 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2014, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This implements the 'bootimg-efi' source plugin class for 'wic'
> @@ -31,8 +17,8 @@ import shutil
> from wic import WicError
> from wic.engine import get_custom_config
> from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import (exec_cmd, exec_native_cmd,
> get_bitbake_var,
> - BOOTDD_EXTRA_SPACE)
> +from wic.misc import (exec_cmd, exec_native_cmd,
> + get_bitbake_var, BOOTDD_EXTRA_SPACE)
>
> logger = logging.getLogger('wic')
>
> @@ -45,7 +31,7 @@ class BootimgEFIPlugin(SourcePlugin):
> name = 'bootimg-efi'
>
> @classmethod
> - def do_configure_grubefi(cls, creator, cr_workdir):
> + def do_configure_grubefi(cls, hdddir, creator, cr_workdir,
> source_params): """
> Create loader-specific (grub-efi) config
> """
> @@ -62,20 +48,52 @@ class BootimgEFIPlugin(SourcePlugin):
> raise WicError("configfile is specified but failed
> to " "get it from %s." % configfile)
>
> + initrd = source_params.get('initrd')
> +
> + if initrd:
> + bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> + if not bootimg_dir:
> + raise WicError("Couldn't find DEPLOY_DIR_IMAGE,
> exiting") +
> + initrds = initrd.split(';')
> + for rd in initrds:
> + cp_cmd = "cp %s/%s %s" % (bootimg_dir, rd, hdddir)
> + exec_cmd(cp_cmd, True)
> + else:
> + logger.debug("Ignoring missing initrd")
> +
> if not custom_cfg:
> # Create grub configuration using parameters from wks
> file bootloader = creator.ks.bootloader
> + title = source_params.get('title')
>
> grubefi_conf = ""
> grubefi_conf += "serial --unit=0 --speed=115200 --word=8
> --parity=no --stop=1\n" grubefi_conf += "default=boot\n"
> grubefi_conf += "timeout=%s\n" % bootloader.timeout
> - grubefi_conf += "menuentry 'boot'{\n"
> + grubefi_conf += "menuentry '%s'{\n" % (title if title
> else "boot") +
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME")) +
> + label = source_params.get('label')
> + label_conf = "root=%s" % creator.rootdev
> + if label:
> + label_conf = "LABEL=%s" % label
>
> - kernel = "/bzImage"
> + grubefi_conf += "linux /%s %s rootwait %s\n" \
> + % (kernel, label_conf, bootloader.append)
> +
> + if initrd:
> + initrds = initrd.split(';')
> + grubefi_conf += "initrd"
> + for rd in initrds:
> + grubefi_conf += " /%s" % rd
> + grubefi_conf += "\n"
>
> - grubefi_conf += "linux %s root=%s rootwait %s\n" \
> - % (kernel, creator.rootdev, bootloader.append)
> grubefi_conf += "}\n"
>
> logger.debug("Writing grubefi config
> %s/hdd/boot/EFI/BOOT/grub.cfg", @@ -109,8 +127,10 @@ class
> BootimgEFIPlugin(SourcePlugin): if not bootimg_dir:
> raise WicError("Couldn't find DEPLOY_DIR_IMAGE,
> exiting")
> - cp_cmd = "cp %s/%s %s" % (bootimg_dir, initrd, hdddir)
> - exec_cmd(cp_cmd, True)
> + initrds = initrd.split(';')
> + for rd in initrds:
> + cp_cmd = "cp %s/%s %s" % (bootimg_dir, rd, hdddir)
> + exec_cmd(cp_cmd, True)
> else:
> logger.debug("Ignoring missing initrd")
>
> @@ -135,16 +155,30 @@ class BootimgEFIPlugin(SourcePlugin):
>
> if not custom_cfg:
> # Create systemd-boot configuration using parameters
> from wks file
> - kernel = "/bzImage"
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME")) +
> + title = source_params.get('title')
>
> boot_conf = ""
> - boot_conf += "title boot\n"
> - boot_conf += "linux %s\n" % kernel
> - boot_conf += "options LABEL=Boot root=%s %s\n" % \
> - (creator.rootdev, bootloader.append)
> + boot_conf += "title %s\n" % (title if title else "boot")
> + boot_conf += "linux /%s\n" % kernel
> +
> + label = source_params.get('label')
> + label_conf = "LABEL=Boot root=%s" % creator.rootdev
> + if label:
> + label_conf = "LABEL=%s" % label
> +
> + boot_conf += "options %s %s\n" % \
> + (label_conf, bootloader.append)
>
> if initrd:
> - boot_conf += "initrd /%s\n" % initrd
> + initrds = initrd.split(';')
> + for rd in initrds:
> + boot_conf += "initrd /%s\n" % rd
>
> logger.debug("Writing systemd-boot config "
> "%s/hdd/boot/loader/entries/boot.conf",
> cr_workdir) @@ -167,7 +201,7 @@ class BootimgEFIPlugin(SourcePlugin):
>
> try:
> if source_params['loader'] == 'grub-efi':
> - cls.do_configure_grubefi(creator, cr_workdir)
> + cls.do_configure_grubefi(hdddir, creator,
> cr_workdir, source_params) elif source_params['loader'] ==
> 'systemd-boot': cls.do_configure_systemdboot(hdddir, creator,
> cr_workdir, source_params) else:
> @@ -194,8 +228,14 @@ class BootimgEFIPlugin(SourcePlugin):
>
> hdddir = "%s/hdd/boot" % cr_workdir
>
> - install_cmd = "install -m 0644 %s/bzImage %s/bzImage" % \
> - (staging_kernel_dir, hdddir)
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME")) +
> + install_cmd = "install -m 0644 %s/%s %s/%s" % \
> + (staging_kernel_dir, kernel, hdddir, kernel)
> exec_cmd(install_cmd)
>
>
> @@ -240,7 +280,10 @@ class BootimgEFIPlugin(SourcePlugin):
> # dosfs image, created by mkdosfs
> bootimg = "%s/boot.img" % cr_workdir
>
> - dosfs_cmd = "mkdosfs -n efi -C %s %d" % (bootimg, blocks)
> + label = part.label if part.label else "ESP"
> +
> + dosfs_cmd = "mkdosfs -n %s -i %s -C %s %d" % \
> + (label, part.fsuuid, bootimg, blocks)
> exec_native_cmd(dosfs_cmd, native_sysroot)
>
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
> diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py
> b/scripts/lib/wic/plugins/source/bootimg-partition.py index
> 13fddbd..138986a 100644 ---
> a/scripts/lib/wic/plugins/source/bootimg-partition.py +++
> b/scripts/lib/wic/plugins/source/bootimg-partition.py @@ -1,18 +1,5 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This implements the 'bootimg-partition' source plugin class for
> @@ -30,8 +17,9 @@ import re
> from glob import glob
>
> from wic import WicError
> +from wic.engine import get_custom_config
> from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import exec_cmd, get_bitbake_var
> +from wic.misc import exec_cmd, get_bitbake_var
>
> logger = logging.getLogger('wic')
>
> @@ -44,17 +32,13 @@ class BootimgPartitionPlugin(SourcePlugin):
> name = 'bootimg-partition'
>
> @classmethod
> - def do_prepare_partition(cls, part, source_params, cr,
> cr_workdir,
> + def do_configure_partition(cls, part, source_params, cr,
> cr_workdir, oe_builddir, bootimg_dir, kernel_dir,
> - rootfs_dir, native_sysroot):
> + native_sysroot):
> """
> - Called to do the actual content population for a partition
> i.e. it
> - 'prepares' the partition to be incorporated into the image.
> - In this case, does the following:
> - - sets up a vfat partition
> - - copies all files listed in IMAGE_BOOT_FILES variable
> + Called before do_prepare_partition(), create u-boot specific
> boot config """
> - hdddir = "%s/boot" % cr_workdir
> + hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
> install_cmd = "install -d %s" % hdddir
> exec_cmd(install_cmd)
>
> @@ -63,12 +47,19 @@ class BootimgPartitionPlugin(SourcePlugin):
> if not kernel_dir:
> raise WicError("Couldn't find DEPLOY_DIR_IMAGE,
> exiting")
> - logger.debug('Kernel dir: %s', bootimg_dir)
> + boot_files = None
> + for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s",
> part.label), (None, None)):
> + if fmt:
> + var = fmt % id
> + else:
> + var = ""
>
> - boot_files = get_bitbake_var("IMAGE_BOOT_FILES")
> + boot_files = get_bitbake_var("IMAGE_BOOT_FILES" + var)
> + if boot_files is not None:
> + break
>
> - if not boot_files:
> - raise WicError('No boot files defined, IMAGE_BOOT_FILES
> unset')
> + if boot_files is None:
> + raise WicError('No boot files defined, IMAGE_BOOT_FILES
> unset for entry #%d' % part.lineno)
> logger.debug('Boot files: %s', boot_files)
>
> @@ -85,9 +76,9 @@ class BootimgPartitionPlugin(SourcePlugin):
> logger.debug('Destination entry: %r', dst_entry)
> deploy_files.append(dst_entry)
>
> + cls.install_task = [];
> for deploy_entry in deploy_files:
> src, dst = deploy_entry
> - install_task = []
> if '*' in src:
> # by default install files under their basename
> entry_name_fn = os.path.basename
> @@ -102,22 +93,102 @@ class BootimgPartitionPlugin(SourcePlugin):
>
> logger.debug('Globbed sources: %s', ', '.join(srcs))
> for entry in srcs:
> + src = os.path.relpath(entry, kernel_dir)
> entry_dst_name = entry_name_fn(entry)
> - install_task.append((entry,
> - os.path.join(hdddir,
> -
> entry_dst_name)))
> + cls.install_task.append((src, entry_dst_name))
> else:
> - install_task = [(os.path.join(kernel_dir, src),
> - os.path.join(hdddir, dst))]
> + cls.install_task.append((src, dst))
> +
> + if source_params.get('loader') != "u-boot":
> + return
> +
> + configfile = cr.ks.bootloader.configfile
> + custom_cfg = None
> + if configfile:
> + custom_cfg = get_custom_config(configfile)
> + if custom_cfg:
> + # Use a custom configuration for extlinux.conf
> + extlinux_conf = custom_cfg
> + logger.debug("Using custom configuration file "
> + "%s for extlinux.cfg", configfile)
> + else:
> + raise WicError("configfile is specified but failed
> to "
> + "get it from %s." % configfile)
> +
> + if not custom_cfg:
> + # The kernel types supported by the sysboot of u-boot
> + kernel_types = ["zImage", "Image", "fitImage", "uImage",
> "vmlinux"]
> + has_dtb = False
> + fdt_dir = '/'
> + kernel_name = None
> +
> + # Find the kernel image name, from the highest
> precedence to lowest
> + for image in kernel_types:
> + for task in cls.install_task:
> + src, dst = task
> + if re.match(image, src):
> + kernel_name = os.path.join('/', dst)
> + break
> + if kernel_name:
> + break
> +
> + for task in cls.install_task:
> + src, dst = task
> + # We suppose that all the dtb are in the same
> directory
> + if re.search(r'\.dtb', src) and fdt_dir == '/':
> + has_dtb = True
> + fdt_dir = os.path.join(fdt_dir,
> os.path.dirname(dst))
> + break
> +
> + if not kernel_name:
> + raise WicError('No kernel file founded')
> +
> + # Compose the extlinux.conf
> + extlinux_conf = "default Yocto\n"
> + extlinux_conf += "label Yocto\n"
> + extlinux_conf += " kernel %s\n" % kernel_name
> + if has_dtb:
> + extlinux_conf += " fdtdir %s\n" % fdt_dir
> + bootloader = cr.ks.bootloader
> + extlinux_conf += "append root=%s rootwait %s\n" \
> + % (cr.rootdev, bootloader.append if
> bootloader.append else '') +
> + install_cmd = "install -d %s/extlinux/" % hdddir
> + exec_cmd(install_cmd)
> + cfg = open("%s/extlinux/extlinux.conf" % hdddir, "w")
> + cfg.write(extlinux_conf)
> + cfg.close()
> +
> +
> + @classmethod
> + def do_prepare_partition(cls, part, source_params, cr,
> cr_workdir,
> + oe_builddir, bootimg_dir, kernel_dir,
> + rootfs_dir, native_sysroot):
> + """
> + Called to do the actual content population for a partition
> i.e. it
> + 'prepares' the partition to be incorporated into the image.
> + In this case, does the following:
> + - sets up a vfat partition
> + - copies all files listed in IMAGE_BOOT_FILES variable
> + """
> + hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
> +
> + if not kernel_dir:
> + kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> + if not kernel_dir:
> + raise WicError("Couldn't find DEPLOY_DIR_IMAGE,
> exiting") +
> + logger.debug('Kernel dir: %s', bootimg_dir)
> +
>
> - for task in install_task:
> - src_path, dst_path = task
> - logger.debug('Install %s as %s',
> - os.path.basename(src_path), dst_path)
> - install_cmd = "install -m 0644 -D %s %s" \
> - % (src_path, dst_path)
> - exec_cmd(install_cmd)
> + for task in cls.install_task:
> + src_path, dst_path = task
> + logger.debug('Install %s as %s', src_path, dst_path)
> + install_cmd = "install -m 0644 -D %s %s" \
> + % (os.path.join(kernel_dir, src_path),
> + os.path.join(hdddir, dst_path))
> + exec_cmd(install_cmd)
>
> logger.debug('Prepare boot partition using rootfs in %s',
> hdddir) part.prepare_rootfs(cr_workdir, oe_builddir, hdddir,
> - native_sysroot)
> + native_sysroot, False)
> diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
> b/scripts/lib/wic/plugins/source/bootimg-pcbios.py index
> 11db304..f2639e7 100644 ---
> a/scripts/lib/wic/plugins/source/bootimg-pcbios.py +++
> b/scripts/lib/wic/plugins/source/bootimg-pcbios.py @@ -1,21 +1,7 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2014, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This implements the 'bootimg-pcbios' source plugin class for 'wic'
> @@ -26,13 +12,13 @@
>
> import logging
> import os
> +import re
>
> from wic import WicError
> from wic.engine import get_custom_config
> -from wic.utils import runner
> from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import (exec_cmd, exec_native_cmd,
> - get_bitbake_var, BOOTDD_EXTRA_SPACE)
> +from wic.misc import (exec_cmd, exec_native_cmd,
> + get_bitbake_var, BOOTDD_EXTRA_SPACE)
>
> logger = logging.getLogger('wic')
>
> @@ -44,19 +30,22 @@ class BootimgPcbiosPlugin(SourcePlugin):
> name = 'bootimg-pcbios'
>
> @classmethod
> - def _get_syslinux_dir(cls, bootimg_dir):
> + def _get_bootimg_dir(cls, bootimg_dir, dirname):
> """
> - Get path to syslinux from either default bootimg_dir
> - or wic-tools STAGING_DIR.
> + Check if dirname exists in default bootimg_dir or in
> STAGING_DIR. """
> - for path in (bootimg_dir, get_bitbake_var("STAGING_DATADIR",
> "wic-tools")):
> - if not path:
> - continue
> - syslinux_dir = os.path.join(path, 'syslinux')
> - if os.path.exists(syslinux_dir):
> - return syslinux_dir
> + staging_datadir = get_bitbake_var("STAGING_DATADIR")
> + for result in (bootimg_dir, staging_datadir):
> + if os.path.exists("%s/%s" % (result, dirname)):
> + return result
> +
> + # STAGING_DATADIR is expanded with MLPREFIX if multilib is
> enabled
> + # but dependency syslinux is still populated to original
> STAGING_DATADIR
> + nonarch_datadir = re.sub('/[^/]*recipe-sysroot',
> '/recipe-sysroot', staging_datadir)
> + if os.path.exists(os.path.join(nonarch_datadir, dirname)):
> + return nonarch_datadir
>
> - raise WicError("Couldn't find syslinux directory, exiting")
> + raise WicError("Couldn't find correct bootimg_dir, exiting")
>
> @classmethod
> def do_install_disk(cls, disk, disk_name, creator, workdir,
> oe_builddir, @@ -65,11 +54,12 @@ class
> BootimgPcbiosPlugin(SourcePlugin): Called after all partitions have
> been prepared and assembled into a disk image. In this case, we
> install the MBR. """
> - syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
> + bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux')
> + mbrfile = "%s/syslinux/" % bootimg_dir
> if creator.ptable_format == 'msdos':
> - mbrfile = os.path.join(syslinux_dir, "mbr.bin")
> + mbrfile += "mbr.bin"
> elif creator.ptable_format == 'gpt':
> - mbrfile = os.path.join(syslinux_dir, "gptmbr.bin")
> + mbrfile += "gptmbr.bin"
> else:
> raise WicError("Unsupported partition table: %s" %
> creator.ptable_format)
> @@ -83,10 +73,8 @@ class BootimgPcbiosPlugin(SourcePlugin):
> logger.debug("Installing MBR on disk %s as %s with size %s
> bytes", disk_name, full_path, disk.min_size)
>
> - rcode = runner.show(['dd', 'if=%s' % mbrfile,
> - 'of=%s' % full_path, 'conv=notrunc'])
> - if rcode != 0:
> - raise WicError("Unable to set MBR to %s" % full_path)
> + dd_cmd = "dd if=%s of=%s conv=notrunc" % (mbrfile, full_path)
> + exec_cmd(dd_cmd, native_sysroot)
>
> @classmethod
> def do_configure_partition(cls, part, source_params, creator,
> cr_workdir, @@ -155,22 +143,28 @@ class
> BootimgPcbiosPlugin(SourcePlugin): 'prepares' the partition to be
> incorporated into the image. In this case, prepare content for legacy
> bios boot partition. """
> - syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
> + bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux')
>
> staging_kernel_dir = kernel_dir
>
> hdddir = "%s/hdd/boot" % cr_workdir
>
> - cmds = ("install -m 0644 %s/bzImage %s/vmlinuz" %
> - (staging_kernel_dir, hdddir),
> - "install -m 444 %s/ldlinux.sys %s/ldlinux.sys" %
> - (syslinux_dir, hdddir),
> - "install -m 0644 %s/vesamenu.c32 %s/vesamenu.c32" %
> - (syslinux_dir, hdddir),
> - "install -m 444 %s/libcom32.c32 %s/libcom32.c32" %
> - (syslinux_dir, hdddir),
> - "install -m 444 %s/libutil.c32 %s/libutil.c32" %
> - (syslinux_dir, hdddir))
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME")) +
> + cmds = ("install -m 0644 %s/%s %s/vmlinuz" %
> + (staging_kernel_dir, kernel, hdddir),
> + "install -m 444 %s/syslinux/ldlinux.sys
> %s/ldlinux.sys" %
> + (bootimg_dir, hdddir),
> + "install -m 0644 %s/syslinux/vesamenu.c32
> %s/vesamenu.c32" %
> + (bootimg_dir, hdddir),
> + "install -m 444 %s/syslinux/libcom32.c32
> %s/libcom32.c32" %
> + (bootimg_dir, hdddir),
> + "install -m 444 %s/syslinux/libutil.c32
> %s/libutil.c32" %
> + (bootimg_dir, hdddir))
>
> for install_cmd in cmds:
> exec_cmd(install_cmd)
> @@ -190,9 +184,10 @@ class BootimgPcbiosPlugin(SourcePlugin):
> extra_blocks, part.mountpoint, blocks)
>
> # dosfs image, created by mkdosfs
> - bootimg = "%s/boot.img" % cr_workdir
> + bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno)
>
> - dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg,
> blocks)
> + dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \
> + (part.fsuuid, bootimg, blocks)
> exec_native_cmd(dosfs_cmd, native_sysroot)
>
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
> diff --git a/scripts/lib/wic/plugins/source/fsimage.py
> b/scripts/lib/wic/plugins/source/fsimage.py deleted file mode 100644
> index f781499..0000000
> --- a/scripts/lib/wic/plugins/source/fsimage.py
> +++ /dev/null
> @@ -1,56 +0,0 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> -#
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. -#
> -
> -import logging
> -import os
> -
> -from wic import WicError
> -from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import get_bitbake_var
> -
> -logger = logging.getLogger('wic')
> -
> -class FSImagePlugin(SourcePlugin):
> - """
> - Add an already existing filesystem image to the partition layout.
> - """
> -
> - name = 'fsimage'
> -
> - @classmethod
> - def do_prepare_partition(cls, part, source_params, cr,
> cr_workdir,
> - oe_builddir, bootimg_dir, kernel_dir,
> - rootfs_dir, native_sysroot):
> - """
> - Called to do the actual content population for a partition
> i.e. it
> - 'prepares' the partition to be incorporated into the image.
> - """
> - if not bootimg_dir:
> - bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> - if not bootimg_dir:
> - raise WicError("Couldn't find DEPLOY_DIR_IMAGE,
> exiting") -
> - logger.debug('Bootimg dir: %s', bootimg_dir)
> -
> - if 'file' not in source_params:
> - raise WicError("No file specified")
> -
> - src = os.path.join(bootimg_dir, source_params['file'])
> -
> -
> - logger.debug('Preparing partition using image %s', src)
> - part.prepare_rootfs_from_fs_image(cr_workdir, src, "")
> diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
> b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py index
> 1ceba62..11326a2 100644 ---
> a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py +++
> b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py @@ -1,18 +1,5
> @@ -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> -
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> #
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This implements the 'isoimage-isohybrid' source plugin class for
> 'wic' @@ -29,7 +16,7 @@ import shutil
> from wic import WicError
> from wic.engine import get_custom_config
> from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var
> +from wic.misc import exec_cmd, exec_native_cmd, get_bitbake_var
>
> logger = logging.getLogger('wic')
>
> @@ -47,7 +34,7 @@ class IsoImagePlugin(SourcePlugin):
>
> Example kickstart file:
> part /boot --source isoimage-isohybrid
> --sourceparams="loader=grub-efi, \\
> - image_name= IsoImage" --ondisk cd --label LIVECD --fstype=ext2
> + image_name= IsoImage" --ondisk cd --label LIVECD
> bootloader --timeout=10 --append=" "
>
> In --sourceparams "loader" specifies the bootloader used for
> booting in EFI @@ -83,8 +70,13 @@ class IsoImagePlugin(SourcePlugin):
> syslinux_conf += "DEFAULT boot\n"
> syslinux_conf += "LABEL boot\n"
>
> - kernel = "/bzImage"
> - syslinux_conf += "KERNEL " + kernel + "\n"
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME")) +
> + syslinux_conf += "KERNEL /" + kernel + "\n"
> syslinux_conf += "APPEND initrd=/initrd LABEL=boot %s\n" \
> % bootloader.append
>
> @@ -95,7 +87,7 @@ class IsoImagePlugin(SourcePlugin):
> cfg.write(syslinux_conf)
>
> @classmethod
> - def do_configure_grubefi(cls, part, creator, cr_workdir):
> + def do_configure_grubefi(cls, part, creator, target_dir):
> """
> Create loader-specific (grub-efi) config
> """
> @@ -109,7 +101,7 @@ class IsoImagePlugin(SourcePlugin):
> raise WicError("configfile is specified "
> "but failed to get it from %s",
> configfile) else:
> - splash = os.path.join(cr_workdir, "EFI/boot/splash.jpg")
> + splash = os.path.join(target_dir, "splash.jpg")
> if os.path.exists(splash):
> splashline = "menu background splash.jpg"
> else:
> @@ -127,9 +119,13 @@ class IsoImagePlugin(SourcePlugin):
> grubefi_conf += "\n"
> grubefi_conf += "menuentry 'boot'{\n"
>
> - kernel = "/bzImage"
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME"))
> - grubefi_conf += "linux %s rootwait %s\n" \
> + grubefi_conf += "linux /%s rootwait %s\n" \
> % (kernel, bootloader.append)
> grubefi_conf += "initrd /initrd \n"
> grubefi_conf += "}\n"
> @@ -137,9 +133,10 @@ class IsoImagePlugin(SourcePlugin):
> if splashline:
> grubefi_conf += "%s\n" % splashline
>
> - logger.debug("Writing grubefi config %s/EFI/BOOT/grub.cfg",
> cr_workdir)
> + cfg_path = os.path.join(target_dir, "grub.cfg")
> + logger.debug("Writing grubefi config %s", cfg_path)
>
> - with open("%s/EFI/BOOT/grub.cfg" % cr_workdir, "w") as cfg:
> + with open(cfg_path, "w") as cfg:
> cfg.write(grubefi_conf)
>
> @staticmethod
> @@ -162,13 +159,14 @@ class IsoImagePlugin(SourcePlugin):
> if not image_type:
> raise WicError("Couldn't find INITRAMFS_FSTYPES,
> exiting.")
> - target_arch = get_bitbake_var("TRANSLATED_TARGET_ARCH")
> - if not target_arch:
> - raise WicError("Couldn't find
> TRANSLATED_TARGET_ARCH, exiting.")
> + machine = os.path.basename(initrd_dir)
>
> - initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir,
> image_name, target_arch, image_type))[0]
> + pattern = '%s/%s*%s.%s' % (initrd_dir, image_name,
> machine, image_type)
> + files = glob.glob(pattern)
> + if files:
> + initrd = files[0]
>
> - if not os.path.exists(initrd):
> + if not initrd or not os.path.exists(initrd):
> # Create initrd from rootfs directory
> initrd = "%s/initrd.cpio.gz" % cr_workdir
> initrd_dir = "%s/INITRD" % cr_workdir
> @@ -189,10 +187,9 @@ class IsoImagePlugin(SourcePlugin):
> else:
> raise WicError("Couldn't find or build initrd,
> exiting.")
> - exec_cmd("cd %s && find . | cpio -o -H newc -R +0:+0
> >./initrd.cpio " \
> - % initrd_dir, as_shell=True)
> - exec_cmd("gzip -f -9 -c %s/initrd.cpio > %s" \
> - % (cr_workdir, initrd), as_shell=True)
> + exec_cmd("cd %s && find . | cpio -o -H newc -R root:root
> >%s/initrd.cpio " \
> + % (initrd_dir, cr_workdir), as_shell=True)
> + exec_cmd("gzip -f -9 %s/initrd.cpio" % cr_workdir,
> as_shell=True) shutil.rmtree(initrd_dir)
>
> return initrd
> @@ -206,8 +203,8 @@ class IsoImagePlugin(SourcePlugin):
> """
> isodir = "%s/ISO/" % cr_workdir
>
> - if os.path.exists(cr_workdir):
> - shutil.rmtree(cr_workdir)
> + if os.path.exists(isodir):
> + shutil.rmtree(isodir)
>
> install_cmd = "install -d %s " % isodir
> exec_cmd(install_cmd)
> @@ -251,33 +248,8 @@ class IsoImagePlugin(SourcePlugin):
> raise WicError("Couldn't find IMAGE_ROOTFS, exiting.")
>
> part.rootfs_dir = rootfs_dir
> -
> - # Prepare rootfs.img
> deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> img_iso_dir = get_bitbake_var("ISODIR")
> - rootfs_img = "%s/rootfs.img" % img_iso_dir
> - if not os.path.isfile(rootfs_img):
> - # check if rootfs.img is in deploydir
> - deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> - image_name = get_bitbake_var("IMAGE_LINK_NAME")
> - rootfs_img = "%s/%s.%s" \
> - % (deploy_dir, image_name, part.fstype)
> -
> - if not os.path.isfile(rootfs_img):
> - # create image file with type specified by --fstype
> - # which contains rootfs
> - du_cmd = "du -bks %s" % rootfs_dir
> - out = exec_cmd(du_cmd)
> - part.size = int(out.split()[0])
> - part.extra_space = 0
> - part.overhead_factor = 1.2
> - part.prepare_rootfs(cr_workdir, oe_builddir, rootfs_dir,
> \
> - native_sysroot)
> - rootfs_img = part.source_file
> -
> - install_cmd = "install -m 0644 %s %s/rootfs.img" \
> - % (rootfs_img, isodir)
> - exec_cmd(install_cmd)
>
> # Remove the temporary file created by part.prepare_rootfs()
> if os.path.isfile(part.source_file):
> @@ -305,27 +277,25 @@ class IsoImagePlugin(SourcePlugin):
> if os.path.isfile("%s/initrd.cpio.gz" % cr_workdir):
> os.remove("%s/initrd.cpio.gz" % cr_workdir)
>
> - # Install bzImage
> - install_cmd = "install -m 0644 %s/bzImage %s/bzImage" % \
> - (kernel_dir, isodir)
> + kernel = get_bitbake_var("KERNEL_IMAGETYPE")
> + if get_bitbake_var("INITRAMFS_IMAGE_BUNDLE") == "1":
> + if get_bitbake_var("INITRAMFS_IMAGE"):
> + kernel = "%s-%s.bin" % \
> + (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME")) +
> + install_cmd = "install -m 0644 %s/%s %s/%s" % \
> + (kernel_dir, kernel, isodir, kernel)
> exec_cmd(install_cmd)
>
> #Create bootloader for efi boot
> try:
> - if source_params['loader'] == 'grub-efi':
> - # Builds grub.cfg if ISODIR didn't exist or
> - # didn't contains grub.cfg
> - bootimg_dir = img_iso_dir
> - if not os.path.exists("%s/EFI/BOOT" % bootimg_dir):
> - bootimg_dir = "%s/bootimg" % cr_workdir
> - if os.path.exists(bootimg_dir):
> - shutil.rmtree(bootimg_dir)
> - install_cmd = "install -d %s/EFI/BOOT" %
> bootimg_dir
> - exec_cmd(install_cmd)
> -
> - if not os.path.isfile("%s/EFI/BOOT/boot.cfg" %
> bootimg_dir):
> - cls.do_configure_grubefi(part, creator,
> bootimg_dir)
> + target_dir = "%s/EFI/BOOT" % isodir
> + if os.path.exists(target_dir):
> + shutil.rmtree(target_dir)
>
> + os.makedirs(target_dir)
> +
> + if source_params['loader'] == 'grub-efi':
> # Builds bootx64.efi/bootia32.efi if ISODIR didn't
> exist or # didn't contains it
> target_arch = get_bitbake_var("TARGET_SYS")
> @@ -333,37 +303,25 @@ class IsoImagePlugin(SourcePlugin):
> raise WicError("Coludn't find target
> architecture")
> if re.match("x86_64", target_arch):
> - grub_target = 'x86_64-efi'
> - grub_image = "bootx64.efi"
> + grub_src_image = "grub-efi-bootx64.efi"
> + grub_dest_image = "bootx64.efi"
> elif re.match('i.86', target_arch):
> - grub_target = 'i386-efi'
> - grub_image = "bootia32.efi"
> + grub_src_image = "grub-efi-bootia32.efi"
> + grub_dest_image = "bootia32.efi"
> else:
> raise WicError("grub-efi is incompatible with
> target %s" % target_arch)
>
> - if not os.path.isfile("%s/EFI/BOOT/%s" \
> - % (bootimg_dir, grub_image)):
> - grub_path = get_bitbake_var("STAGING_LIBDIR",
> "wic-tools")
> - if not grub_path:
> - raise WicError("Couldn't find
> STAGING_LIBDIR, exiting.") -
> - grub_core = "%s/grub/%s" % (grub_path,
> grub_target)
> - if not os.path.exists(grub_core):
> - raise WicError("Please build grub-efi first")
> -
> - grub_cmd = "grub-mkimage -p '/EFI/BOOT' "
> - grub_cmd += "-d %s " % grub_core
> - grub_cmd += "-O %s -o %s/EFI/BOOT/%s " \
> - % (grub_target, bootimg_dir,
> grub_image)
> - grub_cmd += "part_gpt part_msdos ntfs ntfscomp
> fat ext2 "
> - grub_cmd += "normal chain boot configfile linux
> multiboot "
> - grub_cmd += "search efi_gop efi_uga font gfxterm
> gfxmenu "
> - grub_cmd += "terminal minicmd test iorw loadenv
> echo help "
> - grub_cmd += "reboot serial terminfo iso9660
> loopback tar "
> - grub_cmd += "memdisk ls search_fs_uuid udf btrfs
> xfs lvm "
> - grub_cmd += "reiserfs ata "
> - exec_native_cmd(grub_cmd, native_sysroot)
> + grub_target = os.path.join(target_dir,
> grub_dest_image)
> + if not os.path.isfile(grub_target):
> + grub_src = os.path.join(deploy_dir,
> grub_src_image)
> + if not os.path.exists(grub_src):
> + raise WicError("Grub loader %s is not found
> in %s. "
> + "Please build grub-efi first"
> % (grub_src_image, deploy_dir))
> + shutil.copy(grub_src, grub_target)
> +
> + if not os.path.isfile(os.path.join(target_dir,
> "boot.cfg")):
> + cls.do_configure_grubefi(part, creator,
> target_dir)
> else:
> raise WicError("unrecognized bootimg-efi loader: %s"
> % @@ -371,15 +329,6 @@ class IsoImagePlugin(SourcePlugin):
> except KeyError:
> raise WicError("bootimg-efi requires a loader, none
> specified")
> - if os.path.exists("%s/EFI/BOOT" % isodir):
> - shutil.rmtree("%s/EFI/BOOT" % isodir)
> -
> - shutil.copytree(bootimg_dir+"/EFI/BOOT", isodir+"/EFI/BOOT")
> -
> - # If exists, remove cr_workdir/bootimg temporary folder
> - if os.path.exists("%s/bootimg" % cr_workdir):
> - shutil.rmtree("%s/bootimg" % cr_workdir)
> -
> # Create efi.img that contains bootloader files for EFI
> booting # if ISODIR didn't exist or didn't contains it
> if os.path.isfile("%s/efi.img" % img_iso_dir):
> @@ -387,19 +336,23 @@ class IsoImagePlugin(SourcePlugin):
> (img_iso_dir, isodir)
> exec_cmd(install_cmd)
> else:
> + # Default to 100 blocks of extra space for file system
> overhead
> + esp_extra_blocks =
> int(source_params.get('esp_extra_blocks', '100')) +
> du_cmd = "du -bks %s/EFI" % isodir
> out = exec_cmd(du_cmd)
> blocks = int(out.split()[0])
> - # Add some extra space for file system overhead
> - blocks += 100
> + blocks += esp_extra_blocks
> logger.debug("Added 100 extra blocks to %s to get to %d "
> "total blocks", part.mountpoint, blocks)
>
> # dosfs image for EFI boot
> bootimg = "%s/efi.img" % isodir
>
> - dosfs_cmd = 'mkfs.vfat -n "EFIimg" -S 512 -C %s %d' \
> - % (bootimg, blocks)
> + esp_label = source_params.get('esp_label', 'EFIimg')
> +
> + dosfs_cmd = 'mkfs.vfat -n \'%s\' -S 512 -C %s %d' \
> + % (esp_label, bootimg, blocks)
> exec_native_cmd(dosfs_cmd, native_sysroot)
>
> mmd_cmd = "mmd -i %s ::/EFI" % bootimg
> @@ -413,7 +366,7 @@ class IsoImagePlugin(SourcePlugin):
> exec_cmd(chmod_cmd)
>
> # Prepare files for legacy boot
> - syslinux_dir = get_bitbake_var("STAGING_DATADIR",
> "wic-tools")
> + syslinux_dir = get_bitbake_var("STAGING_DATADIR")
> if not syslinux_dir:
> raise WicError("Couldn't find STAGING_DATADIR, exiting.")
>
> diff --git a/scripts/lib/wic/plugins/source/rawcopy.py
> b/scripts/lib/wic/plugins/source/rawcopy.py index e1c4f5e..3c4997d
> 100644 --- a/scripts/lib/wic/plugins/source/rawcopy.py
> +++ b/scripts/lib/wic/plugins/source/rawcopy.py
> @@ -1,18 +1,5 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
>
> import logging
> @@ -20,7 +7,7 @@ import os
>
> from wic import WicError
> from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import exec_cmd, get_bitbake_var
> +from wic.misc import exec_cmd, get_bitbake_var
> from wic.filemap import sparse_copy
>
> logger = logging.getLogger('wic')
> @@ -32,6 +19,25 @@ class RawCopyPlugin(SourcePlugin):
>
> name = 'rawcopy'
>
> + @staticmethod
> + def do_image_label(fstype, dst, label):
> + if fstype.startswith('ext'):
> + cmd = 'tune2fs -L %s %s' % (label, dst)
> + elif fstype in ('msdos', 'vfat'):
> + cmd = 'dosfslabel %s %s' % (dst, label)
> + elif fstype == 'btrfs':
> + cmd = 'btrfs filesystem label %s %s' % (dst, label)
> + elif fstype == 'swap':
> + cmd = 'mkswap -L %s %s' % (label, dst)
> + elif fstype == 'squashfs':
> + raise WicError("It's not possible to update a squashfs "
> + "filesystem label '%s'" % (label))
> + else:
> + raise WicError("Cannot update filesystem label: "
> + "Unknown fstype: '%s'" % (fstype))
> +
> + exec_cmd(cmd)
> +
> @classmethod
> def do_prepare_partition(cls, part, source_params, cr,
> cr_workdir, oe_builddir, bootimg_dir, kernel_dir,
> @@ -51,7 +57,10 @@ class RawCopyPlugin(SourcePlugin):
> raise WicError("No file specified")
>
> src = os.path.join(kernel_dir, source_params['file'])
> - dst = os.path.join(cr_workdir, "%s.%s" %
> (source_params['file'], part.lineno))
> + dst = os.path.join(cr_workdir, "%s.%s" %
> (os.path.basename(source_params['file']), part.lineno)) +
> + if not os.path.exists(os.path.dirname(dst)):
> + os.makedirs(os.path.dirname(dst))
>
> if 'skip' in source_params:
> sparse_copy(src, dst, skip=int(source_params['skip']))
> @@ -66,4 +75,7 @@ class RawCopyPlugin(SourcePlugin):
> if filesize > part.size:
> part.size = filesize
>
> + if part.label:
> + RawCopyPlugin.do_image_label(part.fstype, dst,
> part.label) +
> part.source_file = dst
> diff --git a/scripts/lib/wic/plugins/source/rootfs.py
> b/scripts/lib/wic/plugins/source/rootfs.py index f2e2ca8..f1db83f
> 100644 --- a/scripts/lib/wic/plugins/source/rootfs.py
> +++ b/scripts/lib/wic/plugins/source/rootfs.py
> @@ -1,21 +1,7 @@
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2014, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION
> # This implements the 'rootfs' source plugin class for 'wic'
> @@ -28,12 +14,14 @@
> import logging
> import os
> import shutil
> +import sys
>
> from oe.path import copyhardlinktree
> +from pathlib import Path
>
> from wic import WicError
> from wic.pluginbase import SourcePlugin
> -from wic.utils.misc import get_bitbake_var, exec_cmd
> +from wic.misc import get_bitbake_var, exec_native_cmd
>
> logger = logging.getLogger('wic')
>
> @@ -44,10 +32,26 @@ class RootfsPlugin(SourcePlugin):
>
> name = 'rootfs'
>
> + @staticmethod
> + def __validate_path(cmd, rootfs_dir, path):
> + if os.path.isabs(path):
> + logger.error("%s: Must be relative: %s" % (cmd,
> orig_path))
> + sys.exit(1)
> +
> + # Disallow climbing outside of parent directory using '..',
> + # because doing so could be quite disastrous (we will delete
> the
> + # directory, or modify a directory outside OpenEmbedded).
> + full_path = os.path.realpath(os.path.join(rootfs_dir, path))
> + if not full_path.startswith(os.path.realpath(rootfs_dir)):
> + logger.error("%s: Must point inside the rootfs:" % (cmd,
> path))
> + sys.exit(1)
> +
> + return full_path
> +
> @staticmethod
> def __get_rootfs_dir(rootfs_dir):
> if os.path.isdir(rootfs_dir):
> - return rootfs_dir
> + return os.path.realpath(rootfs_dir)
>
> image_rootfs_dir = get_bitbake_var("IMAGE_ROOTFS",
> rootfs_dir) if not os.path.isdir(image_rootfs_dir):
> @@ -55,7 +59,16 @@ class RootfsPlugin(SourcePlugin):
> "named %s has been found at %s, exiting."
> % (rootfs_dir, image_rootfs_dir))
>
> - return image_rootfs_dir
> + return os.path.realpath(image_rootfs_dir)
> +
> + @staticmethod
> + def __get_pseudo(native_sysroot, rootfs, pseudo_dir):
> + pseudo = "export PSEUDO_PREFIX=%s/usr;" % native_sysroot
> + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir
> + pseudo += "export PSEUDO_PASSWD=%s;" % rootfs
> + pseudo += "export PSEUDO_NOSYMLINKEXP=1;"
> + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
> + return pseudo
>
> @classmethod
> def do_prepare_partition(cls, part, source_params, cr,
> cr_workdir, @@ -80,33 +93,111 @@ class RootfsPlugin(SourcePlugin):
> raise WicError("Couldn't find --rootfs-dir=%s
> connection or " "it is not a valid path, exiting" % part.rootfs_dir)
>
> - real_rootfs_dir = cls.__get_rootfs_dir(rootfs_dir)
> + part.rootfs_dir = cls.__get_rootfs_dir(rootfs_dir)
> + pseudo_dir = os.path.join(part.rootfs_dir, "../pseudo")
> + if not os.path.lexists(pseudo_dir):
> + logger.warn("%s folder does not exist. "
> + "Usernames and permissions will be invalid "
> % pseudo_dir)
> + pseudo_dir = None
>
> + new_rootfs = None
> + new_pseudo = None
> # Handle excluded paths.
> - if part.exclude_path is not None:
> + if part.exclude_path or part.include_path or
> part.change_directory: # We need a new rootfs directory we can delete
> files from. Copy to # workdir.
> - new_rootfs = os.path.realpath(os.path.join(cr_workdir,
> "rootfs"))
> + new_rootfs = os.path.realpath(os.path.join(cr_workdir,
> "rootfs%d" % part.lineno))
> if os.path.lexists(new_rootfs):
> shutil.rmtree(os.path.join(new_rootfs))
>
> - copyhardlinktree(real_rootfs_dir, new_rootfs)
> + if part.change_directory:
> + cd = part.change_directory
> + if cd[-1] == '/':
> + cd = cd[:-1]
> + orig_dir = cls.__validate_path("--change-directory",
> part.rootfs_dir, cd)
> + else:
> + orig_dir = part.rootfs_dir
> + copyhardlinktree(orig_dir, new_rootfs)
> +
> + # Convert the pseudo directory to its new location
> + if (pseudo_dir):
> + new_pseudo = os.path.realpath(
> + os.path.join(cr_workdir, "pseudo%d" %
> part.lineno))
> + if os.path.lexists(new_pseudo):
> + shutil.rmtree(new_pseudo)
> + os.mkdir(new_pseudo)
> + shutil.copy(os.path.join(pseudo_dir, "files.db"),
> + os.path.join(new_pseudo, "files.db"))
> +
> + pseudo_cmd = "%s -B -m %s -M %s" %
> (cls.__get_pseudo(native_sysroot,
> +
> new_rootfs,
> +
> new_pseudo),
> + orig_dir,
> new_rootfs)
> + exec_native_cmd(pseudo_cmd, native_sysroot)
> +
> + for in_path in part.include_path or []:
> + #parse arguments
> + include_path = in_path[0]
> + if len(in_path) > 2:
> + logger.error("'Invalid number of arguments for
> include-path")
> + sys.exit(1)
> + if len(in_path) == 2:
> + path = in_path[1]
> + else:
> + path = None
> +
> + # Pack files to be included into a tar file.
> + # We need to create a tar file, because that way we
> can keep the
> + # permissions from the files even when they belong
> to different
> + # pseudo enviroments.
> + # If we simply copy files using
> copyhardlinktree/copytree... the
> + # copied files will belong to the user running wic.
> + tar_file = os.path.realpath(
> + os.path.join(cr_workdir,
> "include-path%d.tar" % part.lineno))
> + if os.path.isfile(include_path):
> + parent =
> os.path.dirname(os.path.realpath(include_path))
> + tar_cmd = "tar c --owner=root --group=root -f %s
> -C %s %s" % (
> + tar_file, parent,
> os.path.relpath(include_path, parent))
> + exec_native_cmd(tar_cmd, native_sysroot)
> + else:
> + if include_path in krootfs_dir:
> + include_path = krootfs_dir[include_path]
> + include_path = cls.__get_rootfs_dir(include_path)
> + include_pseudo = os.path.join(include_path,
> "../pseudo")
> + if os.path.lexists(include_pseudo):
> + pseudo = cls.__get_pseudo(native_sysroot,
> include_path,
> + include_pseudo)
> + tar_cmd = "tar cf %s -C %s ." % (tar_file,
> include_path)
> + else:
> + pseudo = None
> + tar_cmd = "tar c --owner=root --group=root
> -f %s -C %s ." % (
> + tar_file, include_path)
> + exec_native_cmd(tar_cmd, native_sysroot, pseudo)
> +
> + #create destination
> + if path:
> + destination =
> cls.__validate_path("--include-path", new_rootfs, path)
> + Path(destination).mkdir(parents=True,
> exist_ok=True)
> + else:
> + destination = new_rootfs
>
> - real_rootfs_dir = new_rootfs
> + #extract destination
> + untar_cmd = "tar xf %s -C %s" % (tar_file,
> destination)
> + if new_pseudo:
> + pseudo = cls.__get_pseudo(native_sysroot,
> new_rootfs, new_pseudo)
> + else:
> + pseudo = None
> + exec_native_cmd(untar_cmd, native_sysroot, pseudo)
> + os.remove(tar_file)
>
> - for orig_path in part.exclude_path:
> + for orig_path in part.exclude_path or []:
> path = orig_path
> - if os.path.isabs(path):
> - msger.error("Must be relative:
> --exclude-path=%s" % orig_path)
> - full_path =
> os.path.realpath(os.path.join(new_rootfs, path))
> + full_path = cls.__validate_path("--exclude-path",
> new_rootfs, path)
> - # Disallow climbing outside of parent directory
> using '..',
> - # because doing so could be quite disastrous (we
> will delete the
> - # directory).
> - if not full_path.startswith(new_rootfs):
> - msger.error("'%s' points to a path outside the
> rootfs" % orig_path)
> + if not os.path.lexists(full_path):
> + continue
>
> if path.endswith(os.sep):
> # Delete content only.
> @@ -120,6 +211,6 @@ class RootfsPlugin(SourcePlugin):
> # Delete whole directory.
> shutil.rmtree(full_path)
>
> - part.rootfs_dir = real_rootfs_dir
> part.prepare_rootfs(cr_workdir, oe_builddir,
> - real_rootfs_dir, native_sysroot)
> + new_rootfs or part.rootfs_dir,
> native_sysroot,
> + pseudo_dir = new_pseudo or pseudo_dir)
> diff --git a/scripts/lib/wic/utils/__init__.py
> b/scripts/lib/wic/utils/__init__.py deleted file mode 100644
> index e69de29..0000000
> diff --git a/scripts/lib/wic/utils/runner.py
> b/scripts/lib/wic/utils/runner.py deleted file mode 100644
> index 56d7ea3..0000000
> --- a/scripts/lib/wic/utils/runner.py
> +++ /dev/null
> @@ -1,114 +0,0 @@
> -#!/usr/bin/env python -tt
> -#
> -# Copyright (c) 2011 Intel, Inc.
> -#
> -# This program is free software; you can redistribute it and/or
> modify it -# under the terms of the GNU General Public License as
> published by the Free -# Software Foundation; version 2 of the License
> -#
> -# This program is distributed in the hope that it will be useful, but
> -# WITHOUT ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY -# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> General Public License -# for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., 59 -# Temple Place - Suite 330, Boston, MA
> 02111-1307, USA. -
> -import logging
> -import os
> -import subprocess
> -
> -from wic import WicError
> -
> -logger = logging.getLogger('wic')
> -
> -def runtool(cmdln_or_args, catch=1):
> - """ wrapper for most of the subprocess calls
> - input:
> - cmdln_or_args: can be both args and cmdln str (shell=True)
> - catch: 0, quitely run
> - 1, only STDOUT
> - 2, only STDERR
> - 3, both STDOUT and STDERR
> - return:
> - (rc, output)
> - if catch==0: the output will always None
> - """
> -
> - if catch not in (0, 1, 2, 3):
> - # invalid catch selection, will cause exception, that's good
> - return None
> -
> - if isinstance(cmdln_or_args, list):
> - cmd = cmdln_or_args[0]
> - shell = False
> - else:
> - import shlex
> - cmd = shlex.split(cmdln_or_args)[0]
> - shell = True
> -
> - if catch != 3:
> - dev_null = os.open("/dev/null", os.O_WRONLY)
> -
> - if catch == 0:
> - sout = dev_null
> - serr = dev_null
> - elif catch == 1:
> - sout = subprocess.PIPE
> - serr = dev_null
> - elif catch == 2:
> - sout = dev_null
> - serr = subprocess.PIPE
> - elif catch == 3:
> - sout = subprocess.PIPE
> - serr = subprocess.STDOUT
> -
> - try:
> - process = subprocess.Popen(cmdln_or_args, stdout=sout,
> - stderr=serr, shell=shell)
> - (sout, serr) = process.communicate()
> - # combine stdout and stderr, filter None out and decode
> - out = ''.join([out.decode('utf-8') for out in [sout, serr]
> if out])
> - except OSError as err:
> - if err.errno == 2:
> - # [Errno 2] No such file or directory
> - raise WicError('Cannot run command: %s, lost
> dependency?' % cmd)
> - else:
> - raise # relay
> - finally:
> - if catch != 3:
> - os.close(dev_null)
> -
> - return (process.returncode, out)
> -
> -def show(cmdln_or_args):
> - """Show all messages using logger.debug."""
> -
> - rcode, out = runtool(cmdln_or_args, catch=3)
> -
> - if isinstance(cmdln_or_args, list):
> - cmd = ' '.join(cmdln_or_args)
> - else:
> - cmd = cmdln_or_args
> -
> - msg = 'running command: "%s"' % cmd
> - if out:
> - out = out.strip()
> - if out:
> - msg += ', with output::'
> - msg += '\n +----------------'
> - for line in out.splitlines():
> - msg += '\n | %s' % line
> - msg += '\n +----------------'
> -
> - logger.debug(msg)
> -
> - return rcode
> -
> -def outs(cmdln_or_args, catch=1):
> - # get the outputs of tools
> - return runtool(cmdln_or_args, catch)[1].strip()
> -
> -def quiet(cmdln_or_args):
> - return runtool(cmdln_or_args, catch=0)[0]
> diff --git a/scripts/wic b/scripts/wic
> index a5f2dbf..24700f3 100755
> --- a/scripts/wic
> +++ b/scripts/wic
> @@ -1,22 +1,8 @@
> #!/usr/bin/env python3
> -# ex:ts=4:sw=4:sts=4:et
> -# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> #
> # Copyright (c) 2013, Intel Corporation.
> -# All rights reserved.
> #
> -# This program is free software; you can redistribute it and/or
> modify -# it under the terms of the GNU General Public License
> version 2 as -# published by the Free Software Foundation.
> -#
> -# This program is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> -# GNU General Public License for more details.
> -#
> -# You should have received a copy of the GNU General Public License
> along -# with this program; if not, write to the Free Software
> Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA. +# SPDX-License-Identifier: GPL-2.0-only
> #
> # DESCRIPTION 'wic' is the OpenEmbedded Image Creator that users can
> # use to generate bootable images. Invoking it without any arguments
> @@ -33,28 +19,41 @@ __version__ = "0.2.0"
> # Python Standard Library modules
> import os
> import sys
> -import optparse
> +import argparse
> import logging
> +import subprocess
> +
> +from collections import namedtuple
> from distutils import spawn
>
> # External modules
> -scripts_path = os.path.abspath(os.path.dirname(__file__))
> +scripts_path = os.path.dirname(os.path.realpath(__file__))
> lib_path = scripts_path + '/lib'
> sys.path.insert(0, lib_path)
> -oe_lib_path = os.path.join(os.path.dirname(scripts_path), 'meta',
> 'lib') -sys.path.insert(0, oe_lib_path)
> +import scriptpath
> +scriptpath.add_oe_lib_path()
> +
> +# Check whether wic is running within eSDK environment
> +sdkroot = scripts_path
> +if os.environ.get('SDKTARGETSYSROOT'):
> + while sdkroot != '' and sdkroot != os.sep:
> + if os.path.exists(os.path.join(sdkroot, '.devtoolbase')):
> + # Set BUILDDIR for wic to work within eSDK
> + os.environ['BUILDDIR'] = sdkroot
> + # .devtoolbase only exists within eSDK
> + # If found, initialize bitbake path for eSDK environment
> and append to PATH
> + sdkroot = os.path.join(os.path.dirname(scripts_path),
> 'bitbake', 'bin')
> + os.environ['PATH'] += ":" + sdkroot
> + break
> + sdkroot = os.path.dirname(sdkroot)
>
> bitbake_exe = spawn.find_executable('bitbake')
> if bitbake_exe:
> - bitbake_path = os.path.join(os.path.dirname(bitbake_exe),
> '../lib')
> - sys.path.insert(0, bitbake_path)
> - from bb import cookerdata
> - from bb.main import bitbake_main, BitBakeConfigParameters
> -else:
> - bitbake_main = None
> + bitbake_path = scriptpath.add_bitbake_lib_path()
> + import bb
>
> from wic import WicError
> -from wic.utils.misc import get_bitbake_var, BB_VARS
> +from wic.misc import get_bitbake_var, BB_VARS
> from wic import engine
> from wic import help as hlp
>
> @@ -85,67 +84,31 @@ def rootfs_dir_to_args(krootfs_dir):
> rootfs_dir += '='.join([key, val])
> return rootfs_dir.strip()
>
> -def callback_rootfs_dir(option, opt, value, parser):
> - """
> - Build a dict using --rootfs_dir connection=dir
> - """
> - if not type(parser.values.rootfs_dir) is dict:
> - parser.values.rootfs_dir = dict()
>
> - if '=' in value:
> - (key, rootfs_dir) = value.split('=')
> - else:
> - key = 'ROOTFS_DIR'
> - rootfs_dir = value
> +class RootfsArgAction(argparse.Action):
> + def __init__(self, **kwargs):
> + super().__init__(**kwargs)
> +
> + def __call__(self, parser, namespace, value, option_string=None):
> + if not "rootfs_dir" in vars(namespace) or \
> + not type(namespace.__dict__['rootfs_dir']) is dict:
> + namespace.__dict__['rootfs_dir'] = {}
>
> - parser.values.rootfs_dir[key] = rootfs_dir
> + if '=' in value:
> + (key, rootfs_dir) = value.split('=')
> + else:
> + key = 'ROOTFS_DIR'
> + rootfs_dir = value
>
> -def wic_create_subcommand(args, usage_str):
> + namespace.__dict__['rootfs_dir'][key] = rootfs_dir
> +
> +
> +def wic_create_subcommand(options, usage_str):
> """
> Command-line handling for image creation. The real work is done
> by image.engine.wic_create()
> """
> - parser = optparse.OptionParser(usage=usage_str)
> -
> - parser.add_option("-o", "--outdir", dest="outdir", default='.',
> - help="name of directory to create image in")
> - parser.add_option("-e", "--image-name", dest="image_name",
> - help="name of the image to use the artifacts
> from "
> - "e.g. core-image-sato")
> - parser.add_option("-r", "--rootfs-dir", dest="rootfs_dir",
> type="string",
> - action="callback",
> callback=callback_rootfs_dir,
> - help="path to the /rootfs dir to use as the "
> - ".wks rootfs source")
> - parser.add_option("-b", "--bootimg-dir", dest="bootimg_dir",
> - help="path to the dir containing the boot
> artifacts "
> - "(e.g. /EFI or /syslinux dirs) to use as
> the "
> - ".wks bootimg source")
> - parser.add_option("-k", "--kernel-dir", dest="kernel_dir",
> - help="path to the dir containing the kernel to
> use "
> - "in the .wks bootimg")
> - parser.add_option("-n", "--native-sysroot",
> dest="native_sysroot",
> - help="path to the native sysroot containing
> the tools "
> - "to use to build the image")
> - parser.add_option("-s", "--skip-build-check", dest="build_check",
> - action="store_false", default=True, help="skip
> the build check")
> - parser.add_option("-f", "--build-rootfs", action="store_true",
> help="build rootfs")
> - parser.add_option("-c", "--compress-with", choices=("gzip",
> "bzip2", "xz"),
> - dest='compressor',
> - help="compress image with specified
> compressor")
> - parser.add_option("-m", "--bmap", action="store_true",
> help="generate .bmap")
> - parser.add_option("-v", "--vars", dest='vars_dir',
> - help="directory with <image>.env files that
> store "
> - "bitbake variables")
> - parser.add_option("-D", "--debug", dest="debug",
> action="store_true",
> - default=False, help="output debug information")
> -
> - (options, args) = parser.parse_args(args)
> -
> - if len(args) != 1:
> - parser.print_help()
> - raise WicError("Wrong number of arguments, exiting")
> -
> - if options.build_rootfs and not bitbake_main:
> + if options.build_rootfs and not bitbake_exe:
> raise WicError("Can't build rootfs as bitbake is not in the
> $PATH")
> if not options.image_name:
> @@ -181,39 +144,37 @@ def wic_create_subcommand(args, usage_str):
> argv.append("--debug")
>
> logger.info("Building rootfs...\n")
> - if bitbake_main(BitBakeConfigParameters(argv),
> - cookerdata.CookerConfiguration()):
> - raise WicError("bitbake exited with error")
> + subprocess.check_call(argv)
>
> rootfs_dir = get_bitbake_var("IMAGE_ROOTFS",
> options.image_name) kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE",
> options.image_name) bootimg_dir = get_bitbake_var("STAGING_DATADIR",
> options.image_name)
> - native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE",
> - options.image_name) #,
> cache=False) +
> + native_sysroot = options.native_sysroot
> + if options.vars_dir and not native_sysroot:
> + native_sysroot =
> get_bitbake_var("RECIPE_SYSROOT_NATIVE", options.image_name) else:
> if options.build_rootfs:
> raise WicError("Image name is not specified, exiting. "
> "(Use -e/--image-name to specify it)")
> native_sysroot = options.native_sysroot
>
> - if not native_sysroot or not os.path.isdir(native_sysroot):
> + if not options.vars_dir and (not native_sysroot or not
> os.path.isdir(native_sysroot)): logger.info("Building wic-tools...\n")
> - if bitbake_main(BitBakeConfigParameters("bitbake
> wic-tools".split()),
> - cookerdata.CookerConfiguration()):
> - raise WicError("bitbake wic-tools failed")
> + subprocess.check_call(["bitbake", "wic-tools"])
> native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE",
> "wic-tools")
> - if not native_sysroot:
> - raise WicError("Unable to find the location of the
> native "
> - "tools sysroot to use")
>
> - wks_file = args[0]
> + if not native_sysroot:
> + raise WicError("Unable to find the location of the native
> tools sysroot") +
> + wks_file = options.wks_file
>
> if not wks_file.endswith(".wks"):
> wks_file = engine.find_canned_image(scripts_path, wks_file)
> if not wks_file:
> raise WicError("No image named %s found, exiting. (Use
> 'wic list images' " "to list available images, or specify a
> fully-qualified OE "
> - "kickstart (.wks) filename)" % args[0])
> + "kickstart (.wks) filename)" %
> options.wks_file)
> if not options.image_name:
> rootfs_dir = ''
> @@ -264,59 +225,312 @@ def wic_list_subcommand(args, usage_str):
> Command-line handling for listing available images.
> The real work is done by image.engine.wic_list()
> """
> - parser = optparse.OptionParser(usage=usage_str)
> - args = parser.parse_args(args)[1]
> -
> if not engine.wic_list(args, scripts_path):
> - parser.print_help()
> raise WicError("Bad list arguments, exiting")
>
>
> -def wic_help_topic_subcommand(args, usage_str):
> +def wic_ls_subcommand(args, usage_str):
> + """
> + Command-line handling for list content of images.
> + The real work is done by engine.wic_ls()
> + """
> + engine.wic_ls(args, args.native_sysroot)
> +
> +def wic_cp_subcommand(args, usage_str):
> + """
> + Command-line handling for copying files/dirs to images.
> + The real work is done by engine.wic_cp()
> + """
> + engine.wic_cp(args, args.native_sysroot)
> +
> +def wic_rm_subcommand(args, usage_str):
> """
> - Command-line handling for help-only 'subcommands'. This is
> - essentially a dummy command that doesn nothing but allow users to
> - use the existing subcommand infrastructure to display help on a
> - particular topic not attached to any particular subcommand.
> + Command-line handling for removing files/dirs from images.
> + The real work is done by engine.wic_rm()
> + """
> + engine.wic_rm(args, args.native_sysroot)
> +
> +def wic_write_subcommand(args, usage_str):
> + """
> + Command-line handling for writing images.
> + The real work is done by engine.wic_write()
> + """
> + engine.wic_write(args, args.native_sysroot)
> +
> +def wic_help_subcommand(args, usage_str):
> + """
> + Command-line handling for help subcommand to keep the current
> + structure of the function definitions.
> """
> pass
>
>
> +def wic_help_topic_subcommand(usage_str, help_str):
> + """
> + Display function for help 'sub-subcommands'.
> + """
> + print(help_str)
> + return
> +
> +
> wic_help_topic_usage = """
> """
>
> -subcommands = {
> - "create": [wic_create_subcommand,
> - hlp.wic_create_usage,
> - hlp.wic_create_help],
> - "list": [wic_list_subcommand,
> - hlp.wic_list_usage,
> - hlp.wic_list_help],
> +helptopics = {
> "plugins": [wic_help_topic_subcommand,
> wic_help_topic_usage,
> - hlp.get_wic_plugins_help],
> + hlp.wic_plugins_help],
> "overview": [wic_help_topic_subcommand,
> wic_help_topic_usage,
> hlp.wic_overview_help],
> "kickstart": [wic_help_topic_subcommand,
> wic_help_topic_usage,
> hlp.wic_kickstart_help],
> + "create": [wic_help_topic_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_create_help],
> + "ls": [wic_help_topic_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_ls_help],
> + "cp": [wic_help_topic_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_cp_help],
> + "rm": [wic_help_topic_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_rm_help],
> + "write": [wic_help_topic_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_write_help],
> + "list": [wic_help_topic_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_list_help]
> }
>
>
> +def wic_init_parser_create(subparser):
> + subparser.add_argument("wks_file")
> +
> + subparser.add_argument("-o", "--outdir", dest="outdir",
> default='.',
> + help="name of directory to create image in")
> + subparser.add_argument("-e", "--image-name", dest="image_name",
> + help="name of the image to use the artifacts
> from "
> + "e.g. core-image-sato")
> + subparser.add_argument("-r", "--rootfs-dir",
> action=RootfsArgAction,
> + help="path to the /rootfs dir to use as the "
> + ".wks rootfs source")
> + subparser.add_argument("-b", "--bootimg-dir", dest="bootimg_dir",
> + help="path to the dir containing the boot
> artifacts "
> + "(e.g. /EFI or /syslinux dirs) to use as
> the "
> + ".wks bootimg source")
> + subparser.add_argument("-k", "--kernel-dir", dest="kernel_dir",
> + help="path to the dir containing the kernel to
> use "
> + "in the .wks bootimg")
> + subparser.add_argument("-n", "--native-sysroot",
> dest="native_sysroot",
> + help="path to the native sysroot containing
> the tools "
> + "to use to build the image")
> + subparser.add_argument("-s", "--skip-build-check",
> dest="build_check",
> + action="store_false", default=True, help="skip
> the build check")
> + subparser.add_argument("-f", "--build-rootfs",
> action="store_true", help="build rootfs")
> + subparser.add_argument("-c", "--compress-with", choices=("gzip",
> "bzip2", "xz"),
> + dest='compressor',
> + help="compress image with specified
> compressor")
> + subparser.add_argument("-m", "--bmap", action="store_true",
> help="generate .bmap")
> + subparser.add_argument("--no-fstab-update" ,action="store_true",
> + help="Do not change fstab file.")
> + subparser.add_argument("-v", "--vars", dest='vars_dir',
> + help="directory with <image>.env files that
> store "
> + "bitbake variables")
> + subparser.add_argument("-D", "--debug", dest="debug",
> action="store_true",
> + default=False, help="output debug information")
> + subparser.add_argument("-i", "--imager", dest="imager",
> + default="direct", help="the wic imager plugin")
> + return
> +
> +
> +def wic_init_parser_list(subparser):
> + subparser.add_argument("list_type",
> + help="can be 'images' or 'source-plugins' "
> + "to obtain a list. "
> + "If value is a valid .wks image file")
> + subparser.add_argument("help_for", default=[], nargs='*',
> + help="If 'list_type' is a valid .wks image
> file "
> + "this value can be 'help' to show the
> help information "
> + "defined inside the .wks file")
> + return
> +
> +def imgtype(arg):
> + """
> + Custom type for ArgumentParser
> + Converts path spec to named tuple: (image, partition, path)
> + """
> + image = arg
> + part = path = None
> + if ':' in image:
> + image, part = image.split(':')
> + if '/' in part:
> + part, path = part.split('/', 1)
> + if not path:
> + path = '/'
> +
> + if not os.path.isfile(image):
> + err = "%s is not a regular file or symlink" % image
> + raise argparse.ArgumentTypeError(err)
> +
> + return namedtuple('ImgType', 'image part path')(image, part,
> path) +
> +def wic_init_parser_ls(subparser):
> + subparser.add_argument("path", type=imgtype,
> + help="image spec: <image>[:<vfat
> partition>[<path>]]")
> + subparser.add_argument("-n", "--native-sysroot",
> + help="path to the native sysroot containing
> the tools") +
> +def imgpathtype(arg):
> + img = imgtype(arg)
> + if img.part is None:
> + raise argparse.ArgumentTypeError("partition number is not
> specified")
> + return img
> +
> +def wic_init_parser_cp(subparser):
> + subparser.add_argument("src",
> + help="image spec: <image>:<vfat
> partition>[<path>] or <file>")
> + subparser.add_argument("dest",
> + help="image spec: <image>:<vfat
> partition>[<path>] or <file>")
> + subparser.add_argument("-n", "--native-sysroot",
> + help="path to the native sysroot containing
> the tools") +
> +def wic_init_parser_rm(subparser):
> + subparser.add_argument("path", type=imgpathtype,
> + help="path: <image>:<vfat partition><path>")
> + subparser.add_argument("-n", "--native-sysroot",
> + help="path to the native sysroot containing
> the tools")
> + subparser.add_argument("-r", dest="recursive_delete",
> action="store_true", default=False,
> + help="remove directories and their contents
> recursively, "
> + " this only applies to ext* partition")
> +
> +def expandtype(rules):
> + """
> + Custom type for ArgumentParser
> + Converts expand rules to the dictionary {<partition>: size}
> + """
> + if rules == 'auto':
> + return {}
> + result = {}
> + for rule in rules.split(','):
> + try:
> + part, size = rule.split(':')
> + except ValueError:
> + raise argparse.ArgumentTypeError("Incorrect rule format:
> %s" % rule) +
> + if not part.isdigit():
> + raise argparse.ArgumentTypeError("Rule '%s': partition
> number must be integer" % rule) +
> + # validate size
> + multiplier = 1
> + for suffix, mult in [('K', 1024), ('M', 1024 * 1024), ('G',
> 1024 * 1024 * 1024)]:
> + if size.upper().endswith(suffix):
> + multiplier = mult
> + size = size[:-1]
> + break
> + if not size.isdigit():
> + raise argparse.ArgumentTypeError("Rule '%s': size must
> be integer" % rule) +
> + result[int(part)] = int(size) * multiplier
> +
> + return result
> +
> +def wic_init_parser_write(subparser):
> + subparser.add_argument("image",
> + help="path to the wic image")
> + subparser.add_argument("target",
> + help="target file or device")
> + subparser.add_argument("-e", "--expand", type=expandtype,
> + help="expand rules: auto or
> <partition>:<size>[,<partition>:<size>]")
> + subparser.add_argument("-n", "--native-sysroot",
> + help="path to the native sysroot containing
> the tools") +
> +def wic_init_parser_help(subparser):
> + helpparsers = subparser.add_subparsers(dest='help_topic',
> help=hlp.wic_usage)
> + for helptopic in helptopics:
> + helpparsers.add_parser(helptopic,
> help=helptopics[helptopic][2])
> + return
> +
> +
> +subcommands = {
> + "create": [wic_create_subcommand,
> + hlp.wic_create_usage,
> + hlp.wic_create_help,
> + wic_init_parser_create],
> + "list": [wic_list_subcommand,
> + hlp.wic_list_usage,
> + hlp.wic_list_help,
> + wic_init_parser_list],
> + "ls": [wic_ls_subcommand,
> + hlp.wic_ls_usage,
> + hlp.wic_ls_help,
> + wic_init_parser_ls],
> + "cp": [wic_cp_subcommand,
> + hlp.wic_cp_usage,
> + hlp.wic_cp_help,
> + wic_init_parser_cp],
> + "rm": [wic_rm_subcommand,
> + hlp.wic_rm_usage,
> + hlp.wic_rm_help,
> + wic_init_parser_rm],
> + "write": [wic_write_subcommand,
> + hlp.wic_write_usage,
> + hlp.wic_write_help,
> + wic_init_parser_write],
> + "help": [wic_help_subcommand,
> + wic_help_topic_usage,
> + hlp.wic_help_help,
> + wic_init_parser_help]
> +}
> +
> +
> +def init_parser(parser):
> + parser.add_argument("--version", action="version",
> + version="%(prog)s {version}".format(version=__version__))
> + parser.add_argument("-D", "--debug", dest="debug",
> action="store_true",
> + default=False, help="output debug information")
> +
> + subparsers = parser.add_subparsers(dest='command',
> help=hlp.wic_usage)
> + for subcmd in subcommands:
> + subparser = subparsers.add_parser(subcmd,
> help=subcommands[subcmd][2])
> + subcommands[subcmd][3](subparser)
> +
> +class WicArgumentParser(argparse.ArgumentParser):
> + def format_help(self):
> + return hlp.wic_help
> +
> def main(argv):
> - parser = optparse.OptionParser(version="wic version %s" %
> __version__,
> - usage=hlp.wic_usage)
> + parser = WicArgumentParser(
> + description="wic version %s" % __version__)
> +
> + init_parser(parser)
>
> - parser.disable_interspersed_args()
> + args = parser.parse_args(argv)
>
> - args = parser.parse_args(argv)[1]
> + if args.debug:
> + logger.setLevel(logging.DEBUG)
>
> - if len(args):
> - if args[0] == "help":
> - if len(args) == 1:
> + if "command" in vars(args):
> + if args.command == "help":
> + if args.help_topic is None:
> parser.print_help()
> - raise WicError("help command requires parameter")
> + elif args.help_topic in helptopics:
> + hlpt = helptopics[args.help_topic]
> + hlpt[0](hlpt[1], hlpt[2])
> + return 0
> +
> + # validate wic cp src and dest parameter to identify which one
> of it is
> + # image and cast it into imgtype
> + if args.command == "cp":
> + if ":" in args.dest:
> + args.dest = imgtype(args.dest)
> + elif ":" in args.src:
> + args.src = imgtype(args.src)
> + else:
> + raise argparse.ArgumentTypeError("no image or partition
> number specified.")
> return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage,
> subcommands)
next prev parent reply other threads:[~2020-09-05 9:00 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-02 18:56 [PATCH v2 00/10] WIC update Vijai Kumar K
2020-09-02 18:56 ` [PATCH v2 01/10] wic: Update to the latest wic from openembedded core Vijai Kumar K
2020-09-05 9:00 ` Henning Schild [this message]
2020-09-05 9:21 ` vijai kumar
2020-09-02 18:56 ` [PATCH v2 02/10] wic/plugins: Fix wic plugins to work with the latest wic Vijai Kumar K
2020-09-02 18:56 ` [PATCH v2 03/10] wic-img: Satisfy the quirks of " Vijai Kumar K
2020-09-05 8:30 ` Henning Schild
2020-09-05 16:24 ` vijaikumar....@gmail.com
2020-09-09 14:52 ` Henning Schild
2020-09-02 18:56 ` [PATCH v2 04/10] oe.path: Add copyhardlink() helper function Vijai Kumar K
2020-09-02 18:56 ` [PATCH v2 05/10] lib/oe/path: try hardlinking instead of guessing when it might fail Vijai Kumar K
2020-09-02 18:56 ` [PATCH v2 06/10] wic_fakeroot: Handle standalone pseudo invocations Vijai Kumar K
2020-09-05 8:38 ` Henning Schild
2020-09-05 15:19 ` vijaikumar....@gmail.com
2020-09-05 9:19 ` Henning Schild
2020-09-05 15:22 ` vijaikumar....@gmail.com
2020-09-02 18:56 ` [PATCH v2 07/10] meta-isar/conf: Add provision to debug WIC Vijai Kumar K
2020-09-02 18:56 ` [PATCH v2 08/10] debian-common: Add tar as a dependency for wic Vijai Kumar K
2020-09-02 19:02 ` [PATCH v2 09/10] wic: misc: Add /bin to the list of searchpaths Vijai Kumar K
2020-09-02 19:02 ` [PATCH v2 10/10] meta-isar/canned-wks: Remove /boot mountpoint Vijai Kumar K
2020-09-05 8:58 ` Henning Schild
2020-09-05 16:06 ` vijaikumar....@gmail.com
2020-09-09 15:09 ` Henning Schild
2020-09-14 5:32 ` vijaikumar....@gmail.com
2020-09-05 8:45 ` [PATCH v2 09/10] wic: misc: Add /bin to the list of searchpaths Henning Schild
2020-09-05 9:33 ` vijai kumar
2020-09-03 5:46 ` [PATCH v2 00/10] WIC update vijaikumar....@gmail.com
2020-09-05 9:04 ` Henning Schild
2020-09-05 9:25 ` vijai kumar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200905110055.6d9ce1f1@md1za8fc.ad001.siemens.net \
--to=henning.schild@siemens.com \
--cc=Vijaikumar_Kanagarajan@mentor.com \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox