* [PATCH v2 01/17] Revert "wic: Make the bootimg-efi plugin generate usable images"
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 02/17] Revert "wic: Introduce the `WicExecError` exception class" Henning Schild
` (17 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This reverts commit c8f10e3f541643f6ed6bb3c971bf36267519fba8.
We will fork that plugin and leave the original as it is.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/lib/wic/plugins/source/bootimg-efi.py | 62 +++------------------------
1 file changed, 6 insertions(+), 56 deletions(-)
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 45635f3..f631e3f 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -69,20 +69,12 @@ class BootimgEFIPlugin(SourcePlugin):
grubefi_conf += "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n"
grubefi_conf += "default=boot\n"
grubefi_conf += "timeout=%s\n" % bootloader.timeout
- grubefi_conf += "set root='hd0,gpt2'\n"
grubefi_conf += "menuentry 'boot'{\n"
- kernel_image = get_bitbake_var("KERNEL_IMAGE")
- kernel = "/boot/%s" % kernel_image
-
- grubefi_conf += "linux %s root=/dev/sda2 rootwait %s\n" \
- % (kernel, bootloader.append)
-
- initrd_image = get_bitbake_var("INITRD_IMAGE")
- initrd = "/boot/%s" % initrd_image
-
- grubefi_conf += "initrd %s\n" % initrd
+ kernel = "/bzImage"
+ grubefi_conf += "linux %s root=%s rootwait %s\n" \
+ % (kernel, creator.rootdev, bootloader.append)
grubefi_conf += "}\n"
logger.debug("Writing grubefi config %s/hdd/boot/EFI/BOOT/grub.cfg",
@@ -91,12 +83,6 @@ class BootimgEFIPlugin(SourcePlugin):
cfg.write(grubefi_conf)
cfg.close()
- cfg = open("%s/hdd/boot/EFI/BOOT/grub-mkimage.cfg" % cr_workdir, "w")
- mkimage_conf = "set root='hd0,gpt1'\n"
- mkimage_conf += "set prefix=($root)/EFI/BOOT\n"
- cfg.write(mkimage_conf)
- cfg.close()
-
@classmethod
def do_configure_systemdboot(cls, hdddir, creator, cr_workdir, source_params):
"""
@@ -148,8 +134,7 @@ class BootimgEFIPlugin(SourcePlugin):
if not custom_cfg:
# Create systemd-boot configuration using parameters from wks file
- kernel_name = get_bitbake_var("KERNEL_IMAGE")
- kernel = "/%s" % kernel_name
+ kernel = "/bzImage"
boot_conf = ""
boot_conf += "title boot\n"
@@ -206,9 +191,8 @@ class BootimgEFIPlugin(SourcePlugin):
hdddir = "%s/hdd/boot" % cr_workdir
- kernel_name = get_bitbake_var("KERNEL_IMAGE")
- install_cmd = "install -m 0644 %s/%s %s/%s" % \
- (staging_kernel_dir, kernel_name, hdddir, kernel_name)
+ install_cmd = "install -m 0644 %s/bzImage %s/bzImage" % \
+ (staging_kernel_dir, hdddir)
exec_cmd(install_cmd)
@@ -216,45 +200,11 @@ class BootimgEFIPlugin(SourcePlugin):
if source_params['loader'] == 'grub-efi':
shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir,
"%s/grub.cfg" % cr_workdir)
- shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub-mkimage.cfg" % cr_workdir,
- "%s/grub-mkimage.cfg" % cr_workdir)
for mod in [x for x in os.listdir(kernel_dir) if x.startswith("grub-efi-")]:
cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[9:])
exec_cmd(cp_cmd, True)
shutil.move("%s/grub.cfg" % cr_workdir,
"%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir)
-
- distro_arch = get_bitbake_var("DISTRO_ARCH")
- if not distro_arch:
- raise WicError("Couldn't find target architecture")
-
- if distro_arch == "amd64":
- grub_target = 'x86_64-efi'
- grub_image = "bootx64.efi"
- elif distro_arch == "i386":
- grub_target = 'i386-efi'
- grub_image = "bootia32.efi"
- else:
- raise WicError("grub-efi is incompatible with target %s" %
- distro_arch)
-
- bootimg_dir = "%s/hdd/boot" % cr_workdir
- if not os.path.isfile("%s/EFI/BOOT/%s" \
- % (bootimg_dir, grub_image)):
-
- # TODO: check that grub-mkimage is available
- grub_cmd = "grub-mkimage -p /EFI/BOOT "
- grub_cmd += "-c %s/grub-mkimage.cfg " % cr_workdir
- grub_cmd += "-O %s -o %s/EFI/BOOT/%s " \
- % (grub_target, bootimg_dir, grub_image)
- grub_cmd += "part_gpt part_msdos ntfs ntfscomp fat ext2 "
- grub_cmd += "normal chain boot configfile linux multiboot "
- grub_cmd += "search efi_gop efi_uga font gfxterm gfxmenu "
- grub_cmd += "terminal minicmd test iorw loadenv echo help "
- grub_cmd += "reboot serial terminfo iso9660 loopback tar "
- grub_cmd += "memdisk ls search_fs_uuid udf btrfs xfs lvm "
- grub_cmd += "reiserfs ata "
- exec_cmd(grub_cmd)
elif source_params['loader'] == 'systemd-boot':
for mod in [x for x in os.listdir(kernel_dir) if x.startswith("systemd-")]:
cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[8:])
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 02/17] Revert "wic: Introduce the `WicExecError` exception class"
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
2018-04-13 14:18 ` [PATCH v2 01/17] Revert "wic: Make the bootimg-efi plugin generate usable images" Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 03/17] Revert "wic: Work around mcopy error" Henning Schild
` (16 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This reverts commit 17f9196e0f2cb7e1952369fb79f156fc8a3c678d.
This was a workaround for a bug in fsck of debian. A later patch will
introduce this workaround outside of wic.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/lib/wic/partition.py | 10 +++-------
scripts/lib/wic/utils/misc.py | 17 +++--------------
2 files changed, 6 insertions(+), 21 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index c137a17..26be958 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -29,7 +29,7 @@ import os
import tempfile
from wic import WicError
-from wic.utils.misc import WicExecError, exec_cmd, get_bitbake_var
+from wic.utils.misc import exec_cmd, get_bitbake_var
from wic.pluginbase import PluginMgr
logger = logging.getLogger('wic')
@@ -253,12 +253,8 @@ class Partition():
(self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
exec_cmd(mkfs_cmd)
- try:
- mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
- exec_cmd(mkfs_cmd)
- except WicExecError as e:
- if e.returncode != 1:
- raise e
+ mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
+ exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir):
"""
diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/utils/misc.py
index b3a619d..4609984 100644
--- a/scripts/lib/wic/utils/misc.py
+++ b/scripts/lib/wic/utils/misc.py
@@ -59,16 +59,6 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
"syslinux": "syslinux"
}
-class WicExecError(WicError):
- def __init__(self, command, output, returncode):
- self.command = command
- self.output = output
- self.returncode = returncode
-
- def __str__(self):
- return "_exec_cmd: %s returned '%s' instead of 0\noutput: %s" % \
- (self.command, self.returncode, self.output)
-
def _exec_cmd(cmd_and_args, as_shell=False):
"""
Execute command, catching stderr, stdout
@@ -79,18 +69,17 @@ def _exec_cmd(cmd_and_args, as_shell=False):
args = cmd_and_args.split()
logger.debug(args)
- cmd = args
if as_shell:
- cmd = cmd_and_args
ret, out = runner.runtool(cmd_and_args)
else:
ret, out = runner.runtool(args)
out = out.strip()
if ret != 0:
- raise WicExecError(cmd, out, ret)
+ raise WicError("_exec_cmd: %s returned '%s' instead of 0\noutput: %s" % \
+ (cmd_and_args, ret, out))
logger.debug("_exec_cmd: output for %s (rc = %d): %s",
- cmd_and_args, out, ret)
+ cmd_and_args, ret, out)
return ret, out
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 03/17] Revert "wic: Work around mcopy error"
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
2018-04-13 14:18 ` [PATCH v2 01/17] Revert "wic: Make the bootimg-efi plugin generate usable images" Henning Schild
2018-04-13 14:18 ` [PATCH v2 02/17] Revert "wic: Introduce the `WicExecError` exception class" Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo" Henning Schild
` (15 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This reverts commit b1511ec6f7bbbb3b872274cdb7320c7d6b868e2c.
Again a workaround for a tool-problem that debian has. Again it can be
fixed outside wic. The environment variable MTOOLS_SKIP_CHECK=1 will be
reintroduced by a later patch for all of wic.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/lib/wic/partition.py | 5 ++---
scripts/lib/wic/plugins/source/bootimg-efi.py | 5 ++---
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 5 ++---
scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 5 ++---
4 files changed, 8 insertions(+), 12 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 26be958..34130d8 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -301,9 +301,8 @@ class Partition():
rootfs, rootfs_size)
exec_cmd(dosfs_cmd)
- mcopy_cmd = "env MTOOLS_SKIP_CHECK=1 mcopy -i %s -s %s/* ::/" % \
- (rootfs, rootfs_dir)
- exec_cmd(mcopy_cmd, True)
+ mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
+ exec_native_cmd(mcopy_cmd, native_sysroot)
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index f631e3f..823a536 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -240,9 +240,8 @@ class BootimgEFIPlugin(SourcePlugin):
dosfs_cmd = "mkdosfs -n efi -C %s %d" % (bootimg, blocks)
exec_cmd(dosfs_cmd)
- mcopy_cmd = "env MTOOLS_SKIP_CHECK=1 mcopy -i %s -s %s/* ::/" % \
- (bootimg, hdddir)
- exec_cmd(mcopy_cmd, True)
+ mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
+ exec_cmd(mcopy_cmd)
chmod_cmd = "chmod 644 %s" % bootimg
exec_cmd(chmod_cmd)
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 61c3837..b029c2f 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -188,9 +188,8 @@ class BootimgPcbiosPlugin(SourcePlugin):
dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, blocks)
exec_cmd(dosfs_cmd)
- mcopy_cmd = "env MTOOLS_SKIP_CHECK=1 mcopy -i %s -s %s/* ::/" % \
- (bootimg, hdddir)
- exec_cmd(mcopy_cmd, True)
+ mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
+ exec_cmd(mcopy_cmd)
syslinux_cmd = "syslinux %s" % bootimg
exec_cmd(syslinux_cmd)
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index 68d2402..dcb6434 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -402,10 +402,9 @@ class IsoImagePlugin(SourcePlugin):
mmd_cmd = "mmd -i %s ::/EFI" % bootimg
exec_cmd(mmd_cmd)
- mcopy_cmd = "env MTOOLS_SKIP_CHECK=1 " \
- "mcopy -i %s -s %s/EFI/* ::/EFI/" \
+ mcopy_cmd = "mcopy -i %s -s %s/EFI/* ::/EFI/" \
% (bootimg, isodir)
- exec_cmd(mcopy_cmd, True)
+ exec_cmd(mcopy_cmd)
chmod_cmd = "chmod 644 %s" % bootimg
exec_cmd(chmod_cmd)
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo"
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (2 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 03/17] Revert "wic: Work around mcopy error" Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-16 10:44 ` Andreas Reichel
2018-04-13 14:18 ` [PATCH v2 05/17] Revert "wic: Remove sysroot support" Henning Schild
` (14 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This reverts commit bf873d3b089474d9061df4c1f0985abfac038ff6.
Not required if you run wic as root or set FAKEROOTCMD.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/lib/wic/partition.py | 27 +++++++++++++++++++--------
1 file changed, 19 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 34130d8..20bc4a2 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -201,6 +201,17 @@ class Partition():
Currently handles ext2/3/4, btrfs and vfat.
"""
+ p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" % native_sysroot)
+ p_localstatedir = os.environ.get("PSEUDO_LOCALSTATEDIR",
+ "%s/../pseudo" % rootfs_dir)
+ p_passwd = os.environ.get("PSEUDO_PASSWD", rootfs_dir)
+ p_nosymlinkexp = os.environ.get("PSEUDO_NOSYMLINKEXP", "1")
+ pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix
+ pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % p_localstatedir
+ pseudo += "export PSEUDO_PASSWD=%s;" % p_passwd
+ pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
+ pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
+
rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
self.lineno, self.fstype)
if os.path.isfile(rootfs):
@@ -226,7 +237,7 @@ class Partition():
self.source_file = rootfs
# get the rootfs size in the right units for kickstart (kB)
- du_cmd = "sudo du -Lbks %s" % rootfs
+ du_cmd = "du -Lbks %s" % rootfs
out = exec_cmd(du_cmd)
self.size = int(out.split()[0])
@@ -234,7 +245,7 @@ class Partition():
"""
Prepare content for an ext2/3/4 rootfs partition.
"""
- du_cmd = "sudo du -ks %s" % rootfs_dir
+ du_cmd = "du -ks %s" % rootfs_dir
out = exec_cmd(du_cmd)
actual_rootfs_size = int(out.split()[0])
@@ -249,7 +260,7 @@ class Partition():
if self.label:
label_str = "-L %s" % self.label
- mkfs_cmd = "sudo mkfs.%s -F %s %s %s -d %s" % \
+ mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
(self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
exec_cmd(mkfs_cmd)
@@ -262,7 +273,7 @@ class Partition():
Currently handles ext2/3/4 and btrfs.
"""
- du_cmd = "sudo du -ks %s" % rootfs_dir
+ du_cmd = "du -ks %s" % rootfs_dir
out = exec_cmd(du_cmd)
actual_rootfs_size = int(out.split()[0])
@@ -275,7 +286,7 @@ class Partition():
if self.label:
label_str = "-L %s" % self.label
- mkfs_cmd = "sudo mkfs.%s -b %d -r %s %s %s" % \
+ mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
(self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
exec_cmd(mkfs_cmd)
@@ -283,7 +294,7 @@ class Partition():
"""
Prepare content for a msdos/vfat rootfs partition.
"""
- du_cmd = "sudo du -bks %s" % rootfs_dir
+ du_cmd = "du -bks %s" % rootfs_dir
out = exec_cmd(du_cmd)
blocks = int(out.split()[0])
@@ -331,7 +342,7 @@ class Partition():
if self.label:
label_str = "-L %s" % self.label
- mkfs_cmd = "sudo mkfs.%s -F %s %s %s" % \
+ mkfs_cmd = "mkfs.%s -F %s %s %s" % \
(self.fstype, extra_imagecmd, label_str, rootfs)
exec_cmd(mkfs_cmd)
@@ -347,7 +358,7 @@ class Partition():
if self.label:
label_str = "-L %s" % self.label
- mkfs_cmd = "sudo mkfs.%s -b %d %s %s" % \
+ mkfs_cmd = "mkfs.%s -b %d %s %s" % \
(self.fstype, self.size * 1024, label_str, rootfs)
exec_cmd(mkfs_cmd)
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo"
2018-04-13 14:18 ` [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo" Henning Schild
@ 2018-04-16 10:44 ` Andreas Reichel
2018-04-16 10:53 ` Andreas Reichel
2018-04-16 10:57 ` Henning Schild
0 siblings, 2 replies; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:44 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Fri, Apr 13, 2018 at 04:18:53PM +0200, Henning Schild wrote:
> This reverts commit bf873d3b089474d9061df4c1f0985abfac038ff6.
>
> Not required if you run wic as root or set FAKEROOTCMD.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> scripts/lib/wic/partition.py | 27 +++++++++++++++++++--------
> 1 file changed, 19 insertions(+), 8 deletions(-)
>
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index 34130d8..20bc4a2 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -201,6 +201,17 @@ class Partition():
>
> Currently handles ext2/3/4, btrfs and vfat.
> """
> + p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" % native_sysroot)
> + p_localstatedir = os.environ.get("PSEUDO_LOCALSTATEDIR",
> + "%s/../pseudo" % rootfs_dir)
> + p_passwd = os.environ.get("PSEUDO_PASSWD", rootfs_dir)
> + p_nosymlinkexp = os.environ.get("PSEUDO_NOSYMLINKEXP", "1")
> + pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix
> + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % p_localstatedir
> + pseudo += "export PSEUDO_PASSWD=%s;" % p_passwd
> + pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
> + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
> +
In patch 11 you write that Isar does/ cannot use Pseudo. So why do you
reinstantiate the variables here. I don't understand their purpose because
they are prefixed with PSEUDO.
> rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> self.lineno, self.fstype)
> if os.path.isfile(rootfs):
> @@ -226,7 +237,7 @@ class Partition():
> self.source_file = rootfs
>
> # get the rootfs size in the right units for kickstart (kB)
> - du_cmd = "sudo du -Lbks %s" % rootfs
> + du_cmd = "du -Lbks %s" % rootfs
> out = exec_cmd(du_cmd)
> self.size = int(out.split()[0])
>
> @@ -234,7 +245,7 @@ class Partition():
> """
> Prepare content for an ext2/3/4 rootfs partition.
> """
> - du_cmd = "sudo du -ks %s" % rootfs_dir
> + du_cmd = "du -ks %s" % rootfs_dir
> out = exec_cmd(du_cmd)
> actual_rootfs_size = int(out.split()[0])
>
> @@ -249,7 +260,7 @@ class Partition():
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "sudo mkfs.%s -F %s %s %s -d %s" % \
> + mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
> (self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
> exec_cmd(mkfs_cmd)
>
> @@ -262,7 +273,7 @@ class Partition():
>
> Currently handles ext2/3/4 and btrfs.
> """
> - du_cmd = "sudo du -ks %s" % rootfs_dir
> + du_cmd = "du -ks %s" % rootfs_dir
> out = exec_cmd(du_cmd)
> actual_rootfs_size = int(out.split()[0])
>
> @@ -275,7 +286,7 @@ class Partition():
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "sudo mkfs.%s -b %d -r %s %s %s" % \
> + mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
> (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
> exec_cmd(mkfs_cmd)
>
> @@ -283,7 +294,7 @@ class Partition():
> """
> Prepare content for a msdos/vfat rootfs partition.
> """
> - du_cmd = "sudo du -bks %s" % rootfs_dir
> + du_cmd = "du -bks %s" % rootfs_dir
> out = exec_cmd(du_cmd)
> blocks = int(out.split()[0])
>
> @@ -331,7 +342,7 @@ class Partition():
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "sudo mkfs.%s -F %s %s %s" % \
> + mkfs_cmd = "mkfs.%s -F %s %s %s" % \
> (self.fstype, extra_imagecmd, label_str, rootfs)
> exec_cmd(mkfs_cmd)
>
> @@ -347,7 +358,7 @@ class Partition():
> if self.label:
> label_str = "-L %s" % self.label
>
> - mkfs_cmd = "sudo mkfs.%s -b %d %s %s" % \
> + mkfs_cmd = "mkfs.%s -b %d %s %s" % \
> (self.fstype, self.size * 1024, label_str, rootfs)
> exec_cmd(mkfs_cmd)
>
> --
> 2.16.1
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo"
2018-04-16 10:44 ` Andreas Reichel
@ 2018-04-16 10:53 ` Andreas Reichel
2018-04-16 10:57 ` Henning Schild
1 sibling, 0 replies; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:53 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Mon, Apr 16, 2018 at 12:44:56PM +0200, [ext] Andreas Reichel wrote:
> On Fri, Apr 13, 2018 at 04:18:53PM +0200, Henning Schild wrote:
> > This reverts commit bf873d3b089474d9061df4c1f0985abfac038ff6.
> >
> > Not required if you run wic as root or set FAKEROOTCMD.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > scripts/lib/wic/partition.py | 27 +++++++++++++++++++--------
> > 1 file changed, 19 insertions(+), 8 deletions(-)
> >
> > diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> > index 34130d8..20bc4a2 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -201,6 +201,17 @@ class Partition():
> >
> > Currently handles ext2/3/4, btrfs and vfat.
> > """
> > + p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" % native_sysroot)
> > + p_localstatedir = os.environ.get("PSEUDO_LOCALSTATEDIR",
> > + "%s/../pseudo" % rootfs_dir)
> > + p_passwd = os.environ.get("PSEUDO_PASSWD", rootfs_dir)
> > + p_nosymlinkexp = os.environ.get("PSEUDO_NOSYMLINKEXP", "1")
> > + pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix
> > + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % p_localstatedir
> > + pseudo += "export PSEUDO_PASSWD=%s;" % p_passwd
> > + pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
> > + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
> > +
> In patch 11 you write that Isar does/ cannot use Pseudo. So why do you
> reinstantiate the variables here. I don't understand their purpose because
> they are prefixed with PSEUDO.
Okay please ignore this. I finally came to the cover letter :)
>
> > rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> > self.lineno, self.fstype)
> > if os.path.isfile(rootfs):
> > @@ -226,7 +237,7 @@ class Partition():
> > self.source_file = rootfs
> >
> > # get the rootfs size in the right units for kickstart (kB)
> > - du_cmd = "sudo du -Lbks %s" % rootfs
> > + du_cmd = "du -Lbks %s" % rootfs
> > out = exec_cmd(du_cmd)
> > self.size = int(out.split()[0])
> >
> > @@ -234,7 +245,7 @@ class Partition():
> > """
> > Prepare content for an ext2/3/4 rootfs partition.
> > """
> > - du_cmd = "sudo du -ks %s" % rootfs_dir
> > + du_cmd = "du -ks %s" % rootfs_dir
> > out = exec_cmd(du_cmd)
> > actual_rootfs_size = int(out.split()[0])
> >
> > @@ -249,7 +260,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -F %s %s %s -d %s" % \
> > + mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
> > (self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
> > exec_cmd(mkfs_cmd)
> >
> > @@ -262,7 +273,7 @@ class Partition():
> >
> > Currently handles ext2/3/4 and btrfs.
> > """
> > - du_cmd = "sudo du -ks %s" % rootfs_dir
> > + du_cmd = "du -ks %s" % rootfs_dir
> > out = exec_cmd(du_cmd)
> > actual_rootfs_size = int(out.split()[0])
> >
> > @@ -275,7 +286,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -b %d -r %s %s %s" % \
> > + mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
> > (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
> > exec_cmd(mkfs_cmd)
> >
> > @@ -283,7 +294,7 @@ class Partition():
> > """
> > Prepare content for a msdos/vfat rootfs partition.
> > """
> > - du_cmd = "sudo du -bks %s" % rootfs_dir
> > + du_cmd = "du -bks %s" % rootfs_dir
> > out = exec_cmd(du_cmd)
> > blocks = int(out.split()[0])
> >
> > @@ -331,7 +342,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -F %s %s %s" % \
> > + mkfs_cmd = "mkfs.%s -F %s %s %s" % \
> > (self.fstype, extra_imagecmd, label_str, rootfs)
> > exec_cmd(mkfs_cmd)
> >
> > @@ -347,7 +358,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -b %d %s %s" % \
> > + mkfs_cmd = "mkfs.%s -b %d %s %s" % \
> > (self.fstype, self.size * 1024, label_str, rootfs)
> > exec_cmd(mkfs_cmd)
> >
> > --
> > 2.16.1
> >
>
> --
> Andreas Reichel
> Dipl.-Phys. (Univ.)
> Software Consultant
>
> Andreas.Reichel@tngtech.com, +49-174-3180074
> TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
> Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
> Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
>
> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To post to this group, send email to isar-users@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20180416104456.GD8866%40iiotirae.
> For more options, visit https://groups.google.com/d/optout.
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo"
2018-04-16 10:44 ` Andreas Reichel
2018-04-16 10:53 ` Andreas Reichel
@ 2018-04-16 10:57 ` Henning Schild
1 sibling, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-16 10:57 UTC (permalink / raw)
To: Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Am Mon, 16 Apr 2018 12:44:56 +0200
schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
> On Fri, Apr 13, 2018 at 04:18:53PM +0200, Henning Schild wrote:
> > This reverts commit bf873d3b089474d9061df4c1f0985abfac038ff6.
> >
> > Not required if you run wic as root or set FAKEROOTCMD.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > scripts/lib/wic/partition.py | 27 +++++++++++++++++++--------
> > 1 file changed, 19 insertions(+), 8 deletions(-)
> >
> > diff --git a/scripts/lib/wic/partition.py
> > b/scripts/lib/wic/partition.py index 34130d8..20bc4a2 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -201,6 +201,17 @@ class Partition():
> >
> > Currently handles ext2/3/4, btrfs and vfat.
> > """
> > + p_prefix = os.environ.get("PSEUDO_PREFIX", "%s/usr" %
> > native_sysroot)
> > + p_localstatedir = os.environ.get("PSEUDO_LOCALSTATEDIR",
> > + "%s/../pseudo" %
> > rootfs_dir)
> > + p_passwd = os.environ.get("PSEUDO_PASSWD", rootfs_dir)
> > + p_nosymlinkexp = os.environ.get("PSEUDO_NOSYMLINKEXP", "1")
> > + pseudo = "export PSEUDO_PREFIX=%s;" % p_prefix
> > + pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" %
> > p_localstatedir
> > + pseudo += "export PSEUDO_PASSWD=%s;" % p_passwd
> > + pseudo += "export PSEUDO_NOSYMLINKEXP=%s;" % p_nosymlinkexp
> > + pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
> > +
> In patch 11 you write that Isar does/ cannot use Pseudo. So why do you
> reinstantiate the variables here. I don't understand their purpose
> because they are prefixed with PSEUDO.
This is not code from me. I just revert unnecessary changes that where
made to our copy of wic.
The code does not hurt if you set FAKEROOTCMD to something custom,
which my patch does.
Henning
> > rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> > self.lineno, self.fstype)
> > if os.path.isfile(rootfs):
> > @@ -226,7 +237,7 @@ class Partition():
> > self.source_file = rootfs
> >
> > # get the rootfs size in the right units for kickstart (kB)
> > - du_cmd = "sudo du -Lbks %s" % rootfs
> > + du_cmd = "du -Lbks %s" % rootfs
> > out = exec_cmd(du_cmd)
> > self.size = int(out.split()[0])
> >
> > @@ -234,7 +245,7 @@ class Partition():
> > """
> > Prepare content for an ext2/3/4 rootfs partition.
> > """
> > - du_cmd = "sudo du -ks %s" % rootfs_dir
> > + du_cmd = "du -ks %s" % rootfs_dir
> > out = exec_cmd(du_cmd)
> > actual_rootfs_size = int(out.split()[0])
> >
> > @@ -249,7 +260,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -F %s %s %s -d %s" % \
> > + mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
> > (self.fstype, extra_imagecmd, rootfs, label_str,
> > rootfs_dir) exec_cmd(mkfs_cmd)
> >
> > @@ -262,7 +273,7 @@ class Partition():
> >
> > Currently handles ext2/3/4 and btrfs.
> > """
> > - du_cmd = "sudo du -ks %s" % rootfs_dir
> > + du_cmd = "du -ks %s" % rootfs_dir
> > out = exec_cmd(du_cmd)
> > actual_rootfs_size = int(out.split()[0])
> >
> > @@ -275,7 +286,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -b %d -r %s %s %s" % \
> > + mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
> > (self.fstype, rootfs_size * 1024, rootfs_dir,
> > label_str, rootfs) exec_cmd(mkfs_cmd)
> >
> > @@ -283,7 +294,7 @@ class Partition():
> > """
> > Prepare content for a msdos/vfat rootfs partition.
> > """
> > - du_cmd = "sudo du -bks %s" % rootfs_dir
> > + du_cmd = "du -bks %s" % rootfs_dir
> > out = exec_cmd(du_cmd)
> > blocks = int(out.split()[0])
> >
> > @@ -331,7 +342,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -F %s %s %s" % \
> > + mkfs_cmd = "mkfs.%s -F %s %s %s" % \
> > (self.fstype, extra_imagecmd, label_str, rootfs)
> > exec_cmd(mkfs_cmd)
> >
> > @@ -347,7 +358,7 @@ class Partition():
> > if self.label:
> > label_str = "-L %s" % self.label
> >
> > - mkfs_cmd = "sudo mkfs.%s -b %d %s %s" % \
> > + mkfs_cmd = "mkfs.%s -b %d %s %s" % \
> > (self.fstype, self.size * 1024, label_str, rootfs)
> > exec_cmd(mkfs_cmd)
> >
> > --
> > 2.16.1
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 05/17] Revert "wic: Remove sysroot support"
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (3 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 04/17] Revert "wic: Use sudo instead of pseudo" Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 06/17] wic: now truly go for the wic version we claim to have Henning Schild
` (13 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This reverts commit dd109de433577a55f26e8a76f04dfb39e6ed95a5.
This is a useful feature, no need to remove it. If we had such a sysroot
it would actually be really nice to have it. Buildchroot contains
dynamically linked binaries so we can not use it as such, we have to
chroot in.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/lib/wic/engine.py | 7 ++-
scripts/lib/wic/partition.py | 59 +++++++++++++---------
scripts/lib/wic/pluginbase.py | 11 ++--
scripts/lib/wic/plugins/imager/direct.py | 49 ++++++++++--------
scripts/lib/wic/plugins/source/bootimg-efi.py | 13 +++--
.../lib/wic/plugins/source/bootimg-partition.py | 6 ++-
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 19 ++++---
.../lib/wic/plugins/source/isoimage-isohybrid.py | 25 +++++----
scripts/lib/wic/plugins/source/rawcopy.py | 2 +-
scripts/lib/wic/plugins/source/rootfs.py | 4 +-
scripts/wic | 27 +++++++++-
11 files changed, 140 insertions(+), 82 deletions(-)
diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 15826f2..f59821f 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -145,7 +145,8 @@ def list_source_plugins():
print(" %s" % plugin)
-def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, options):
+def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir,
+ native_sysroot, options):
"""
Create image
@@ -153,6 +154,7 @@ def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, options):
rootfs_dir - absolute path to the build's /rootfs dir
bootimg_dir - absolute path to the build's boot artifacts directory
kernel_dir - absolute path to the build's kernel directory
+ native_sysroot - absolute path to the build's native sysroots dir
image_output_dir - dirname to create for image
options - wic command line options (debug, bmap, etc)
@@ -164,6 +166,7 @@ def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, options):
rootfs_dir: IMAGE_ROOTFS
kernel_dir: DEPLOY_DIR_IMAGE
+ native_sysroot: STAGING_DIR_NATIVE
In the above case, bootimg_dir remains unset and the
plugin-specific image creation code is responsible for finding the
@@ -187,7 +190,7 @@ def wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, options):
raise WicError('Unknown plugin: %s' % pname)
plugin = plugin_class(wks_file, rootfs_dir, bootimg_dir, kernel_dir,
- oe_builddir, options)
+ native_sysroot, oe_builddir, options)
plugin.do_create()
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 20bc4a2..939e667 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -29,7 +29,7 @@ import os
import tempfile
from wic import WicError
-from wic.utils.misc import exec_cmd, get_bitbake_var
+from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var
from wic.pluginbase import PluginMgr
logger = logging.getLogger('wic')
@@ -125,7 +125,7 @@ class Partition():
return self.fixed_size if self.fixed_size else self.size
def prepare(self, creator, cr_workdir, oe_builddir, rootfs_dir,
- bootimg_dir, kernel_dir):
+ bootimg_dir, kernel_dir, native_sysroot):
"""
Prepare content for individual partitions, depending on
partition command parameters.
@@ -137,7 +137,8 @@ class Partition():
"partition." % self.mountpoint)
if self.fstype == "swap":
- self.prepare_swap_partition(cr_workdir, oe_builddir)
+ self.prepare_swap_partition(cr_workdir, oe_builddir,
+ native_sysroot)
self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype)
else:
if self.fstype == 'squashfs':
@@ -151,7 +152,7 @@ class Partition():
prefix = "ext" if self.fstype.startswith("ext") else self.fstype
method = getattr(self, "prepare_empty_partition_" + prefix)
- method(rootfs, oe_builddir)
+ method(rootfs, oe_builddir, native_sysroot)
self.source_file = rootfs
return
@@ -174,13 +175,13 @@ class Partition():
plugin = PluginMgr.get_plugins('source')[self.source]
plugin.do_configure_partition(self, srcparams_dict, creator,
cr_workdir, oe_builddir, bootimg_dir,
- kernel_dir)
+ kernel_dir, native_sysroot)
plugin.do_stage_partition(self, srcparams_dict, creator,
cr_workdir, oe_builddir, bootimg_dir,
- kernel_dir)
+ kernel_dir, native_sysroot)
plugin.do_prepare_partition(self, srcparams_dict, creator,
cr_workdir, oe_builddir, bootimg_dir,
- kernel_dir, rootfs_dir)
+ kernel_dir, rootfs_dir, native_sysroot)
# further processing required Partition.size to be an integer, make
# sure that it is one
@@ -194,7 +195,8 @@ class Partition():
"larger (%d kB) than its allowed size %d kB" %
(self.mountpoint, self.size, self.fixed_size))
- def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir):
+ def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
+ native_sysroot):
"""
Prepare content for a rootfs partition i.e. create a partition
and fill it from a /rootfs dir.
@@ -233,7 +235,7 @@ class Partition():
prefix = "ext" if self.fstype.startswith("ext") else self.fstype
method = getattr(self, "prepare_rootfs_" + prefix)
- method(rootfs, oe_builddir, rootfs_dir)
+ method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
self.source_file = rootfs
# get the rootfs size in the right units for kickstart (kB)
@@ -241,7 +243,8 @@ class Partition():
out = exec_cmd(du_cmd)
self.size = int(out.split()[0])
- def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir):
+ def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
"""
Prepare content for an ext2/3/4 rootfs partition.
"""
@@ -262,12 +265,13 @@ class Partition():
mkfs_cmd = "mkfs.%s -F %s %s %s -d %s" % \
(self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
- exec_cmd(mkfs_cmd)
+ exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
- def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir):
+ def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
"""
Prepare content for a btrfs rootfs partition.
@@ -288,9 +292,10 @@ class Partition():
mkfs_cmd = "mkfs.%s -b %d -r %s %s %s" % \
(self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
- exec_cmd(mkfs_cmd)
+ exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
- def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir):
+ def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
"""
Prepare content for a msdos/vfat rootfs partition.
"""
@@ -310,7 +315,7 @@ class Partition():
dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
rootfs, rootfs_size)
- exec_cmd(dosfs_cmd)
+ exec_native_cmd(dosfs_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
exec_native_cmd(mcopy_cmd, native_sysroot)
@@ -320,15 +325,17 @@ class Partition():
prepare_rootfs_vfat = prepare_rootfs_msdos
- def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir):
+ def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
"""
Prepare content for a squashfs rootfs partition.
"""
squashfs_cmd = "mksquashfs %s %s -noappend" % \
(rootfs_dir, rootfs)
- exec_cmd(squashfs_cmd)
+ exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo)
- def prepare_empty_partition_ext(self, rootfs, oe_builddir):
+ def prepare_empty_partition_ext(self, rootfs, oe_builddir,
+ native_sysroot):
"""
Prepare an empty ext2/3/4 partition.
"""
@@ -344,9 +351,10 @@ class Partition():
mkfs_cmd = "mkfs.%s -F %s %s %s" % \
(self.fstype, extra_imagecmd, label_str, rootfs)
- exec_cmd(mkfs_cmd)
+ exec_native_cmd(mkfs_cmd, native_sysroot)
- def prepare_empty_partition_btrfs(self, rootfs, oe_builddir):
+ def prepare_empty_partition_btrfs(self, rootfs, oe_builddir,
+ native_sysroot):
"""
Prepare an empty btrfs partition.
"""
@@ -360,9 +368,10 @@ class Partition():
mkfs_cmd = "mkfs.%s -b %d %s %s" % \
(self.fstype, self.size * 1024, label_str, rootfs)
- exec_cmd(mkfs_cmd)
+ exec_native_cmd(mkfs_cmd, native_sysroot)
- def prepare_empty_partition_msdos(self, rootfs, oe_builddir):
+ def prepare_empty_partition_msdos(self, rootfs, oe_builddir,
+ native_sysroot):
"""
Prepare an empty vfat partition.
"""
@@ -378,14 +387,14 @@ class Partition():
dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
rootfs, blocks)
- exec_cmd(dosfs_cmd)
+ exec_native_cmd(dosfs_cmd, native_sysroot)
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
prepare_empty_partition_vfat = prepare_empty_partition_msdos
- def prepare_swap_partition(self, cr_workdir, oe_builddir):
+ def prepare_swap_partition(self, cr_workdir, oe_builddir, native_sysroot):
"""
Prepare a swap partition.
"""
@@ -399,4 +408,4 @@ class Partition():
if self.label:
label_str = "-L %s" % self.label
mkswap_cmd = "mkswap %s -U %s %s" % (label_str, str(uuid.uuid1()), path)
- exec_cmd(mkswap_cmd)
+ exec_native_cmd(mkswap_cmd, native_sysroot)
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index 86d4a6d..fb3d179 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -92,7 +92,7 @@ class SourcePlugin(metaclass=PluginMeta):
@classmethod
def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
- bootimg_dir, kernel_dir):
+ bootimg_dir, kernel_dir, native_sysroot):
"""
Called after all partitions have been prepared and assembled into a
disk image. This provides a hook to allow finalization of a
@@ -102,7 +102,8 @@ class SourcePlugin(metaclass=PluginMeta):
@classmethod
def do_stage_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
"""
Special content staging hook called before do_prepare_partition(),
normally empty.
@@ -118,7 +119,8 @@ class SourcePlugin(metaclass=PluginMeta):
@classmethod
def do_configure_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
"""
Called before do_prepare_partition(), typically used to create
custom configuration files for a partition, for example
@@ -128,7 +130,8 @@ class SourcePlugin(metaclass=PluginMeta):
@classmethod
def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir, rootfs_dir):
+ oe_builddir, bootimg_dir, kernel_dir, rootfs_dir,
+ native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 2e587b5..f2e6127 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -36,7 +36,7 @@ from wic import WicError
from wic.filemap import sparse_copy
from wic.ksparser import KickStart, KickStartError
from wic.pluginbase import PluginMgr, ImagerPlugin
-from wic.utils.misc import get_bitbake_var, exec_cmd
+from wic.utils.misc import get_bitbake_var, exec_cmd, exec_native_cmd
logger = logging.getLogger('wic')
@@ -52,7 +52,7 @@ class DirectPlugin(ImagerPlugin):
name = 'direct'
def __init__(self, wks_file, rootfs_dir, bootimg_dir, kernel_dir,
- oe_builddir, options):
+ native_sysroot, oe_builddir, options):
try:
self.ks = KickStart(wks_file)
except KickStartError as err:
@@ -62,6 +62,7 @@ class DirectPlugin(ImagerPlugin):
self.rootfs_dir = dict(rdir.split('=') for rdir in rootfs_dir.split(' '))
self.bootimg_dir = bootimg_dir
self.kernel_dir = kernel_dir
+ self.native_sysroot = native_sysroot
self.oe_builddir = oe_builddir
self.outdir = options.outdir
@@ -84,7 +85,7 @@ class DirectPlugin(ImagerPlugin):
image_path = self._full_path(self.workdir, self.parts[0].disk, "direct")
self._image = PartitionedImage(image_path, self.ptable_format,
- self.parts)
+ self.parts, self.native_sysroot)
def do_create(self):
"""
@@ -198,13 +199,14 @@ class DirectPlugin(ImagerPlugin):
plugin = PluginMgr.get_plugins('source')[source_plugin]
plugin.do_install_disk(self._image, disk_name, self, self.workdir,
self.oe_builddir, self.bootimg_dir,
- self.kernel_dir)
+ self.kernel_dir, self.native_sysroot)
full_path = self._image.path
# Generate .bmap
if self.bmap:
logger.debug("Generating bmap file for %s", disk_name)
- exec_cmd("bmaptool create %s -o %s.bmap" % (full_path, full_path))
+ exec_native_cmd("bmaptool create %s -o %s.bmap" % (full_path, full_path),
+ self.native_sysroot)
# Compress the image
if self.compressor:
logger.debug("Compressing disk %s with %s", disk_name, self.compressor)
@@ -235,6 +237,7 @@ class DirectPlugin(ImagerPlugin):
msg += ' BOOTIMG_DIR: %s\n' % self.bootimg_dir
msg += ' KERNEL_DIR: %s\n' % self.kernel_dir
+ msg += ' NATIVE_SYSROOT: %s\n' % self.native_sysroot
logger.info(msg)
@@ -284,7 +287,7 @@ class PartitionedImage():
Partitioned image in a file.
"""
- def __init__(self, path, ptable_format, partitions):
+ def __init__(self, path, ptable_format, partitions, native_sysroot=None):
self.path = path # Path to the image file
self.numpart = 0 # Number of allocated partitions
self.realpart = 0 # Number of partitions in the partition table
@@ -299,6 +302,7 @@ class PartitionedImage():
self.partimages = []
# Size of a sector used in calculations
self.sector_size = SECTOR_SIZE
+ self.native_sysroot = native_sysroot
# calculate the real partition number, accounting for partitions not
# in the partition table and logical partitions
@@ -328,7 +332,7 @@ class PartitionedImage():
# sizes before we can add them and do the layout.
part.prepare(imager, imager.workdir, imager.oe_builddir,
imager.rootfs_dir, imager.bootimg_dir,
- imager.kernel_dir)
+ imager.kernel_dir, imager.native_sysroot)
# Converting kB to sectors for parted
part.size_sec = part.disk_size * 1024 // self.sector_size
@@ -439,7 +443,7 @@ class PartitionedImage():
cmd += " %s" % fstype
cmd += " %d %d" % (start, end)
- return exec_cmd(cmd)
+ return exec_native_cmd(cmd, self.native_sysroot)
def create(self):
logger.debug("Creating sparse file %s", self.path)
@@ -447,8 +451,8 @@ class PartitionedImage():
os.ftruncate(sparse.fileno(), self.min_size)
logger.debug("Initializing partition table for %s", self.path)
- exec_cmd("parted -s %s mklabel %s" %
- (self.path, self.ptable_format))
+ exec_native_cmd("parted -s %s mklabel %s" %
+ (self.path, self.ptable_format), self.native_sysroot)
logger.debug("Set disk identifier %x", self.identifier)
with open(self.path, 'r+b') as img:
@@ -504,30 +508,35 @@ class PartitionedImage():
if part.part_type:
logger.debug("partition %d: set type UID to %s",
part.num, part.part_type)
- exec_cmd("sgdisk --typecode=%d:%s %s" % \
- (part.num, part.part_type, self.path))
+ exec_native_cmd("sgdisk --typecode=%d:%s %s" % \
+ (part.num, part.part_type,
+ self.path), self.native_sysroot)
if part.uuid and self.ptable_format == "gpt":
logger.debug("partition %d: set UUID to %s",
part.num, part.uuid)
- exec_cmd("sgdisk --partition-guid=%d:%s %s" % \
- (part.num, part.uuid, self.path))
+ exec_native_cmd("sgdisk --partition-guid=%d:%s %s" % \
+ (part.num, part.uuid, self.path),
+ self.native_sysroot)
if part.label and self.ptable_format == "gpt":
logger.debug("partition %d: set name to %s",
part.num, part.label)
- exec_cmd("parted -s %s name %d %s" % \
- (self.path, part.num, part.label))
+ exec_native_cmd("parted -s %s name %d %s" % \
+ (self.path, part.num, part.label),
+ self.native_sysroot)
if part.active:
flag_name = "legacy_boot" if self.ptable_format == 'gpt' else "boot"
logger.debug("Set '%s' flag for partition '%s' on disk '%s'",
flag_name, part.num, self.path)
- exec_cmd("parted -s %s set %d %s on" % \
- (self.path, part.num, flag_name))
+ exec_native_cmd("parted -s %s set %d %s on" % \
+ (self.path, part.num, flag_name),
+ self.native_sysroot)
if part.system_id:
- exec_cmd("sfdisk --part-type %s %s %s" % \
- (self.path, part.num, part.system_id))
+ exec_native_cmd("sfdisk --part-type %s %s %s" % \
+ (self.path, part.num, part.system_id),
+ self.native_sysroot)
def cleanup(self):
# remove partition images
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 823a536..9879cb9 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -31,7 +31,8 @@ import shutil
from wic import WicError
from wic.engine import get_custom_config
from wic.pluginbase import SourcePlugin
-from wic.utils.misc import (exec_cmd, get_bitbake_var, BOOTDD_EXTRA_SPACE)
+from wic.utils.misc import (exec_cmd, exec_native_cmd, get_bitbake_var,
+ BOOTDD_EXTRA_SPACE)
logger = logging.getLogger('wic')
@@ -154,7 +155,8 @@ class BootimgEFIPlugin(SourcePlugin):
@classmethod
def do_configure_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
"""
Called before do_prepare_partition(), creates loader-specific config
"""
@@ -176,7 +178,8 @@ class BootimgEFIPlugin(SourcePlugin):
@classmethod
def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir, rootfs_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
@@ -238,10 +241,10 @@ class BootimgEFIPlugin(SourcePlugin):
bootimg = "%s/boot.img" % cr_workdir
dosfs_cmd = "mkdosfs -n efi -C %s %d" % (bootimg, blocks)
- exec_cmd(dosfs_cmd)
+ exec_native_cmd(dosfs_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
- exec_cmd(mcopy_cmd)
+ exec_native_cmd(mcopy_cmd, native_sysroot)
chmod_cmd = "chmod 644 %s" % bootimg
exec_cmd(chmod_cmd)
diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py b/scripts/lib/wic/plugins/source/bootimg-partition.py
index a00dc44..13fddbd 100644
--- a/scripts/lib/wic/plugins/source/bootimg-partition.py
+++ b/scripts/lib/wic/plugins/source/bootimg-partition.py
@@ -45,7 +45,8 @@ class BootimgPartitionPlugin(SourcePlugin):
@classmethod
def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir, rootfs_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
@@ -118,4 +119,5 @@ class BootimgPartitionPlugin(SourcePlugin):
exec_cmd(install_cmd)
logger.debug('Prepare boot partition using rootfs in %s', hdddir)
- part.prepare_rootfs(cr_workdir, oe_builddir, hdddir)
+ part.prepare_rootfs(cr_workdir, oe_builddir, hdddir,
+ native_sysroot)
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index b029c2f..5890c12 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -31,7 +31,8 @@ from wic import WicError
from wic.engine import get_custom_config
from wic.utils import runner
from wic.pluginbase import SourcePlugin
-from wic.utils.misc import (exec_cmd, get_bitbake_var, BOOTDD_EXTRA_SPACE)
+from wic.utils.misc import (exec_cmd, exec_native_cmd,
+ get_bitbake_var, BOOTDD_EXTRA_SPACE)
logger = logging.getLogger('wic')
@@ -56,7 +57,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
@classmethod
def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
- bootimg_dir, kernel_dir):
+ bootimg_dir, kernel_dir, native_sysroot):
"""
Called after all partitions have been prepared and assembled into a
disk image. In this case, we install the MBR.
@@ -81,11 +82,12 @@ class BootimgPcbiosPlugin(SourcePlugin):
disk_name, full_path, disk.min_size)
dd_cmd = "dd if=%s of=%s conv=notrunc" % (mbrfile, full_path)
- exec_cmd(dd_cmd)
+ exec_cmd(dd_cmd, native_sysroot)
@classmethod
def do_configure_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
"""
Called before do_prepare_partition(), creates syslinux config
"""
@@ -142,7 +144,8 @@ class BootimgPcbiosPlugin(SourcePlugin):
@classmethod
def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir, rootfs_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
@@ -186,13 +189,13 @@ class BootimgPcbiosPlugin(SourcePlugin):
bootimg = "%s/boot.img" % cr_workdir
dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, blocks)
- exec_cmd(dosfs_cmd)
+ exec_native_cmd(dosfs_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
- exec_cmd(mcopy_cmd)
+ exec_native_cmd(mcopy_cmd, native_sysroot)
syslinux_cmd = "syslinux %s" % bootimg
- exec_cmd(syslinux_cmd)
+ exec_native_cmd(syslinux_cmd, native_sysroot)
chmod_cmd = "chmod 644 %s" % bootimg
exec_cmd(chmod_cmd)
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index dcb6434..1ceba62 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -29,7 +29,7 @@ import shutil
from wic import WicError
from wic.engine import get_custom_config
from wic.pluginbase import SourcePlugin
-from wic.utils.misc import exec_cmd, get_bitbake_var
+from wic.utils.misc import exec_cmd, exec_native_cmd, get_bitbake_var
logger = logging.getLogger('wic')
@@ -199,7 +199,8 @@ class IsoImagePlugin(SourcePlugin):
@classmethod
def do_configure_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
"""
Called before do_prepare_partition(), creates loader-specific config
"""
@@ -220,7 +221,8 @@ class IsoImagePlugin(SourcePlugin):
@classmethod
def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
- oe_builddir, bootimg_dir, kernel_dir, rootfs_dir):
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
@@ -269,7 +271,8 @@ class IsoImagePlugin(SourcePlugin):
part.size = int(out.split()[0])
part.extra_space = 0
part.overhead_factor = 1.2
- part.prepare_rootfs(cr_workdir, oe_builddir, rootfs_dir)
+ part.prepare_rootfs(cr_workdir, oe_builddir, rootfs_dir, \
+ native_sysroot)
rootfs_img = part.source_file
install_cmd = "install -m 0644 %s %s/rootfs.img" \
@@ -360,7 +363,7 @@ class IsoImagePlugin(SourcePlugin):
grub_cmd += "reboot serial terminfo iso9660 loopback tar "
grub_cmd += "memdisk ls search_fs_uuid udf btrfs xfs lvm "
grub_cmd += "reiserfs ata "
- exec_cmd(grub_cmd)
+ exec_native_cmd(grub_cmd, native_sysroot)
else:
raise WicError("unrecognized bootimg-efi loader: %s" %
@@ -397,14 +400,14 @@ class IsoImagePlugin(SourcePlugin):
dosfs_cmd = 'mkfs.vfat -n "EFIimg" -S 512 -C %s %d' \
% (bootimg, blocks)
- exec_cmd(dosfs_cmd)
+ exec_native_cmd(dosfs_cmd, native_sysroot)
mmd_cmd = "mmd -i %s ::/EFI" % bootimg
- exec_cmd(mmd_cmd)
+ exec_native_cmd(mmd_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/EFI/* ::/EFI/" \
% (bootimg, isodir)
- exec_cmd(mcopy_cmd)
+ exec_native_cmd(mcopy_cmd, native_sysroot)
chmod_cmd = "chmod 644 %s" % bootimg
exec_cmd(chmod_cmd)
@@ -453,7 +456,7 @@ class IsoImagePlugin(SourcePlugin):
mkisofs_cmd += "-no-emul-boot %s " % isodir
logger.debug("running command: %s", mkisofs_cmd)
- exec_cmd(mkisofs_cmd)
+ exec_native_cmd(mkisofs_cmd, native_sysroot)
shutil.rmtree(isodir)
@@ -466,7 +469,7 @@ class IsoImagePlugin(SourcePlugin):
@classmethod
def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
- bootimg_dir, kernel_dir):
+ bootimg_dir, kernel_dir, native_sysroot):
"""
Called after all partitions have been prepared and assembled into a
disk image. In this case, we insert/modify the MBR using isohybrid
@@ -479,7 +482,7 @@ class IsoImagePlugin(SourcePlugin):
isohybrid_cmd = "isohybrid -u %s" % iso_img
logger.debug("running command: %s", isohybrid_cmd)
- exec_cmd(isohybrid_cmd)
+ exec_native_cmd(isohybrid_cmd, native_sysroot)
# Replace the image created by direct plugin with the one created by
# mkisofs command. This is necessary because the iso image created by
diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
index 392ae7e..e1c4f5e 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -35,7 +35,7 @@ class RawCopyPlugin(SourcePlugin):
@classmethod
def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
oe_builddir, bootimg_dir, kernel_dir,
- rootfs_dir):
+ rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index e6cdc88..f2e2ca8 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -60,7 +60,7 @@ class RootfsPlugin(SourcePlugin):
@classmethod
def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
oe_builddir, bootimg_dir, kernel_dir,
- krootfs_dir):
+ krootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
@@ -122,4 +122,4 @@ class RootfsPlugin(SourcePlugin):
part.rootfs_dir = real_rootfs_dir
part.prepare_rootfs(cr_workdir, oe_builddir,
- real_rootfs_dir)
+ real_rootfs_dir, native_sysroot)
diff --git a/scripts/wic b/scripts/wic
index 641dd2e..a5f2dbf 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -123,6 +123,9 @@ def wic_create_subcommand(args, usage_str):
parser.add_option("-k", "--kernel-dir", dest="kernel_dir",
help="path to the dir containing the kernel to use "
"in the .wks bootimg")
+ parser.add_option("-n", "--native-sysroot", dest="native_sysroot",
+ help="path to the native sysroot containing the tools "
+ "to use to build the image")
parser.add_option("-s", "--skip-build-check", dest="build_check",
action="store_false", default=True, help="skip the build check")
parser.add_option("-f", "--build-rootfs", action="store_true", help="build rootfs")
@@ -149,7 +152,8 @@ def wic_create_subcommand(args, usage_str):
missed = []
for val, opt in [(options.rootfs_dir, 'rootfs-dir'),
(options.bootimg_dir, 'bootimg-dir'),
- (options.kernel_dir, 'kernel-dir')]:
+ (options.kernel_dir, 'kernel-dir'),
+ (options.native_sysroot, 'native-sysroot')]:
if not val:
missed.append(opt)
if missed:
@@ -184,10 +188,23 @@ def wic_create_subcommand(args, usage_str):
rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", options.image_name)
kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE", options.image_name)
bootimg_dir = get_bitbake_var("STAGING_DATADIR", options.image_name)
+ native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE",
+ options.image_name) #, cache=False)
else:
if options.build_rootfs:
raise WicError("Image name is not specified, exiting. "
"(Use -e/--image-name to specify it)")
+ native_sysroot = options.native_sysroot
+
+ if not native_sysroot or not os.path.isdir(native_sysroot):
+ logger.info("Building wic-tools...\n")
+ if bitbake_main(BitBakeConfigParameters("bitbake wic-tools".split()),
+ cookerdata.CookerConfiguration()):
+ raise WicError("bitbake wic-tools failed")
+ native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
+ if not native_sysroot:
+ raise WicError("Unable to find the location of the native "
+ "tools sysroot to use")
wks_file = args[0]
@@ -204,18 +221,23 @@ def wic_create_subcommand(args, usage_str):
rootfs_dir = options.rootfs_dir['ROOTFS_DIR']
bootimg_dir = options.bootimg_dir
kernel_dir = options.kernel_dir
+ native_sysroot = options.native_sysroot
if rootfs_dir and not os.path.isdir(rootfs_dir):
raise WicError("--rootfs-dir (-r) not found, exiting")
if not os.path.isdir(bootimg_dir):
raise WicError("--bootimg-dir (-b) not found, exiting")
if not os.path.isdir(kernel_dir):
raise WicError("--kernel-dir (-k) not found, exiting")
+ if not os.path.isdir(native_sysroot):
+ raise WicError("--native-sysroot (-n) not found, exiting")
else:
not_found = not_found_dir = ""
if not os.path.isdir(rootfs_dir):
(not_found, not_found_dir) = ("rootfs-dir", rootfs_dir)
elif not os.path.isdir(kernel_dir):
(not_found, not_found_dir) = ("kernel-dir", kernel_dir)
+ elif not os.path.isdir(native_sysroot):
+ (not_found, not_found_dir) = ("native-sysroot", native_sysroot)
if not_found:
if not not_found_dir:
not_found_dir = "Completely missing artifact - wrong image (.wks) used?"
@@ -233,7 +255,8 @@ def wic_create_subcommand(args, usage_str):
rootfs_dir = rootfs_dir_to_args(krootfs_dir)
logger.info("Creating image(s)...\n")
- engine.wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir, options)
+ engine.wic_create(wks_file, rootfs_dir, bootimg_dir, kernel_dir,
+ native_sysroot, options)
def wic_list_subcommand(args, usage_str):
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 06/17] wic: now truly go for the wic version we claim to have
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (4 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 05/17] Revert "wic: Remove sysroot support" Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 07/17] Revert "isar-init-build-env: Add /sbin to PATH" Henning Schild
` (12 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
2b164b18fd639c9 claims to introduce wic hash 131629ca6238ea05
This commit really carries that version of wic.
Issue:
- the wic version in Isar was modified
- that causes:
- confusion, maintainability and updateability issues
- potential quality issues
Impact:
This patch and the previous reverts get wic back to a state where all
these Issues are solved. We could now just update our wic without having
to worry about local patches. In case of a wic-update Isar and layers
on top would still have to review their plugins.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/lib/wic/canned-wks/qemux86-directdisk.wks | 2 +-
scripts/lib/wic/filemap.py | 6 +-
scripts/lib/wic/help.py | 2 -
scripts/lib/wic/ksparser.py | 4 +-
scripts/lib/wic/partition.py | 114 ++++++++++++++--------
scripts/lib/wic/plugins/imager/direct.py | 17 +++-
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 48 ++++-----
scripts/lib/wic/plugins/source/fsimage.py | 56 +++++++++++
scripts/lib/wic/utils/misc.py | 16 +--
scripts/lib/wic/utils/runner.py | 74 ++++++++++++--
10 files changed, 250 insertions(+), 89 deletions(-)
create mode 100644 scripts/lib/wic/plugins/source/fsimage.py
diff --git a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
index db30bbc..a6518a0 100644
--- a/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
+++ b/scripts/lib/wic/canned-wks/qemux86-directdisk.wks
@@ -4,5 +4,5 @@
include common.wks.inc
-bootloader --timeout=0 --append="vga=0 uvesafb.mode_option=640x480-32 root=/dev/sda2 rw mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 oprofile.timer=1 rootfstype=ext4 "
+bootloader --timeout=0 --append="vga=0 uvesafb.mode_option=640x480-32 root=/dev/vda2 rw mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 oprofile.timer=1 rootfstype=ext4 "
diff --git a/scripts/lib/wic/filemap.py b/scripts/lib/wic/filemap.py
index 1f1aacc..080668e 100644
--- a/scripts/lib/wic/filemap.py
+++ b/scripts/lib/wic/filemap.py
@@ -530,11 +530,9 @@ def filemap(image, log=None):
except ErrorNotSupp:
return FilemapSeek(image, log)
-def sparse_copy(src_fname, dst_fname, offset=0, skip=0, api=None):
+def sparse_copy(src_fname, dst_fname, offset=0, skip=0):
"""Efficiently copy sparse file to or into another file."""
- if not api:
- api = filemap
- fmap = api(src_fname)
+ fmap = filemap(src_fname)
try:
dst_file = open(dst_fname, 'r+b')
except IOError:
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index aee2451..148da89 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -687,8 +687,6 @@ DESCRIPTION
apply to partitions created using '--source rootfs' (see
--source above). Valid values are:
- vfat
- msdos
ext2
ext3
ext4
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index d026caa..a039300 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -136,9 +136,7 @@ class KickStart():
part.add_argument('--exclude-path', nargs='+')
part.add_argument("--extra-space", type=sizetype)
part.add_argument('--fsoptions', dest='fsopts')
- part.add_argument('--fstype', default='vfat',
- choices=('ext2', 'ext3', 'ext4', 'btrfs',
- 'squashfs', 'vfat', 'msdos', 'swap'))
+ part.add_argument('--fstype')
part.add_argument('--label')
part.add_argument('--no-table', action='store_true')
part.add_argument('--ondisk', '--ondrive', dest='disk', default='sda')
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 939e667..8e32afc 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -136,24 +136,22 @@ class Partition():
"specify a non-zero --size/--fixed-size for that "
"partition." % self.mountpoint)
- if self.fstype == "swap":
+ if self.fstype and self.fstype == "swap":
self.prepare_swap_partition(cr_workdir, oe_builddir,
native_sysroot)
self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype)
- else:
- if self.fstype == 'squashfs':
- raise WicError("It's not possible to create empty squashfs "
- "partition '%s'" % (self.mountpoint))
-
+ elif self.fstype:
rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label,
self.lineno, self.fstype)
if os.path.isfile(rootfs):
os.remove(rootfs)
-
- prefix = "ext" if self.fstype.startswith("ext") else self.fstype
- method = getattr(self, "prepare_empty_partition_" + prefix)
- method(rootfs, oe_builddir, native_sysroot)
- self.source_file = rootfs
+ for prefix in ("ext", "btrfs", "vfat", "squashfs"):
+ if self.fstype.startswith(prefix):
+ method = getattr(self,
+ "prepare_empty_partition_" + prefix)
+ method(rootfs, oe_builddir, native_sysroot)
+ self.source_file = rootfs
+ break
return
plugins = PluginMgr.get_plugins('source')
@@ -195,6 +193,19 @@ class Partition():
"larger (%d kB) than its allowed size %d kB" %
(self.mountpoint, self.size, self.fixed_size))
+ def prepare_rootfs_from_fs_image(self, cr_workdir, oe_builddir,
+ rootfs_dir):
+ """
+ Handle an already-created partition e.g. xxx.ext3
+ """
+ rootfs = oe_builddir
+ du_cmd = "du -Lbks %s" % rootfs
+ out = exec_cmd(du_cmd)
+ rootfs_size = out.split()[0]
+
+ self.size = int(rootfs_size)
+ self.source_file = rootfs
+
def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot):
"""
@@ -219,6 +230,10 @@ class Partition():
if os.path.isfile(rootfs):
os.remove(rootfs)
+ if not self.fstype:
+ raise WicError("File system for partition %s not specified in "
+ "kickstart, use --fstype option" % self.mountpoint)
+
# Get rootfs size from bitbake variable if it's not set in .ks file
if not self.size:
# Bitbake variable ROOTFS_SIZE is calculated in
@@ -233,15 +248,19 @@ class Partition():
'--overhead-factor will be applied')
self.size = int(round(float(rsize_bb)))
- prefix = "ext" if self.fstype.startswith("ext") else self.fstype
- method = getattr(self, "prepare_rootfs_" + prefix)
- method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
- self.source_file = rootfs
+ for prefix in ("ext", "btrfs", "vfat", "squashfs"):
+ if self.fstype.startswith(prefix):
+ method = getattr(self, "prepare_rootfs_" + prefix)
+ method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
- # get the rootfs size in the right units for kickstart (kB)
- du_cmd = "du -Lbks %s" % rootfs
- out = exec_cmd(du_cmd)
- self.size = int(out.split()[0])
+ self.source_file = rootfs
+
+ # get the rootfs size in the right units for kickstart (kB)
+ du_cmd = "du -Lbks %s" % rootfs
+ out = exec_cmd(du_cmd)
+ self.size = int(out.split()[0])
+
+ break
def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
@@ -267,7 +286,7 @@ class Partition():
(self.fstype, extra_imagecmd, rootfs, label_str, rootfs_dir)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
- mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
+ mkfs_cmd = "fsck.%s -fy %s" % (self.fstype, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
@@ -294,10 +313,10 @@ class Partition():
(self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
- def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir,
- native_sysroot, pseudo):
+ def prepare_rootfs_vfat(self, rootfs, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
"""
- Prepare content for a msdos/vfat rootfs partition.
+ Prepare content for a vfat rootfs partition.
"""
du_cmd = "du -bks %s" % rootfs_dir
out = exec_cmd(du_cmd)
@@ -309,12 +328,7 @@ class Partition():
if self.label:
label_str = "-n %s" % self.label
- size_str = ""
- if self.fstype == 'msdos':
- size_str = "-F 16" # FAT 16
-
- dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
- rootfs, rootfs_size)
+ dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, rootfs, rootfs_size)
exec_native_cmd(dosfs_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
@@ -323,8 +337,6 @@ class Partition():
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
- prepare_rootfs_vfat = prepare_rootfs_msdos
-
def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
@@ -370,8 +382,8 @@ class Partition():
(self.fstype, self.size * 1024, label_str, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot)
- def prepare_empty_partition_msdos(self, rootfs, oe_builddir,
- native_sysroot):
+ def prepare_empty_partition_vfat(self, rootfs, oe_builddir,
+ native_sysroot):
"""
Prepare an empty vfat partition.
"""
@@ -381,18 +393,40 @@ class Partition():
if self.label:
label_str = "-n %s" % self.label
- size_str = ""
- if self.fstype == 'msdos':
- size_str = "-F 16" # FAT 16
-
- dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
- rootfs, blocks)
+ dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, rootfs, blocks)
exec_native_cmd(dosfs_cmd, native_sysroot)
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
- prepare_empty_partition_vfat = prepare_empty_partition_msdos
+ def prepare_empty_partition_squashfs(self, cr_workdir, oe_builddir,
+ native_sysroot):
+ """
+ Prepare an empty squashfs partition.
+ """
+ logger.warning("Creating of an empty squashfs %s partition was attempted. "
+ "Proceeding as requested.", self.mountpoint)
+
+ path = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype)
+ if os.path.isfile(path):
+ os.remove(path)
+
+ # it is not possible to create a squashfs without source data,
+ # thus prepare an empty temp dir that is used as source
+ tmpdir = tempfile.mkdtemp()
+
+ squashfs_cmd = "mksquashfs %s %s -noappend" % \
+ (tmpdir, path)
+ exec_native_cmd(squashfs_cmd, native_sysroot)
+
+ os.rmdir(tmpdir)
+
+ # get the rootfs size in the right units for kickstart (kB)
+ du_cmd = "du -Lbks %s" % path
+ out = exec_cmd(du_cmd)
+ fs_size = out.split()[0]
+
+ self.size = int(fs_size)
def prepare_swap_partition(self, cr_workdir, oe_builddir, native_sysroot):
"""
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index f2e6127..7d38ab3 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -323,7 +323,7 @@ class PartitionedImage():
if self.ptable_format == 'gpt':
part.uuid = str(uuid.uuid4())
else: # msdos partition table
- part.uuid = '%08x-%02d' % (self.identifier, part.realnum)
+ part.uuid = '%0x-%02d' % (self.identifier, part.realnum)
def prepare(self, imager):
"""Prepare an image. Call prepare method of all image partitions."""
@@ -487,8 +487,8 @@ class PartitionedImage():
parted_fs_type = "fat32"
elif part.fstype == "msdos":
parted_fs_type = "fat16"
- if not part.system_id:
- part.system_id = '0x6' # FAT16
+ elif part.fstype == "ontrackdm6aux3":
+ parted_fs_type = "ontrackdm6aux3"
else:
# Type for ext2/ext3/ext4/btrfs
parted_fs_type = "ext2"
@@ -538,6 +538,17 @@ class PartitionedImage():
(self.path, part.num, part.system_id),
self.native_sysroot)
+ # Parted defaults to enabling the lba flag for fat16 partitions,
+ # which causes compatibility issues with some firmware (and really
+ # isn't necessary).
+ if parted_fs_type == "fat16":
+ if self.ptable_format == 'msdos':
+ logger.debug("Disable 'lba' flag for partition '%s' on disk '%s'",
+ part.num, self.path)
+ exec_native_cmd("parted -s %s set %d lba off" % \
+ (self.path, part.num),
+ self.native_sysroot)
+
def cleanup(self):
# remove partition images
for image in set(self.partimages):
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index 5890c12..11db304 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -44,16 +44,19 @@ class BootimgPcbiosPlugin(SourcePlugin):
name = 'bootimg-pcbios'
@classmethod
- def _get_bootimg_dir(cls, bootimg_dir, dirname):
+ def _get_syslinux_dir(cls, bootimg_dir):
"""
- Check if dirname exists in default bootimg_dir or
- in wic-tools STAGING_DIR.
+ Get path to syslinux from either default bootimg_dir
+ or wic-tools STAGING_DIR.
"""
- for result in (bootimg_dir, get_bitbake_var("STAGING_DATADIR", "wic-tools")):
- if os.path.exists("%s/%s" % (result, dirname)):
- return result
+ for path in (bootimg_dir, get_bitbake_var("STAGING_DATADIR", "wic-tools")):
+ if not path:
+ continue
+ syslinux_dir = os.path.join(path, 'syslinux')
+ if os.path.exists(syslinux_dir):
+ return syslinux_dir
- raise WicError("Couldn't find correct bootimg_dir, exiting")
+ raise WicError("Couldn't find syslinux directory, exiting")
@classmethod
def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
@@ -62,12 +65,11 @@ class BootimgPcbiosPlugin(SourcePlugin):
Called after all partitions have been prepared and assembled into a
disk image. In this case, we install the MBR.
"""
- bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux')
- mbrfile = "%s/syslinux/" % bootimg_dir
+ syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
if creator.ptable_format == 'msdos':
- mbrfile += "mbr.bin"
+ mbrfile = os.path.join(syslinux_dir, "mbr.bin")
elif creator.ptable_format == 'gpt':
- mbrfile += "gptmbr.bin"
+ mbrfile = os.path.join(syslinux_dir, "gptmbr.bin")
else:
raise WicError("Unsupported partition table: %s" %
creator.ptable_format)
@@ -81,8 +83,10 @@ class BootimgPcbiosPlugin(SourcePlugin):
logger.debug("Installing MBR on disk %s as %s with size %s bytes",
disk_name, full_path, disk.min_size)
- dd_cmd = "dd if=%s of=%s conv=notrunc" % (mbrfile, full_path)
- exec_cmd(dd_cmd, native_sysroot)
+ rcode = runner.show(['dd', 'if=%s' % mbrfile,
+ 'of=%s' % full_path, 'conv=notrunc'])
+ if rcode != 0:
+ raise WicError("Unable to set MBR to %s" % full_path)
@classmethod
def do_configure_partition(cls, part, source_params, creator, cr_workdir,
@@ -151,7 +155,7 @@ class BootimgPcbiosPlugin(SourcePlugin):
'prepares' the partition to be incorporated into the image.
In this case, prepare content for legacy bios boot partition.
"""
- bootimg_dir = cls._get_bootimg_dir(bootimg_dir, 'syslinux')
+ syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
staging_kernel_dir = kernel_dir
@@ -159,14 +163,14 @@ class BootimgPcbiosPlugin(SourcePlugin):
cmds = ("install -m 0644 %s/bzImage %s/vmlinuz" %
(staging_kernel_dir, hdddir),
- "install -m 444 %s/syslinux/ldlinux.sys %s/ldlinux.sys" %
- (bootimg_dir, hdddir),
- "install -m 0644 %s/syslinux/vesamenu.c32 %s/vesamenu.c32" %
- (bootimg_dir, hdddir),
- "install -m 444 %s/syslinux/libcom32.c32 %s/libcom32.c32" %
- (bootimg_dir, hdddir),
- "install -m 444 %s/syslinux/libutil.c32 %s/libutil.c32" %
- (bootimg_dir, hdddir))
+ "install -m 444 %s/ldlinux.sys %s/ldlinux.sys" %
+ (syslinux_dir, hdddir),
+ "install -m 0644 %s/vesamenu.c32 %s/vesamenu.c32" %
+ (syslinux_dir, hdddir),
+ "install -m 444 %s/libcom32.c32 %s/libcom32.c32" %
+ (syslinux_dir, hdddir),
+ "install -m 444 %s/libutil.c32 %s/libutil.c32" %
+ (syslinux_dir, hdddir))
for install_cmd in cmds:
exec_cmd(install_cmd)
diff --git a/scripts/lib/wic/plugins/source/fsimage.py b/scripts/lib/wic/plugins/source/fsimage.py
new file mode 100644
index 0000000..f781499
--- /dev/null
+++ b/scripts/lib/wic/plugins/source/fsimage.py
@@ -0,0 +1,56 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+
+import logging
+import os
+
+from wic import WicError
+from wic.pluginbase import SourcePlugin
+from wic.utils.misc import get_bitbake_var
+
+logger = logging.getLogger('wic')
+
+class FSImagePlugin(SourcePlugin):
+ """
+ Add an already existing filesystem image to the partition layout.
+ """
+
+ name = 'fsimage'
+
+ @classmethod
+ def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
+ """
+ Called to do the actual content population for a partition i.e. it
+ 'prepares' the partition to be incorporated into the image.
+ """
+ if not bootimg_dir:
+ bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ if not bootimg_dir:
+ raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+
+ logger.debug('Bootimg dir: %s', bootimg_dir)
+
+ if 'file' not in source_params:
+ raise WicError("No file specified")
+
+ src = os.path.join(bootimg_dir, source_params['file'])
+
+
+ logger.debug('Preparing partition using image %s', src)
+ part.prepare_rootfs_from_fs_image(cr_workdir, src, "")
diff --git a/scripts/lib/wic/utils/misc.py b/scripts/lib/wic/utils/misc.py
index 4609984..c941112 100644
--- a/scripts/lib/wic/utils/misc.py
+++ b/scripts/lib/wic/utils/misc.py
@@ -59,7 +59,7 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
"syslinux": "syslinux"
}
-def _exec_cmd(cmd_and_args, as_shell=False):
+def _exec_cmd(cmd_and_args, as_shell=False, catch=3):
"""
Execute command, catching stderr, stdout
@@ -70,9 +70,9 @@ def _exec_cmd(cmd_and_args, as_shell=False):
logger.debug(args)
if as_shell:
- ret, out = runner.runtool(cmd_and_args)
+ ret, out = runner.runtool(cmd_and_args, catch)
else:
- ret, out = runner.runtool(args)
+ ret, out = runner.runtool(args, catch)
out = out.strip()
if ret != 0:
raise WicError("_exec_cmd: %s returned '%s' instead of 0\noutput: %s" % \
@@ -84,14 +84,14 @@ def _exec_cmd(cmd_and_args, as_shell=False):
return ret, out
-def exec_cmd(cmd_and_args, as_shell=False):
+def exec_cmd(cmd_and_args, as_shell=False, catch=3):
"""
Execute command, return output
"""
- return _exec_cmd(cmd_and_args, as_shell)[1]
+ return _exec_cmd(cmd_and_args, as_shell, catch)[1]
-def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""):
+def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, pseudo=""):
"""
Execute native command, catching stderr, stdout
@@ -118,7 +118,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""):
# If the command isn't in the native sysroot say we failed.
if spawn.find_executable(args[0], native_paths):
- ret, out = _exec_cmd(native_cmd_and_args, True)
+ ret, out = _exec_cmd(native_cmd_and_args, True, catch)
else:
ret = 127
out = "can't find native executable %s in %s" % (args[0], native_paths)
@@ -131,7 +131,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""):
"was not found (see details above).\n\n" % prog
recipe = NATIVE_RECIPES.get(prog)
if recipe:
- msg += "Please make sure wic-tools have %s-native in its DEPENDS, bake it with 'bitbake wic-tools' "\
+ msg += "Please bake it with 'bitbake %s-native' "\
"and try again.\n" % recipe
else:
msg += "Wic failed to find a recipe to build native %s. Please "\
diff --git a/scripts/lib/wic/utils/runner.py b/scripts/lib/wic/utils/runner.py
index 4aa00fb..56d7ea3 100644
--- a/scripts/lib/wic/utils/runner.py
+++ b/scripts/lib/wic/utils/runner.py
@@ -14,17 +14,32 @@
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc., 59
# Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+
+import logging
+import os
import subprocess
from wic import WicError
-def runtool(cmdln_or_args):
+logger = logging.getLogger('wic')
+
+def runtool(cmdln_or_args, catch=1):
""" wrapper for most of the subprocess calls
input:
cmdln_or_args: can be both args and cmdln str (shell=True)
+ catch: 0, quitely run
+ 1, only STDOUT
+ 2, only STDERR
+ 3, both STDOUT and STDERR
return:
- rc, output
+ (rc, output)
+ if catch==0: the output will always None
"""
+
+ if catch not in (0, 1, 2, 3):
+ # invalid catch selection, will cause exception, that's good
+ return None
+
if isinstance(cmdln_or_args, list):
cmd = cmdln_or_args[0]
shell = False
@@ -33,13 +48,26 @@ def runtool(cmdln_or_args):
cmd = shlex.split(cmdln_or_args)[0]
shell = True
- sout = subprocess.PIPE
- serr = subprocess.STDOUT
+ if catch != 3:
+ dev_null = os.open("/dev/null", os.O_WRONLY)
+
+ if catch == 0:
+ sout = dev_null
+ serr = dev_null
+ elif catch == 1:
+ sout = subprocess.PIPE
+ serr = dev_null
+ elif catch == 2:
+ sout = dev_null
+ serr = subprocess.PIPE
+ elif catch == 3:
+ sout = subprocess.PIPE
+ serr = subprocess.STDOUT
try:
process = subprocess.Popen(cmdln_or_args, stdout=sout,
stderr=serr, shell=shell)
- sout, serr = process.communicate()
+ (sout, serr) = process.communicate()
# combine stdout and stderr, filter None out and decode
out = ''.join([out.decode('utf-8') for out in [sout, serr] if out])
except OSError as err:
@@ -48,5 +76,39 @@ def runtool(cmdln_or_args):
raise WicError('Cannot run command: %s, lost dependency?' % cmd)
else:
raise # relay
+ finally:
+ if catch != 3:
+ os.close(dev_null)
+
+ return (process.returncode, out)
+
+def show(cmdln_or_args):
+ """Show all messages using logger.debug."""
+
+ rcode, out = runtool(cmdln_or_args, catch=3)
+
+ if isinstance(cmdln_or_args, list):
+ cmd = ' '.join(cmdln_or_args)
+ else:
+ cmd = cmdln_or_args
+
+ msg = 'running command: "%s"' % cmd
+ if out:
+ out = out.strip()
+ if out:
+ msg += ', with output::'
+ msg += '\n +----------------'
+ for line in out.splitlines():
+ msg += '\n | %s' % line
+ msg += '\n +----------------'
+
+ logger.debug(msg)
+
+ return rcode
+
+def outs(cmdln_or_args, catch=1):
+ # get the outputs of tools
+ return runtool(cmdln_or_args, catch)[1].strip()
- return process.returncode, out
+def quiet(cmdln_or_args):
+ return runtool(cmdln_or_args, catch=0)[0]
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 07/17] Revert "isar-init-build-env: Add /sbin to PATH"
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (5 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 06/17] wic: now truly go for the wic version we claim to have Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 08/17] classes: image: introduce size measuring function, for before do_*_image Henning Schild
` (11 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This reverts commit 19ae5231e28c0930930a799cadb14051cc80b6b0.
We will not run wic on the host any longer.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
isar-init-build-env | 4 ----
1 file changed, 4 deletions(-)
diff --git a/isar-init-build-env b/isar-init-build-env
index 80f6140..8f1d039 100755
--- a/isar-init-build-env
+++ b/isar-init-build-env
@@ -57,8 +57,4 @@ export ISARROOT
}
unset ISARROOT
-# wic executes mkdosfs
-PATH=/sbin:$PATH
-export PATH
-
[ -z "$BUILDDIR" ] || cd "$BUILDDIR"
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 08/17] classes: image: introduce size measuring function, for before do_*_image
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (6 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 07/17] Revert "isar-init-build-env: Add /sbin to PATH" Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:18 ` [PATCH v2 09/17] meta/image: Fix broken variables KERNEL_IMAGE and INITRD_IMAGE Henning Schild
` (10 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
Measure the size and export it to the variable ROOTFS_SIZE just before
the imaging step. The concept is inspired by OE.
Issue:
The size of the rootfs is required for all sort of images, at the moment
Isar only supports one "real" image. Later patches will introduce
another image. And this patch prepares for that by moving the size
measurement into our image base class.
Impact:
- Issue is solved
- nothing changes for users
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
meta/classes/ext4-img.bbclass | 7 ++-----
meta/classes/image.bbclass | 19 +++++++++++++++++++
2 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
index 8a978e6..d19d036 100644
--- a/meta/classes/ext4-img.bbclass
+++ b/meta/classes/ext4-img.bbclass
@@ -1,9 +1,6 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2017 ilbers GmbH
-# Extra space for rootfs in MB
-ROOTFS_EXTRA ?= "64"
-
ROOTFS_TYPE = "ext4"
EXT4_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${MACHINE}.ext4.img"
@@ -14,8 +11,7 @@ do_ext4_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
do_ext4_image() {
rm -f ${EXT4_IMAGE_FILE}
- ROOTFS_SIZE=`sudo du -sm ${IMAGE_ROOTFS} | awk '{print $1 + ${ROOTFS_EXTRA};}'`
- dd if=/dev/zero of=${EXT4_IMAGE_FILE} bs=1M count=${ROOTFS_SIZE}
+ dd if=/dev/zero of=${EXT4_IMAGE_FILE} bs=1k count=${ROOTFS_SIZE}
sudo mkfs.ext4 -F ${EXT4_IMAGE_FILE}
@@ -33,3 +29,4 @@ do_ext4_image() {
}
addtask ext4_image before do_build after do_copy_boot_files
+do_ext4_image[prefuncs] = 'set_image_size'
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index a6537f9..2243860 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -5,6 +5,9 @@ IMAGE_INSTALL ?= ""
IMAGE_TYPE ?= "ext4-img"
IMAGE_ROOTFS = "${WORKDIR}/rootfs"
+# Extra space for rootfs in MB
+ROOTFS_EXTRA ?= "64"
+
def get_image_name(d, name_link):
S = d.getVar("IMAGE_ROOTFS", True)
path_link = os.path.join(S, name_link)
@@ -15,6 +18,22 @@ def get_image_name(d, name_link):
'/'))
return ""
+def get_rootfs_size(d):
+ import subprocess
+ rootfs_extra = int(d.getVar("ROOTFS_EXTRA", True))
+
+ output = subprocess.check_output(['sudo', 'du', '-s', '--block-size=1k',
+ d.getVar("IMAGE_ROOTFS", True)])
+ base_size = int(output.split()[0])
+
+ return base_size + rootfs_extra * 1024
+
+python set_image_size () {
+ rootfs_size = get_rootfs_size(d)
+ d.setVar('ROOTFS_SIZE', str(rootfs_size))
+ d.setVarFlag('ROOTFS_SIZE', 'export', '1')
+}
+
# These variables are used by wic and start_vm
KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')}"
INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')}"
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 09/17] meta/image: Fix broken variables KERNEL_IMAGE and INITRD_IMAGE
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (7 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 08/17] classes: image: introduce size measuring function, for before do_*_image Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-16 10:45 ` Andreas Reichel
2018-04-13 14:18 ` [PATCH v2 10/17] isar-init-build-env: make ISARROOT available in bitbake Henning Schild
` (9 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
Commit 5d15bb11f30c6f581c6 renamed the kernel in initrd in the deploy
folder. While that is correct only one user of those variables was
updated. So wic will still look for a file called exactly what the
variables say.
This commit goes back to the original way, where the variables contain
actual filenames while maintaining the file renaming pattern.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
meta/classes/image.bbclass | 20 ++++++++++++--------
scripts/start_vm | 4 ++--
2 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 2243860..bb2c1e3 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -12,11 +12,15 @@ def get_image_name(d, name_link):
S = d.getVar("IMAGE_ROOTFS", True)
path_link = os.path.join(S, name_link)
if os.path.exists(path_link):
- return os.path.basename(os.path.realpath(path_link))
+ base = os.path.basename(os.path.realpath(path_link))
+ full = base
+ full += "_" + d.getVar("DISTRO", True)
+ full += "-" + d.getVar("MACHINE", True)
+ return [base, full]
if os.path.islink(path_link):
return get_image_name(d, os.path.relpath(os.path.realpath(path_link),
'/'))
- return ""
+ return ["", ""]
def get_rootfs_size(d):
import subprocess
@@ -35,8 +39,8 @@ python set_image_size () {
}
# These variables are used by wic and start_vm
-KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')}"
-INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')}"
+KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')[1]}"
+INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')[1]}"
inherit ${IMAGE_TYPE}
@@ -51,14 +55,14 @@ addtask rootfs before do_build after do_unpack
do_rootfs[deptask] = "do_deploy_deb"
do_copy_boot_files() {
- KERNEL_IMAGE=${@get_image_name(d, 'vmlinuz')}
+ KERNEL_IMAGE=${@get_image_name(d, 'vmlinuz')[1]}
if [ -n "${KERNEL_IMAGE}" ]; then
- cp -f ${IMAGE_ROOTFS}/boot/${KERNEL_IMAGE} ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE}_${DISTRO}-${MACHINE}
+ cp -f ${IMAGE_ROOTFS}/boot/${@get_image_name(d, 'vmlinuz')[0]} ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE}
fi
- INITRD_IMAGE=${@get_image_name(d, 'initrd.img')}
+ INITRD_IMAGE=${@get_image_name(d, 'initrd.img')[1]}
if [ -n "${INITRD_IMAGE}" ]; then
- sudo cp -f ${IMAGE_ROOTFS}/boot/${INITRD_IMAGE} ${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE}_${DISTRO}-${MACHINE}
+ sudo cp -f ${IMAGE_ROOTFS}/boot/${@get_image_name(d, 'initrd.img')[0]} ${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE}
fi
}
diff --git a/scripts/start_vm b/scripts/start_vm
index 26274be..6c89ce9 100755
--- a/scripts/start_vm
+++ b/scripts/start_vm
@@ -108,9 +108,9 @@ readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=")
eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_IMAGE=")
-QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}_debian-$DISTRO-qemu$ARCH
+QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
QINITRD=/dev/null
-[ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}_debian-$DISTRO-qemu$ARCH
+[ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
readonly ISARROOT="$(dirname "$0")"/..
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 09/17] meta/image: Fix broken variables KERNEL_IMAGE and INITRD_IMAGE
2018-04-13 14:18 ` [PATCH v2 09/17] meta/image: Fix broken variables KERNEL_IMAGE and INITRD_IMAGE Henning Schild
@ 2018-04-16 10:45 ` Andreas Reichel
0 siblings, 0 replies; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:45 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Fri, Apr 13, 2018 at 04:18:58PM +0200, Henning Schild wrote:
> Commit 5d15bb11f30c6f581c6 renamed the kernel in initrd in the deploy
kernel AND initrd
> folder. While that is correct only one user of those variables was
> updated. So wic will still look for a file called exactly what the
> variables say.
>
> This commit goes back to the original way, where the variables contain
> actual filenames while maintaining the file renaming pattern.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> meta/classes/image.bbclass | 20 ++++++++++++--------
> scripts/start_vm | 4 ++--
> 2 files changed, 14 insertions(+), 10 deletions(-)
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 2243860..bb2c1e3 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -12,11 +12,15 @@ def get_image_name(d, name_link):
> S = d.getVar("IMAGE_ROOTFS", True)
> path_link = os.path.join(S, name_link)
> if os.path.exists(path_link):
> - return os.path.basename(os.path.realpath(path_link))
> + base = os.path.basename(os.path.realpath(path_link))
> + full = base
> + full += "_" + d.getVar("DISTRO", True)
> + full += "-" + d.getVar("MACHINE", True)
> + return [base, full]
> if os.path.islink(path_link):
> return get_image_name(d, os.path.relpath(os.path.realpath(path_link),
> '/'))
> - return ""
> + return ["", ""]
>
> def get_rootfs_size(d):
> import subprocess
> @@ -35,8 +39,8 @@ python set_image_size () {
> }
>
> # These variables are used by wic and start_vm
> -KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')}"
> -INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')}"
> +KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')[1]}"
> +INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')[1]}"
>
> inherit ${IMAGE_TYPE}
>
> @@ -51,14 +55,14 @@ addtask rootfs before do_build after do_unpack
> do_rootfs[deptask] = "do_deploy_deb"
>
> do_copy_boot_files() {
> - KERNEL_IMAGE=${@get_image_name(d, 'vmlinuz')}
> + KERNEL_IMAGE=${@get_image_name(d, 'vmlinuz')[1]}
> if [ -n "${KERNEL_IMAGE}" ]; then
> - cp -f ${IMAGE_ROOTFS}/boot/${KERNEL_IMAGE} ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE}_${DISTRO}-${MACHINE}
> + cp -f ${IMAGE_ROOTFS}/boot/${@get_image_name(d, 'vmlinuz')[0]} ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE}
> fi
>
> - INITRD_IMAGE=${@get_image_name(d, 'initrd.img')}
> + INITRD_IMAGE=${@get_image_name(d, 'initrd.img')[1]}
> if [ -n "${INITRD_IMAGE}" ]; then
> - sudo cp -f ${IMAGE_ROOTFS}/boot/${INITRD_IMAGE} ${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE}_${DISTRO}-${MACHINE}
> + sudo cp -f ${IMAGE_ROOTFS}/boot/${@get_image_name(d, 'initrd.img')[0]} ${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE}
> fi
> }
>
> diff --git a/scripts/start_vm b/scripts/start_vm
> index 26274be..6c89ce9 100755
> --- a/scripts/start_vm
> +++ b/scripts/start_vm
> @@ -108,9 +108,9 @@ readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
>
> eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=")
> eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_IMAGE=")
> -QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}_debian-$DISTRO-qemu$ARCH
> +QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> QINITRD=/dev/null
> -[ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}_debian-$DISTRO-qemu$ARCH
> +[ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
>
> readonly ISARROOT="$(dirname "$0")"/..
>
> --
> 2.16.1
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 10/17] isar-init-build-env: make ISARROOT available in bitbake
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (8 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 09/17] meta/image: Fix broken variables KERNEL_IMAGE and INITRD_IMAGE Henning Schild
@ 2018-04-13 14:18 ` Henning Schild
2018-04-13 14:19 ` [PATCH v2 11/17] images: New class wic-img for wic intregration Henning Schild
` (8 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:18 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This will be used by a later patch where we bind-mount scripts/ into the
buildchroot to execute wic there.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
isar-init-build-env | 1 -
scripts/isar-buildenv-internal | 2 +-
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/isar-init-build-env b/isar-init-build-env
index 8f1d039..e19d984 100755
--- a/isar-init-build-env
+++ b/isar-init-build-env
@@ -55,6 +55,5 @@ export ISARROOT
unset ISARROOT
return 1
}
-unset ISARROOT
[ -z "$BUILDDIR" ] || cd "$BUILDDIR"
diff --git a/scripts/isar-buildenv-internal b/scripts/isar-buildenv-internal
index e1f7149..2476d90 100755
--- a/scripts/isar-buildenv-internal
+++ b/scripts/isar-buildenv-internal
@@ -66,5 +66,5 @@ export PATH
BBPATH="${BUILDDIR}"
export BBPATH
-BB_ENV_EXTRAWHITE="BUILDDIR http_proxy https_proxy ftp_proxy no_proxy"
+BB_ENV_EXTRAWHITE="BUILDDIR ISARROOT http_proxy https_proxy ftp_proxy no_proxy"
export BB_ENV_EXTRAWHITE
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 11/17] images: New class wic-img for wic intregration
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (9 preceding siblings ...)
2018-04-13 14:18 ` [PATCH v2 10/17] isar-init-build-env: make ISARROOT available in bitbake Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-13 14:44 ` Henning Schild
2018-04-13 14:19 ` [PATCH v2 12/17] wic: Add pcibios boot plugins and wks files Henning Schild
` (7 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This patch integrates wic into the bitbake workflow, wic will be used
for the imaging step, no need to call it manually.
After all the previous reverts we now use an unmodified version of wic.
Issues:
- wic was never integrated
- you always had to build an ext4-img to create a wic image later
- there was never a way to control the size of wic disks/partition,
only directly in the wks
- wic used to leak the hosts bootloader into the final image
Impact:
The patch solves the Issues, but drops the ability to manually start
wic after bitbake. And it drops support for building wic images for
Distros before stretch.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
doc/user_manual.md | 6 -
.../scripts/lib/wic/canned-wks/sdimage-efi.wks | 2 +-
.../lib/wic/plugins/source/bootimg-efi-isar.py | 297 +++++++++++++++++++++
meta/classes/wic-img.bbclass | 78 ++++++
meta/recipes-devtools/buildchroot/buildchroot.bb | 19 ++
scripts/wic_fakeroot | 37 +++
6 files changed, 432 insertions(+), 7 deletions(-)
create mode 100644 meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
create mode 100644 meta/classes/wic-img.bbclass
create mode 100755 scripts/wic_fakeroot
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 058f7bd..7bd52f4 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -52,16 +52,10 @@ The steps below describe how to build the images provided by default.
Install the following packages:
```
dosfstools
-e2fsprogs/jessie-backports # wic: e2fsprogs -d
-gdisk # wic
git
-grub-efi-amd64-bin # wic UEFI: /usr/lib/grub/x86_64-efi/moddep.lst
-grub-efi-ia32-bin # wic UEFI: /usr/lib/grub/i386-efi/moddep.lst
-mtools # wic FAT: mcopy
debootstrap
parted
python
-python3 # wic
qemu
qemu-user-static
rxvt-unicode # build_parallel
diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
index 1171a26..580ad21 100644
--- a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
+++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi.wks
@@ -2,7 +2,7 @@
# long-description: Creates a partitioned EFI disk image without any swap that
# the user can directly dd to boot media.
-part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
+part /boot --source bootimg-efi-isar --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
diff --git a/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py b/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
new file mode 100644
index 0000000..fccf96c
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
@@ -0,0 +1,297 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2014, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This implements the 'bootimg-efi-isar' source plugin class for 'wic'
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+import logging
+import os
+import shutil
+
+from wic import WicError
+from wic.engine import get_custom_config
+from wic.pluginbase import SourcePlugin
+from wic.utils.misc import (exec_cmd, get_bitbake_var, BOOTDD_EXTRA_SPACE)
+
+logger = logging.getLogger('wic')
+
+class BootimgEFIPlugin(SourcePlugin):
+ """
+ Create EFI boot partition.
+ This plugin supports GRUB 2 and systemd-boot bootloaders.
+ """
+
+ name = 'bootimg-efi-isar'
+
+ @classmethod
+ def do_configure_grubefi(cls, creator, cr_workdir):
+ """
+ Create loader-specific (grub-efi) config
+ """
+ configfile = creator.ks.bootloader.configfile
+ custom_cfg = None
+ if configfile:
+ custom_cfg = get_custom_config(configfile)
+ if custom_cfg:
+ # Use a custom configuration for grub
+ grubefi_conf = custom_cfg
+ logger.debug("Using custom configuration file "
+ "%s for grub.cfg", configfile)
+ else:
+ raise WicError("configfile is specified but failed to "
+ "get it from %s." % configfile)
+
+ if not custom_cfg:
+ # Create grub configuration using parameters from wks file
+ bootloader = creator.ks.bootloader
+
+ grubefi_conf = ""
+ grubefi_conf += "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1\n"
+ grubefi_conf += "default=boot\n"
+ grubefi_conf += "timeout=%s\n" % bootloader.timeout
+ grubefi_conf += "set root='hd0,gpt2'\n"
+ grubefi_conf += "menuentry 'boot'{\n"
+
+ kernel = "/vmlinuz"
+
+ grubefi_conf += "linux %s root=%s rootwait %s\n" \
+ % (kernel, creator.rootdev, bootloader.append)
+
+ initrd = "/initrd.img"
+
+ grubefi_conf += "initrd %s\n" % initrd
+
+ grubefi_conf += "}\n"
+
+ logger.debug("Writing grubefi config %s/hdd/boot/EFI/BOOT/grub.cfg",
+ cr_workdir)
+ cfg = open("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir, "w")
+ cfg.write(grubefi_conf)
+ cfg.close()
+
+ cfg = open("%s/hdd/boot/EFI/BOOT/grub-mkimage.cfg" % cr_workdir, "w")
+ mkimage_conf = "set root='hd0,gpt1'\n"
+ mkimage_conf += "set prefix=($root)/EFI/BOOT\n"
+ cfg.write(mkimage_conf)
+ cfg.close()
+
+ @classmethod
+ def do_configure_systemdboot(cls, hdddir, creator, cr_workdir, source_params):
+ """
+ Create loader-specific systemd-boot/gummiboot config
+ """
+ install_cmd = "install -d %s/loader" % hdddir
+ exec_cmd(install_cmd)
+
+ install_cmd = "install -d %s/loader/entries" % hdddir
+ exec_cmd(install_cmd)
+
+ bootloader = creator.ks.bootloader
+
+ loader_conf = ""
+ loader_conf += "default boot\n"
+ loader_conf += "timeout %d\n" % bootloader.timeout
+
+ initrd = source_params.get('initrd')
+
+ if initrd:
+ # obviously we need to have a common common deploy var
+ bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ if not bootimg_dir:
+ raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+
+ cp_cmd = "cp %s/%s %s" % (bootimg_dir, initrd, hdddir)
+ exec_cmd(cp_cmd, True)
+ else:
+ logger.debug("Ignoring missing initrd")
+
+ logger.debug("Writing systemd-boot config "
+ "%s/hdd/boot/loader/loader.conf", cr_workdir)
+ cfg = open("%s/hdd/boot/loader/loader.conf" % cr_workdir, "w")
+ cfg.write(loader_conf)
+ cfg.close()
+
+ configfile = creator.ks.bootloader.configfile
+ custom_cfg = None
+ if configfile:
+ custom_cfg = get_custom_config(configfile)
+ if custom_cfg:
+ # Use a custom configuration for systemd-boot
+ boot_conf = custom_cfg
+ logger.debug("Using custom configuration file "
+ "%s for systemd-boots's boot.conf", configfile)
+ else:
+ raise WicError("configfile is specified but failed to "
+ "get it from %s.", configfile)
+
+ if not custom_cfg:
+ # Create systemd-boot configuration using parameters from wks file
+ kernel = "/vmlinuz"
+
+ boot_conf = ""
+ boot_conf += "title boot\n"
+ boot_conf += "linux %s\n" % kernel
+ boot_conf += "options LABEL=Boot root=%s %s\n" % \
+ (creator.rootdev, bootloader.append)
+
+ if initrd:
+ boot_conf += "initrd /%s\n" % initrd
+
+ logger.debug("Writing systemd-boot config "
+ "%s/hdd/boot/loader/entries/boot.conf", cr_workdir)
+ cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w")
+ cfg.write(boot_conf)
+ cfg.close()
+
+
+ @classmethod
+ def do_configure_partition(cls, part, source_params, creator, cr_workdir,
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
+ """
+ Called before do_prepare_partition(), creates loader-specific config
+ """
+ hdddir = "%s/hdd/boot" % cr_workdir
+
+ install_cmd = "install -d %s/EFI/BOOT" % hdddir
+ exec_cmd(install_cmd)
+
+ try:
+ if source_params['loader'] == 'grub-efi':
+ cls.do_configure_grubefi(creator, cr_workdir)
+ elif source_params['loader'] == 'systemd-boot':
+ cls.do_configure_systemdboot(hdddir, creator, cr_workdir, source_params)
+ else:
+ raise WicError("unrecognized bootimg-efi-isar loader: %s" % source_params['loader'])
+ except KeyError:
+ raise WicError("bootimg-efi-isar requires a loader, none specified")
+
+
+ @classmethod
+ def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
+ """
+ Called to do the actual content population for a partition i.e. it
+ 'prepares' the partition to be incorporated into the image.
+ In this case, prepare content for an EFI (grub) boot partition.
+ """
+ if not kernel_dir:
+ kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ if not kernel_dir:
+ raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+
+ staging_kernel_dir = kernel_dir
+
+ hdddir = "%s/hdd/boot" % cr_workdir
+
+ try:
+ if source_params['loader'] == 'grub-efi':
+ shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir,
+ "%s/grub.cfg" % cr_workdir)
+ shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub-mkimage.cfg" % cr_workdir,
+ "%s/grub-mkimage.cfg" % cr_workdir)
+ for mod in [x for x in os.listdir(kernel_dir) if x.startswith("grub-efi-")]:
+ cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[9:])
+ exec_cmd(cp_cmd, True)
+ shutil.move("%s/grub.cfg" % cr_workdir,
+ "%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir)
+
+ distro_arch = get_bitbake_var("DISTRO_ARCH")
+ if not distro_arch:
+ raise WicError("Couldn't find target architecture")
+
+ if distro_arch == "amd64":
+ grub_target = 'x86_64-efi'
+ grub_image = "bootx64.efi"
+ elif distro_arch == "i386":
+ grub_target = 'i386-efi'
+ grub_image = "bootia32.efi"
+ else:
+ raise WicError("grub-efi is incompatible with target %s" %
+ distro_arch)
+
+ bootimg_dir = "%s/hdd/boot" % cr_workdir
+ if not os.path.isfile("%s/EFI/BOOT/%s" \
+ % (bootimg_dir, grub_image)):
+
+ # TODO: check that grub-mkimage is available
+ grub_cmd = "grub-mkimage -p /EFI/BOOT "
+ grub_cmd += "-c %s/grub-mkimage.cfg " % cr_workdir
+ grub_cmd += "-O %s -o %s/EFI/BOOT/%s " \
+ % (grub_target, bootimg_dir, grub_image)
+ grub_cmd += "part_gpt part_msdos ntfs ntfscomp fat ext2 "
+ grub_cmd += "normal chain boot configfile linux multiboot "
+ grub_cmd += "search efi_gop efi_uga font gfxterm gfxmenu "
+ grub_cmd += "terminal minicmd test iorw loadenv echo help "
+ grub_cmd += "reboot serial terminfo iso9660 loopback tar "
+ grub_cmd += "memdisk ls search_fs_uuid udf btrfs xfs lvm "
+ grub_cmd += "reiserfs ata "
+ exec_cmd(grub_cmd)
+ elif source_params['loader'] == 'systemd-boot':
+ for mod in [x for x in os.listdir(kernel_dir) if x.startswith("systemd-")]:
+ cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, hdddir, mod[8:])
+ exec_cmd(cp_cmd, True)
+ else:
+ raise WicError("unrecognized bootimg-efi-isar loader: %s" %
+ source_params['loader'])
+ except KeyError:
+ raise WicError("bootimg-efi-isar requires a loader, none specified")
+
+ startup = os.path.join(kernel_dir, "startup.nsh")
+ if os.path.exists(startup):
+ cp_cmd = "cp %s %s/" % (startup, hdddir)
+ exec_cmd(cp_cmd, True)
+
+ du_cmd = "du -bks %s" % hdddir
+ out = exec_cmd(du_cmd)
+ blocks = int(out.split()[0])
+
+ extra_blocks = part.get_extra_block_count(blocks)
+
+ if extra_blocks < BOOTDD_EXTRA_SPACE:
+ extra_blocks = BOOTDD_EXTRA_SPACE
+
+ blocks += extra_blocks
+
+ logger.debug("Added %d extra blocks to %s to get to %d total blocks",
+ extra_blocks, part.mountpoint, blocks)
+
+ # dosfs image, created by mkdosfs
+ bootimg = "%s/boot.img" % cr_workdir
+
+ dosfs_cmd = "mkdosfs -n efi -C %s %d" % (bootimg, blocks)
+ exec_cmd(dosfs_cmd)
+
+ mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
+ exec_cmd(mcopy_cmd, True)
+
+ chmod_cmd = "chmod 644 %s" % bootimg
+ exec_cmd(chmod_cmd)
+
+ du_cmd = "du -Lbks %s" % bootimg
+ out = exec_cmd(du_cmd)
+ bootimg_size = out.split()[0]
+
+ part.size = int(bootimg_size)
+ part.source_file = bootimg
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
new file mode 100644
index 0000000..d1deff3
--- /dev/null
+++ b/meta/classes/wic-img.bbclass
@@ -0,0 +1,78 @@
+# This software is a part of ISAR.
+# Copyright (C) 2018 Siemens AG
+#
+# this class is heavily inspired by OEs ./meta/classes/image_types_wic.bbclass
+#
+
+WKS_FILE ?= "sdimage-efi"
+ROOTFS_TYPE ?= "ext4"
+
+STAGING_DATADIR ?= "/usr/lib/"
+STAGING_LIBDIR ?= "/usr/lib/"
+STAGING_DIR ?= "${TMPDIR}"
+IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}"
+FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
+RECIPE_SYSROOT_NATIVE ?= "/"
+
+do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+
+WIC_CREATE_EXTRA_ARGS ?= ""
+
+WICVARS ?= "\
+ BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD IMAGE_BASENAME IMAGE_BOOT_FILES \
+ IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
+ ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS TRANSLATED_TARGET_ARCH"
+
+# Isar specific vars used in our plugins
+WICVARS += "KERNEL_IMAGE INITRD_IMAGE DISTRO_ARCH"
+
+python do_rootfs_wicenv () {
+ wicvars = d.getVar('WICVARS', True)
+ if not wicvars:
+ return
+
+ stdir = d.getVar('STAGING_DIR', True)
+ outdir = os.path.join(stdir, d.getVar('MACHINE', True), 'imgdata')
+ bb.utils.mkdirhier(outdir)
+ basename = d.getVar('IMAGE_BASENAME', True)
+ with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
+ for var in wicvars.split():
+ value = d.getVar(var, True)
+ if value:
+ envf.write('%s="%s"\n' % (var, value.strip()))
+
+ # this part is stolen from OE ./meta/recipes-core/meta/wic-tools.bb
+ with open(os.path.join(outdir, "wic-tools.env"), 'w') as envf:
+ for var in ('RECIPE_SYSROOT_NATIVE', 'STAGING_DATADIR', 'STAGING_LIBDIR'):
+ envf.write('%s="%s"\n' % (var, d.getVar(var, True).strip()))
+
+}
+
+addtask do_rootfs_wicenv after do_copy_boot_files before do_wic_image
+do_rootfs_wicenv[vardeps] += "${WICVARS}"
+do_rootfs_wicenv[prefuncs] = 'set_image_size'
+
+WIC_IMAGE_FILE ="${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${MACHINE}.wic.img"
+
+do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+
+do_wic_image() {
+ if ! grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts; then
+ sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev
+ sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
+ fi
+ for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts; do
+ sudo mkdir -p ${BUILDCHROOT_DIR}/$dir
+ sudo mount --bind $dir ${BUILDCHROOT_DIR}/$dir
+ done
+ export FAKEROOTCMD=${FAKEROOTCMD}
+ export BUILDDIR=${BUILDDIR}
+ export MTOOLS_SKIP_CHECK=1
+
+ sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
+ cp -f `ls -t -1 ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct | head -1` ${WIC_IMAGE_FILE}
+}
+
+do_wic_image[depends] = "buildchroot:do_build"
+
+addtask wic_image before do_build after do_copy_boot_files
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index b16e63a..bd4d003 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -28,6 +28,25 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
devscripts \
equivs"
+BUILDCHROOT_PREINSTALL_WIC = " \
+ parted \
+ gdisk \
+ util-linux \
+ syslinux \
+ syslinux-common \
+ dosfstools \
+ mtools \
+ e2fsprogs \
+ grub-efi-amd64-bin \
+ grub-efi-ia32-bin \
+ python3"
+
+python () {
+ if d.getVar('IMAGE_TYPE', True) == 'wic-img':
+ d.appendVar('BUILDCHROOT_PREINSTALL',
+ d.getVar('BUILDCHROOT_PREINSTALL_WIC', True))
+}
+
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
new file mode 100755
index 0000000..9e01c38
--- /dev/null
+++ b/scripts/wic_fakeroot
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+#
+# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar we do/can not
+# use pseudo. And we call wic as root to begin with, so this script could be a
+# dummy doing nothing. It is almost a dummy ...
+#
+# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" can be used
+#
+# This software is a part of Isar.
+# Copyright (C) 2018 Siemens AG
+#
+import os
+import sys
+import shutil
+import subprocess
+
+args = sys.argv
+args.pop(0)
+cmd = args[0]
+
+# expect to be running as root
+# we could loosen that and execv(sudo, args) but even some early
+# "du"s fail, which do not use the fakeroot-wrapper
+# i.e. in wics partition.py the "du -ks" fails on
+# var/cache/apt/archives/partial
+# rootfs/root ...
+assert 'root' == os.environ["USER"]
+
+# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before affected)
+# treat 1 as safe ... the filesystem was successfully repaired and is OK
+if cmd.startswith('fsck.'):
+ ret = subprocess.call(args)
+ if ret == 0 or ret == 1:
+ sys.exit(0)
+ sys.exit(ret)
+
+os.execv(shutil.which(cmd), args)
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 11/17] images: New class wic-img for wic intregration
2018-04-13 14:19 ` [PATCH v2 11/17] images: New class wic-img for wic intregration Henning Schild
@ 2018-04-13 14:44 ` Henning Schild
2018-04-16 10:17 ` Claudius Heine
0 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:44 UTC (permalink / raw)
To: isar-users
So this is the patch that does the real magic, unfortunately i have to
call it that ...
Am Fri, 13 Apr 2018 16:19:00 +0200
schrieb Henning Schild <henning.schild@siemens.com>:
> This patch integrates wic into the bitbake workflow, wic will be used
> for the imaging step, no need to call it manually.
> After all the previous reverts we now use an unmodified version of
> wic.
>
> Issues:
> - wic was never integrated
> - you always had to build an ext4-img to create a wic image later
> - there was never a way to control the size of wic disks/partition,
> only directly in the wks
> - wic used to leak the hosts bootloader into the final image
>
> Impact:
> The patch solves the Issues, but drops the ability to manually start
> wic after bitbake. And it drops support for building wic images for
> Distros before stretch.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> doc/user_manual.md | 6 -
> .../scripts/lib/wic/canned-wks/sdimage-efi.wks | 2 +-
> .../lib/wic/plugins/source/bootimg-efi-isar.py | 297
> +++++++++++++++++++++
> meta/classes/wic-img.bbclass | 78 ++++++
> meta/recipes-devtools/buildchroot/buildchroot.bb | 19 ++
> scripts/wic_fakeroot | 37 +++ 6 files
> changed, 432 insertions(+), 7 deletions(-) create mode 100644
> meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py create
> mode 100644 meta/classes/wic-img.bbclass create mode 100755
> scripts/wic_fakeroot
>
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 058f7bd..7bd52f4 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -52,16 +52,10 @@ The steps below describe how to build the images
> provided by default. Install the following packages:
> ```
> dosfstools
> -e2fsprogs/jessie-backports # wic: e2fsprogs -d
> -gdisk # wic
> git
> -grub-efi-amd64-bin # wic
> UEFI: /usr/lib/grub/x86_64-efi/moddep.lst
> -grub-efi-ia32-bin # wic
> UEFI: /usr/lib/grub/i386-efi/moddep.lst -mtools
> # wic FAT: mcopy debootstrap parted
> python
> -python3 # wic
> qemu
> qemu-user-static
> rxvt-unicode # build_parallel
We are now running wic in buildchroot and do not need all that anymore.
wic uses a set of tools for partitioning etc. and it has dependencies
on versions of these tools. In OE people know those and build a native
sysroot. But we have debian tools that have slighly different versions.
That is why we carry a few workarounds to make wic still like our tools,
i will point them out later.
In OE the native sysroot contains tools that are all statically linked
so you can execute them without a problem. In Isar we can just do that
with the host tools. Things we have in buildchroot or the image rootfs
can only be used when chrooting in.
So why not keep all that and use the host tools? Things like mkfs or
parted are fine, but bootloader installers leak host information into
our images. So we would build a wheezy with a jessie grub, not the best
idea.
So i decided to run wic in buildchroot instead of the host. It also
shifts the whole sudo on the host discussion.
Running wic in buildchroot solves several problems but introduces
another "interesting" aspect. When doing that with wheezy or jessie you
are beaming wic so far into the past that the few tool workarounds will
not work anymore. The tools are too old and even do not support certain
command line switches. One example we already had for jessie:
> -e2fsprogs/jessie-backports # wic: e2fsprogs -d
So the pragmatic decision was to simply not support distros that
predate wic itself. It is unlikely we will have to deal with such
problems looking in the future. Because the new tools in newer distros
are not likely to be backward incompatible.
> a/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
> b/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py new
...
> file mode 100644 index 0000000..fccf96c --- /dev/null
> + grubefi_conf += "menuentry 'boot'{\n"
> +
> + kernel = "/vmlinuz"
I did not find a better way than to fork the plugins. But they need to
be slightly different to what we get from OE. One example is the name
of the kernel that is hardcoded there.
But the diffs are small and should not be hard to maintain.
> diff --git a/meta/classes/wic-img.bbclass
> b/meta/classes/wic-img.bbclass new file mode 100644
> index 0000000..d1deff3
> --- /dev/null
> +++ b/meta/classes/wic-img.bbclass
> @@ -0,0 +1,78 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2018 Siemens AG
> +#
> +# this class is heavily inspired by
> OEs ./meta/classes/image_types_wic.bbclass +#
> +
> +WKS_FILE ?= "sdimage-efi"
> +ROOTFS_TYPE ?= "ext4"
> +
> +STAGING_DATADIR ?= "/usr/lib/"
> +STAGING_LIBDIR ?= "/usr/lib/"
> +STAGING_DIR ?= "${TMPDIR}"
> +IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}"
> +FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
> +RECIPE_SYSROOT_NATIVE ?= "/"
> +
> +do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> +
> +WIC_CREATE_EXTRA_ARGS ?= ""
> +
> +WICVARS ?= "\
> + BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> IMAGE_BASENAME IMAGE_BOOT_FILES \
> + IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
> + ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> TARGET_SYS TRANSLATED_TARGET_ARCH" +
> +# Isar specific vars used in our plugins
> +WICVARS += "KERNEL_IMAGE INITRD_IMAGE DISTRO_ARCH"
> +
> +python do_rootfs_wicenv () {
> + wicvars = d.getVar('WICVARS', True)
> + if not wicvars:
> + return
> +
> + stdir = d.getVar('STAGING_DIR', True)
> + outdir = os.path.join(stdir, d.getVar('MACHINE', True),
> 'imgdata')
> + bb.utils.mkdirhier(outdir)
> + basename = d.getVar('IMAGE_BASENAME', True)
> + with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> + for var in wicvars.split():
> + value = d.getVar(var, True)
> + if value:
> + envf.write('%s="%s"\n' % (var, value.strip()))
> +
> + # this part is stolen from
> OE ./meta/recipes-core/meta/wic-tools.bb
> + with open(os.path.join(outdir, "wic-tools.env"), 'w') as envf:
> + for var in ('RECIPE_SYSROOT_NATIVE', 'STAGING_DATADIR',
> 'STAGING_LIBDIR'):
> + envf.write('%s="%s"\n' % (var, d.getVar(var,
> True).strip())) +
> +}
> +
> +addtask do_rootfs_wicenv after do_copy_boot_files before do_wic_image
> +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> +
> +WIC_IMAGE_FILE
> ="${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${MACHINE}.wic.img" +
> +do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +
> +do_wic_image() {
> + if ! grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts; then
> + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs
> ${BUILDCHROOT_DIR}/dev
> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
> + fi
> + for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts; do
> + sudo mkdir -p ${BUILDCHROOT_DIR}/$dir
> + sudo mount --bind $dir ${BUILDCHROOT_DIR}/$dir
> + done
> + export FAKEROOTCMD=${FAKEROOTCMD}
> + export BUILDDIR=${BUILDDIR}
> + export MTOOLS_SKIP_CHECK=1
This is one of the tool workarounds, where i reverted a patch against
wic.
> +
> + sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create
> ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e
> ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
> + cp -f `ls -t -1 ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct |
> head -1` ${WIC_IMAGE_FILE} +}
> +
> +do_wic_image[depends] = "buildchroot:do_build"
> +
> +addtask wic_image before do_build after do_copy_boot_files
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
> b/meta/recipes-devtools/buildchroot/buildchroot.bb index
> b16e63a..bd4d003 100644 ---
> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++
> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -28,6 +28,25 @@
> BUILDCHROOT_PREINSTALL ?= "gcc \ devscripts \
> equivs"
>
> +BUILDCHROOT_PREINSTALL_WIC = " \
> + parted \
> + gdisk \
> + util-linux \
> + syslinux \
> + syslinux-common \
> + dosfstools \
> + mtools \
> + e2fsprogs \
> + grub-efi-amd64-bin \
> + grub-efi-ia32-bin \
> + python3"
> +
> +python () {
> + if d.getVar('IMAGE_TYPE', True) == 'wic-img':
> + d.appendVar('BUILDCHROOT_PREINSTALL',
> + d.getVar('BUILDCHROOT_PREINSTALL_WIC', True))
> +}
Now the wic tools become part of buildchroot if you chose that
IMAGE_TYPE.
> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>
> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
> new file mode 100755
> index 0000000..9e01c38
> --- /dev/null
> +++ b/scripts/wic_fakeroot
> @@ -0,0 +1,37 @@
> +#!/usr/bin/env python3
> +#
> +# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar we
> do/can not +# use pseudo. And we call wic as root to begin with, so
> this script could be a +# dummy doing nothing. It is almost a
> dummy ... +#
> +# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" can
> be used +#
> +# This software is a part of Isar.
> +# Copyright (C) 2018 Siemens AG
> +#
> +import os
> +import sys
> +import shutil
> +import subprocess
> +
> +args = sys.argv
> +args.pop(0)
> +cmd = args[0]
> +
> +# expect to be running as root
> +# we could loosen that and execv(sudo, args) but even some early
> +# "du"s fail, which do not use the fakeroot-wrapper
> +# i.e. in wics partition.py the "du -ks" fails on
> +# var/cache/apt/archives/partial
> +# rootfs/root ...
> +assert 'root' == os.environ["USER"]
> +
> +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
> affected) +# treat 1 as safe ... the filesystem was successfully
> repaired and is OK +if cmd.startswith('fsck.'):
> + ret = subprocess.call(args)
> + if ret == 0 or ret == 1:
> + sys.exit(0)
> + sys.exit(ret)
This is another tool workaround, now outside of wic.
> +
> +os.execv(shutil.which(cmd), args)
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 11/17] images: New class wic-img for wic intregration
2018-04-13 14:44 ` Henning Schild
@ 2018-04-16 10:17 ` Claudius Heine
2018-04-16 10:25 ` Henning Schild
2018-04-16 10:31 ` Andreas Reichel
0 siblings, 2 replies; 47+ messages in thread
From: Claudius Heine @ 2018-04-16 10:17 UTC (permalink / raw)
To: [ext] Henning Schild, isar-users
Hi Henning,
first of, great work getting wic functional!
On 2018-04-13 16:44, [ext] Henning Schild wrote:
> So this is the patch that does the real magic, unfortunately i have to
> call it that ...
>
> Am Fri, 13 Apr 2018 16:19:00 +0200
> schrieb Henning Schild <henning.schild@siemens.com>:
>
>> This patch integrates wic into the bitbake workflow, wic will be used
>> for the imaging step, no need to call it manually.
>> After all the previous reverts we now use an unmodified version of
>> wic.
>>
>> Issues:
>> - wic was never integrated
>> - you always had to build an ext4-img to create a wic image later
>> - there was never a way to control the size of wic disks/partition,
>> only directly in the wks
>> - wic used to leak the hosts bootloader into the final image
>>
>> Impact:
>> The patch solves the Issues, but drops the ability to manually start
>> wic after bitbake. And it drops support for building wic images for
>> Distros before stretch.
>>
>> Signed-off-by: Henning Schild <henning.schild@siemens.com>
>> ---
>> doc/user_manual.md | 6 -
>> .../scripts/lib/wic/canned-wks/sdimage-efi.wks | 2 +-
>> .../lib/wic/plugins/source/bootimg-efi-isar.py | 297
>> +++++++++++++++++++++
>> meta/classes/wic-img.bbclass | 78 ++++++
>> meta/recipes-devtools/buildchroot/buildchroot.bb | 19 ++
>> scripts/wic_fakeroot | 37 +++ 6 files
>> changed, 432 insertions(+), 7 deletions(-) create mode 100644
>> meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py create
>> mode 100644 meta/classes/wic-img.bbclass create mode 100755
>> scripts/wic_fakeroot
>>
>> diff --git a/doc/user_manual.md b/doc/user_manual.md
>> index 058f7bd..7bd52f4 100644
>> --- a/doc/user_manual.md
>> +++ b/doc/user_manual.md
>> @@ -52,16 +52,10 @@ The steps below describe how to build the images
>> provided by default. Install the following packages:
>> ```
>> dosfstools
>> -e2fsprogs/jessie-backports # wic: e2fsprogs -d
>> -gdisk # wic
>> git
>> -grub-efi-amd64-bin # wic
>> UEFI: /usr/lib/grub/x86_64-efi/moddep.lst
>> -grub-efi-ia32-bin # wic
>> UEFI: /usr/lib/grub/i386-efi/moddep.lst -mtools
>> # wic FAT: mcopy debootstrap parted
>> python
>> -python3 # wic
>> qemu
>> qemu-user-static
>> rxvt-unicode # build_parallel
>
> We are now running wic in buildchroot and do not need all that anymore.
>
> wic uses a set of tools for partitioning etc. and it has dependencies
> on versions of these tools. In OE people know those and build a native
> sysroot. But we have debian tools that have slighly different versions.
>
> That is why we carry a few workarounds to make wic still like our tools,
> i will point them out later.
>
> In OE the native sysroot contains tools that are all statically linked
> so you can execute them without a problem. In Isar we can just do that
> with the host tools. Things we have in buildchroot or the image rootfs
> can only be used when chrooting in.
>
> So why not keep all that and use the host tools? Things like mkfs or
> parted are fine, but bootloader installers leak host information into
> our images. So we would build a wheezy with a jessie grub, not the best
> idea.
>
> So i decided to run wic in buildchroot instead of the host. It also
> shifts the whole sudo on the host discussion.
>
> Running wic in buildchroot solves several problems but introduces
> another "interesting" aspect. When doing that with wheezy or jessie you
> are beaming wic so far into the past that the few tool workarounds will
> not work anymore. The tools are too old and even do not support certain
> command line switches. One example we already had for jessie:
>
>> -e2fsprogs/jessie-backports # wic: e2fsprogs -d
>
> So the pragmatic decision was to simply not support distros that
> predate wic itself. It is unlikely we will have to deal with such
> problems looking in the future. Because the new tools in newer distros
> are not likely to be backward incompatible.
>
>> a/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
>> b/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py new
> ...
>> file mode 100644 index 0000000..fccf96c --- /dev/null
>> + grubefi_conf += "menuentry 'boot'{\n"
>> +
>> + kernel = "/vmlinuz"
>
> I did not find a better way than to fork the plugins. But they need to
> be slightly different to what we get from OE. One example is the name
> of the kernel that is hardcoded there.
>
> But the diffs are small and should not be hard to maintain.
Would it make sense to commit first the unmodified plugins from oe here,
and then patch them to work with isar? That way maintaining those
patches could be easier, couldn't it?
>
>> diff --git a/meta/classes/wic-img.bbclass
>> b/meta/classes/wic-img.bbclass new file mode 100644
>> index 0000000..d1deff3
>> --- /dev/null
>> +++ b/meta/classes/wic-img.bbclass
>> @@ -0,0 +1,78 @@
>> +# This software is a part of ISAR.
>> +# Copyright (C) 2018 Siemens AG
>> +#
>> +# this class is heavily inspired by
>> OEs ./meta/classes/image_types_wic.bbclass +#
>> +
>> +WKS_FILE ?= "sdimage-efi"
>> +ROOTFS_TYPE ?= "ext4"
>> +
>> +STAGING_DATADIR ?= "/usr/lib/"
>> +STAGING_LIBDIR ?= "/usr/lib/"
>> +STAGING_DIR ?= "${TMPDIR}"
>> +IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}"
I currently don't understand your value for the IMAGE_BASENAME. Why does
it have multiconfig in its name? Also if those should be valid bitbake
targets, DISTRO is set to "debian-stretch" resulting in
'multiconfig:qemuamd64-debian-stretch:isar-image-base' and that
multiconfig doesn't exist AFAIK.
>> +FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
>> +RECIPE_SYSROOT_NATIVE ?= "/"
>> +
>> +do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
>> +
>> +WIC_CREATE_EXTRA_ARGS ?= ""
>> +
>> +WICVARS ?= "\
>> + BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
>> IMAGE_BASENAME IMAGE_BOOT_FILES \
>> + IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
>> INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
>> + ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
>> TARGET_SYS TRANSLATED_TARGET_ARCH" +
>> +# Isar specific vars used in our plugins
>> +WICVARS += "KERNEL_IMAGE INITRD_IMAGE DISTRO_ARCH"
>> +
>> +python do_rootfs_wicenv () {
>> + wicvars = d.getVar('WICVARS', True)
>> + if not wicvars:
>> + return
>> +
>> + stdir = d.getVar('STAGING_DIR', True)
>> + outdir = os.path.join(stdir, d.getVar('MACHINE', True),
>> 'imgdata')
>> + bb.utils.mkdirhier(outdir)
>> + basename = d.getVar('IMAGE_BASENAME', True)
>> + with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
>> + for var in wicvars.split():
>> + value = d.getVar(var, True)
>> + if value:
>> + envf.write('%s="%s"\n' % (var, value.strip()))
>> +
>> + # this part is stolen from
>> OE ./meta/recipes-core/meta/wic-tools.bb
>> + with open(os.path.join(outdir, "wic-tools.env"), 'w') as envf:
>> + for var in ('RECIPE_SYSROOT_NATIVE', 'STAGING_DATADIR',
>> 'STAGING_LIBDIR'):
>> + envf.write('%s="%s"\n' % (var, d.getVar(var,
>> True).strip())) +
>> +}
>> +
>> +addtask do_rootfs_wicenv after do_copy_boot_files before do_wic_image
>> +do_rootfs_wicenv[vardeps] += "${WICVARS}"
>> +do_rootfs_wicenv[prefuncs] = 'set_image_size'
>> +
>> +WIC_IMAGE_FILE
>> ="${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${MACHINE}.wic.img" +
>> +do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>> +
>> +do_wic_image() {
>> + if ! grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts; then
>> + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs
>> ${BUILDCHROOT_DIR}/dev
>> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>> + fi
>> + for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts; do
>> + sudo mkdir -p ${BUILDCHROOT_DIR}/$dir
>> + sudo mount --bind $dir ${BUILDCHROOT_DIR}/$dir
>> + done
>> + export FAKEROOTCMD=${FAKEROOTCMD}
>> + export BUILDDIR=${BUILDDIR}
>> + export MTOOLS_SKIP_CHECK=1
>
> This is one of the tool workarounds, where i reverted a patch against
> wic.
>
>> +
>> + sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create
>> ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e
>> ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
>> + cp -f `ls -t -1 ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct |
>> head -1` ${WIC_IMAGE_FILE} +}
>> +
>> +do_wic_image[depends] = "buildchroot:do_build"
>> +
>> +addtask wic_image before do_build after do_copy_boot_files
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
>> b/meta/recipes-devtools/buildchroot/buildchroot.bb index
>> b16e63a..bd4d003 100644 ---
>> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++
>> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -28,6 +28,25 @@
>> BUILDCHROOT_PREINSTALL ?= "gcc \ devscripts \
>> equivs"
>>
>> +BUILDCHROOT_PREINSTALL_WIC = " \
>> + parted \
>> + gdisk \
>> + util-linux \
>> + syslinux \
>> + syslinux-common \
>> + dosfstools \
>> + mtools \
>> + e2fsprogs \
>> + grub-efi-amd64-bin \
>> + grub-efi-ia32-bin \
>> + python3"
>> +
>> +python () {
>> + if d.getVar('IMAGE_TYPE', True) == 'wic-img':
That limits how many images types one image recipe has. With OE you
could create multiple image types from one image recipe. Here you only
have one. That might be fine. But just as a note. I currently don't have
a usecase to support multiple image types for one image.
OE supports different post-processes to the wic image. For instance
'wic.xz' creates a compressed wic file and 'wic.bmap' creates a
bmap-tool file in addition to the wic image.
I don't know if you currently have plans to implement such expansions,
but maybe think about how such could be implemented with your current
design.
regards,
Claudius
>> + d.appendVar('BUILDCHROOT_PREINSTALL',
>> + d.getVar('BUILDCHROOT_PREINSTALL_WIC', True))
>> +}
>
> Now the wic tools become part of buildchroot if you chose that
> IMAGE_TYPE.
>
>> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>
>> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>> diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
>> new file mode 100755
>> index 0000000..9e01c38
>> --- /dev/null
>> +++ b/scripts/wic_fakeroot
>> @@ -0,0 +1,37 @@
>> +#!/usr/bin/env python3
>> +#
>> +# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar we
>> do/can not +# use pseudo. And we call wic as root to begin with, so
>> this script could be a +# dummy doing nothing. It is almost a
>> dummy ... +#
>> +# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" can
>> be used +#
>> +# This software is a part of Isar.
>> +# Copyright (C) 2018 Siemens AG
>> +#
>> +import os
>> +import sys
>> +import shutil
>> +import subprocess
>> +
>> +args = sys.argv
>> +args.pop(0)
>> +cmd = args[0]
>> +
>> +# expect to be running as root
>> +# we could loosen that and execv(sudo, args) but even some early
>> +# "du"s fail, which do not use the fakeroot-wrapper
>> +# i.e. in wics partition.py the "du -ks" fails on
>> +# var/cache/apt/archives/partial
>> +# rootfs/root ...
>> +assert 'root' == os.environ["USER"]
>> +
>> +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
>> affected) +# treat 1 as safe ... the filesystem was successfully
>> repaired and is OK +if cmd.startswith('fsck.'):
>> + ret = subprocess.call(args)
>> + if ret == 0 or ret == 1:
>> + sys.exit(0)
>> + sys.exit(ret)
>
> This is another tool workaround, now outside of wic.
>
>> +
>> +os.execv(shutil.which(cmd), args)
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 11/17] images: New class wic-img for wic intregration
2018-04-16 10:17 ` Claudius Heine
@ 2018-04-16 10:25 ` Henning Schild
2018-04-16 10:31 ` Andreas Reichel
1 sibling, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-16 10:25 UTC (permalink / raw)
To: Claudius Heine; +Cc: isar-users
Am Mon, 16 Apr 2018 12:17:55 +0200
schrieb Claudius Heine <claudius.heine.ext@siemens.com>:
> Hi Henning,
>
> first of, great work getting wic functional!
>
> On 2018-04-13 16:44, [ext] Henning Schild wrote:
> > So this is the patch that does the real magic, unfortunately i have
> > to call it that ...
> >
> > Am Fri, 13 Apr 2018 16:19:00 +0200
> > schrieb Henning Schild <henning.schild@siemens.com>:
> >
> >> This patch integrates wic into the bitbake workflow, wic will be
> >> used for the imaging step, no need to call it manually.
> >> After all the previous reverts we now use an unmodified version of
> >> wic.
> >>
> >> Issues:
> >> - wic was never integrated
> >> - you always had to build an ext4-img to create a wic image
> >> later
> >> - there was never a way to control the size of wic
> >> disks/partition, only directly in the wks
> >> - wic used to leak the hosts bootloader into the final image
> >>
> >> Impact:
> >> The patch solves the Issues, but drops the ability to manually
> >> start wic after bitbake. And it drops support for building wic
> >> images for Distros before stretch.
> >>
> >> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> >> ---
> >> doc/user_manual.md | 6 -
> >> .../scripts/lib/wic/canned-wks/sdimage-efi.wks | 2 +-
> >> .../lib/wic/plugins/source/bootimg-efi-isar.py | 297
> >> +++++++++++++++++++++
> >> meta/classes/wic-img.bbclass | 78 ++++++
> >> meta/recipes-devtools/buildchroot/buildchroot.bb | 19 ++
> >> scripts/wic_fakeroot | 37 +++ 6
> >> files changed, 432 insertions(+), 7 deletions(-) create mode 100644
> >> meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py create
> >> mode 100644 meta/classes/wic-img.bbclass create mode 100755
> >> scripts/wic_fakeroot
> >>
> >> diff --git a/doc/user_manual.md b/doc/user_manual.md
> >> index 058f7bd..7bd52f4 100644
> >> --- a/doc/user_manual.md
> >> +++ b/doc/user_manual.md
> >> @@ -52,16 +52,10 @@ The steps below describe how to build the
> >> images provided by default. Install the following packages:
> >> ```
> >> dosfstools
> >> -e2fsprogs/jessie-backports # wic: e2fsprogs -d
> >> -gdisk # wic
> >> git
> >> -grub-efi-amd64-bin # wic
> >> UEFI: /usr/lib/grub/x86_64-efi/moddep.lst
> >> -grub-efi-ia32-bin # wic
> >> UEFI: /usr/lib/grub/i386-efi/moddep.lst -mtools
> >> # wic FAT: mcopy debootstrap parted
> >> python
> >> -python3 # wic
> >> qemu
> >> qemu-user-static
> >> rxvt-unicode # build_parallel
> >
> > We are now running wic in buildchroot and do not need all that
> > anymore.
> >
> > wic uses a set of tools for partitioning etc. and it has
> > dependencies on versions of these tools. In OE people know those
> > and build a native sysroot. But we have debian tools that have
> > slighly different versions.
> >
> > That is why we carry a few workarounds to make wic still like our
> > tools, i will point them out later.
> >
> > In OE the native sysroot contains tools that are all statically
> > linked so you can execute them without a problem. In Isar we can
> > just do that with the host tools. Things we have in buildchroot or
> > the image rootfs can only be used when chrooting in.
> >
> > So why not keep all that and use the host tools? Things like mkfs or
> > parted are fine, but bootloader installers leak host information
> > into our images. So we would build a wheezy with a jessie grub, not
> > the best idea.
> >
> > So i decided to run wic in buildchroot instead of the host. It also
> > shifts the whole sudo on the host discussion.
> >
> > Running wic in buildchroot solves several problems but introduces
> > another "interesting" aspect. When doing that with wheezy or jessie
> > you are beaming wic so far into the past that the few tool
> > workarounds will not work anymore. The tools are too old and even
> > do not support certain command line switches. One example we
> > already had for jessie:
> >> -e2fsprogs/jessie-backports # wic: e2fsprogs -d
> >
> > So the pragmatic decision was to simply not support distros that
> > predate wic itself. It is unlikely we will have to deal with such
> > problems looking in the future. Because the new tools in newer
> > distros are not likely to be backward incompatible.
> >
> >> a/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
> >> b/meta-isar/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
> >> new
> > ...
> >> file mode 100644 index 0000000..fccf96c --- /dev/null
> >> + grubefi_conf += "menuentry 'boot'{\n"
> >> +
> >> + kernel = "/vmlinuz"
> >
> > I did not find a better way than to fork the plugins. But they need
> > to be slightly different to what we get from OE. One example is the
> > name of the kernel that is hardcoded there.
> >
> > But the diffs are small and should not be hard to maintain.
>
> Would it make sense to commit first the unmodified plugins from oe
> here, and then patch them to work with isar? That way maintaining
> those patches could be easier, couldn't it?
For sure, i thought about that as well and then decided a simple diff
would be good enough. But you are right, i will split it up to do that
and review all changes again.
> >
> >> diff --git a/meta/classes/wic-img.bbclass
> >> b/meta/classes/wic-img.bbclass new file mode 100644
> >> index 0000000..d1deff3
> >> --- /dev/null
> >> +++ b/meta/classes/wic-img.bbclass
> >> @@ -0,0 +1,78 @@
> >> +# This software is a part of ISAR.
> >> +# Copyright (C) 2018 Siemens AG
> >> +#
> >> +# this class is heavily inspired by
> >> OEs ./meta/classes/image_types_wic.bbclass +#
> >> +
> >> +WKS_FILE ?= "sdimage-efi"
> >> +ROOTFS_TYPE ?= "ext4"
> >> +
> >> +STAGING_DATADIR ?= "/usr/lib/"
> >> +STAGING_LIBDIR ?= "/usr/lib/"
> >> +STAGING_DIR ?= "${TMPDIR}"
> >> +IMAGE_BASENAME ?= "multiconfig:${MACHINE}-${DISTRO}:${PN}"
>
> I currently don't understand your value for the IMAGE_BASENAME. Why
> does it have multiconfig in its name? Also if those should be valid
> bitbake targets, DISTRO is set to "debian-stretch" resulting in
> 'multiconfig:qemuamd64-debian-stretch:isar-image-base' and that
> multiconfig doesn't exist AFAIK.
To be honest i will have to check that again. I think there is an
expectation in wic when it is looking for the .env file.
> >> +FAKEROOTCMD ?= "${ISARROOT}/scripts/wic_fakeroot"
> >> +RECIPE_SYSROOT_NATIVE ?= "/"
> >> +
> >> +do_wic_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> >> +
> >> +WIC_CREATE_EXTRA_ARGS ?= ""
> >> +
> >> +WICVARS ?= "\
> >> + BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> >> IMAGE_BASENAME IMAGE_BOOT_FILES \
> >> + IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> >> INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
> >> + ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> >> TARGET_SYS TRANSLATED_TARGET_ARCH" +
> >> +# Isar specific vars used in our plugins
> >> +WICVARS += "KERNEL_IMAGE INITRD_IMAGE DISTRO_ARCH"
> >> +
> >> +python do_rootfs_wicenv () {
> >> + wicvars = d.getVar('WICVARS', True)
> >> + if not wicvars:
> >> + return
> >> +
> >> + stdir = d.getVar('STAGING_DIR', True)
> >> + outdir = os.path.join(stdir, d.getVar('MACHINE', True),
> >> 'imgdata')
> >> + bb.utils.mkdirhier(outdir)
> >> + basename = d.getVar('IMAGE_BASENAME', True)
> >> + with open(os.path.join(outdir, basename) + '.env', 'w') as
> >> envf:
> >> + for var in wicvars.split():
> >> + value = d.getVar(var, True)
> >> + if value:
> >> + envf.write('%s="%s"\n' % (var, value.strip()))
> >> +
> >> + # this part is stolen from
> >> OE ./meta/recipes-core/meta/wic-tools.bb
> >> + with open(os.path.join(outdir, "wic-tools.env"), 'w') as envf:
> >> + for var in ('RECIPE_SYSROOT_NATIVE', 'STAGING_DATADIR',
> >> 'STAGING_LIBDIR'):
> >> + envf.write('%s="%s"\n' % (var, d.getVar(var,
> >> True).strip())) +
> >> +}
> >> +
> >> +addtask do_rootfs_wicenv after do_copy_boot_files before
> >> do_wic_image +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> >> +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> >> +
> >> +WIC_IMAGE_FILE
> >> ="${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${MACHINE}.wic.img" +
> >> +do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >> +
> >> +do_wic_image() {
> >> + if ! grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts; then
> >> + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs
> >> ${BUILDCHROOT_DIR}/dev
> >> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
> >> + fi
> >> + for dir in ${BBLAYERS} ${STAGING_DIR} ${ISARROOT}/scripts; do
> >> + sudo mkdir -p ${BUILDCHROOT_DIR}/$dir
> >> + sudo mount --bind $dir ${BUILDCHROOT_DIR}/$dir
> >> + done
> >> + export FAKEROOTCMD=${FAKEROOTCMD}
> >> + export BUILDDIR=${BUILDDIR}
> >> + export MTOOLS_SKIP_CHECK=1
> >
> > This is one of the tool workarounds, where i reverted a patch
> > against wic.
> >
> >> +
> >> + sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic
> >> create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/"
> >> -o /tmp/ -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
> >> + cp -f `ls -t -1 ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct |
> >> head -1` ${WIC_IMAGE_FILE} +}
> >> +
> >> +do_wic_image[depends] = "buildchroot:do_build"
> >> +
> >> +addtask wic_image before do_build after do_copy_boot_files
> >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
> >> b/meta/recipes-devtools/buildchroot/buildchroot.bb index
> >> b16e63a..bd4d003 100644 ---
> >> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++
> >> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -28,6 +28,25
> >> @@ BUILDCHROOT_PREINSTALL ?= "gcc \ devscripts \
> >> equivs"
> >>
> >> +BUILDCHROOT_PREINSTALL_WIC = " \
> >> + parted \
> >> + gdisk \
> >> + util-linux \
> >> + syslinux \
> >> + syslinux-common \
> >> + dosfstools \
> >> + mtools \
> >> + e2fsprogs \
> >> + grub-efi-amd64-bin \
> >> + grub-efi-ia32-bin \
> >> + python3"
> >> +
> >> +python () {
> >> + if d.getVar('IMAGE_TYPE', True) == 'wic-img':
>
> That limits how many images types one image recipe has. With OE you
> could create multiple image types from one image recipe. Here you
> only have one. That might be fine. But just as a note. I currently
> don't have a usecase to support multiple image types for one image.
I actually had "'wic-img' in IMAGE_TYPE" there, which broke building
external layers. IMAGE_TYPE currently is a string an not an array. If
we ever switch to an array, that line needs to change. But right now
it is a string.
Henning
> OE supports different post-processes to the wic image. For instance
> 'wic.xz' creates a compressed wic file and 'wic.bmap' creates a
> bmap-tool file in addition to the wic image.
>
> I don't know if you currently have plans to implement such
> expansions, but maybe think about how such could be implemented with
> your current design.
>
> regards,
> Claudius
>
> >> + d.appendVar('BUILDCHROOT_PREINSTALL',
> >> + d.getVar('BUILDCHROOT_PREINSTALL_WIC', True))
> >> +}
> >
> > Now the wic tools become part of buildchroot if you chose that
> > IMAGE_TYPE.
> >
> >> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
> >>
> >> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >> diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
> >> new file mode 100755
> >> index 0000000..9e01c38
> >> --- /dev/null
> >> +++ b/scripts/wic_fakeroot
> >> @@ -0,0 +1,37 @@
> >> +#!/usr/bin/env python3
> >> +#
> >> +# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar
> >> we do/can not +# use pseudo. And we call wic as root to begin
> >> with, so this script could be a +# dummy doing nothing. It is
> >> almost a dummy ... +#
> >> +# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;"
> >> can be used +#
> >> +# This software is a part of Isar.
> >> +# Copyright (C) 2018 Siemens AG
> >> +#
> >> +import os
> >> +import sys
> >> +import shutil
> >> +import subprocess
> >> +
> >> +args = sys.argv
> >> +args.pop(0)
> >> +cmd = args[0]
> >> +
> >> +# expect to be running as root
> >> +# we could loosen that and execv(sudo, args) but even some early
> >> +# "du"s fail, which do not use the fakeroot-wrapper
> >> +# i.e. in wics partition.py the "du -ks" fails on
> >> +# var/cache/apt/archives/partial
> >> +# rootfs/root ...
> >> +assert 'root' == os.environ["USER"]
> >> +
> >> +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
> >> affected) +# treat 1 as safe ... the filesystem was successfully
> >> repaired and is OK +if cmd.startswith('fsck.'):
> >> + ret = subprocess.call(args)
> >> + if ret == 0 or ret == 1:
> >> + sys.exit(0)
> >> + sys.exit(ret)
> >
> > This is another tool workaround, now outside of wic.
> >
> >> +
> >> +os.execv(shutil.which(cmd), args)
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 11/17] images: New class wic-img for wic intregration
2018-04-16 10:17 ` Claudius Heine
2018-04-16 10:25 ` Henning Schild
@ 2018-04-16 10:31 ` Andreas Reichel
1 sibling, 0 replies; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:31 UTC (permalink / raw)
To: [ext] Claudius Heine; +Cc: [ext] Henning Schild, isar-users
On Mon, Apr 16, 2018 at 12:17:55PM +0200, [ext] Claudius Heine wrote:
> > > + grub-efi-amd64-bin \
> > > + grub-efi-ia32-bin \
> > > + python3"
> > > +
> > > +python () {
> > > + if d.getVar('IMAGE_TYPE', True) == 'wic-img':
>
> That limits how many images types one image recipe has. With OE you could
> create multiple image types from one image recipe. Here you only have one.
> That might be fine. But just as a note. I currently don't have a usecase to
> support multiple image types for one image.
>
> OE supports different post-processes to the wic image. For instance 'wic.xz'
> creates a compressed wic file and 'wic.bmap' creates a bmap-tool file in
> addition to the wic image.
>
> I don't know if you currently have plans to implement such expansions, but
> maybe think about how such could be implemented with your current design.
>
I agree. At least with swupdate we already had the necessity to also add
'ext4.gz'. So I would just test if 'wic-image' is contained in the
IMAGE_TYPE. Nobody knows what image times are needed for special use
cases and an exact match is not required here.
> regards,
> Claudius
>
> > > + d.appendVar('BUILDCHROOT_PREINSTALL',
> > > + d.getVar('BUILDCHROOT_PREINSTALL_WIC', True))
> > > +}
> >
> > Now the wic tools become part of buildchroot if you chose that
> > IMAGE_TYPE.
> >
> > > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
> > > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> > > diff --git a/scripts/wic_fakeroot b/scripts/wic_fakeroot
> > > new file mode 100755
> > > index 0000000..9e01c38
> > > --- /dev/null
> > > +++ b/scripts/wic_fakeroot
> > > @@ -0,0 +1,37 @@
> > > +#!/usr/bin/env python3
> > > +#
> > > +# wic needs a FAKEROOT cmd to run, the default is pseudo. In Isar we
> > > do/can not +# use pseudo. And we call wic as root to begin with, so
> > > this script could be a +# dummy doing nothing. It is almost a
> > > dummy ... +#
> > > +# If the fsck hack ever becomes obsolete, FAKEROOTCMD ?= "true;" can
> > > be used +#
> > > +# This software is a part of Isar.
> > > +# Copyright (C) 2018 Siemens AG
> > > +#
> > > +import os
> > > +import sys
> > > +import shutil
> > > +import subprocess
> > > +
> > > +args = sys.argv
> > > +args.pop(0)
> > > +cmd = args[0]
> > > +
> > > +# expect to be running as root
> > > +# we could loosen that and execv(sudo, args) but even some early
> > > +# "du"s fail, which do not use the fakeroot-wrapper
> > > +# i.e. in wics partition.py the "du -ks" fails on
> > > +# var/cache/apt/archives/partial
> > > +# rootfs/root ...
> > > +assert 'root' == os.environ["USER"]
> > > +
> > > +# e2fsck <= 1.43.5 returns 1 on non-errors (stretch and before
> > > affected) +# treat 1 as safe ... the filesystem was successfully
> > > repaired and is OK +if cmd.startswith('fsck.'):
> > > + ret = subprocess.call(args)
> > > + if ret == 0 or ret == 1:
> > > + sys.exit(0)
> > > + sys.exit(ret)
> >
> > This is another tool workaround, now outside of wic.
> >
I don't get this, sorry :) You have a general script to execute commands
in a fake root environment. But then you always return 0 for all
programs no matter if some programs fail with 1 or not. Also, where does
the output of the failed command go? Can't we use 'popen' and write the
output to stdout to get the error message? Just my two cents...
> > > +
> > > +os.execv(shutil.which(cmd), args)
> >
>
> --
> DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
>
> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To post to this group, send email to isar-users@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/1f89f1a6-9472-ba91-d1f3-fa93722dc2d4%40siemens.com.
> For more options, visit https://groups.google.com/d/optout.
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 12/17] wic: Add pcibios boot plugins and wks files
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (10 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 11/17] images: New class wic-img for wic intregration Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-16 10:52 ` Andreas Reichel
2018-04-13 14:19 ` [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images Henning Schild
` (6 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
Create a modified version of directdisk.wks that enabled legacy boot
images in Isar. "bootimg-pcbios-isar" expects the buildchroot to contain
the packages "syslinux syslinux-common" and will create a disk using this
bootloader.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
.../scripts/lib/wic/canned-wks/common-isar.wks.inc | 3 +
.../scripts/lib/wic/canned-wks/directdisk-isar.wks | 7 +
.../lib/wic/plugins/source/bootimg-pcbios-isar.py | 217 +++++++++++++++++++++
3 files changed, 227 insertions(+)
create mode 100644 meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
create mode 100644 meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
create mode 100644 meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
diff --git a/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc b/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
new file mode 100644
index 0000000..c8ea4c2
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
@@ -0,0 +1,3 @@
+# This file is included into 3 canned wks files from this directory
+part /boot --source bootimg-pcbios-isar --ondisk sda --label boot --active --align 1024
+part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024
diff --git a/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks b/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
new file mode 100644
index 0000000..2b9576d
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
@@ -0,0 +1,7 @@
+# short-description: Create a 'pcbios' direct disk image
+# long-description: Creates a partitioned legacy BIOS disk image that the user
+# can directly dd to boot media.
+
+include common-isar.wks.inc
+
+bootloader --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8"
diff --git a/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py b/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
new file mode 100644
index 0000000..0423862
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
@@ -0,0 +1,217 @@
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (c) 2014, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This implements the 'bootimg-pcbios-isar' source plugin class for 'wic'
+#
+# AUTHORS
+# Tom Zanussi <tom.zanussi (at] linux.intel.com>
+#
+
+import logging
+import os
+
+from wic import WicError
+from wic.engine import get_custom_config
+from wic.utils import runner
+from wic.pluginbase import SourcePlugin
+from wic.utils.misc import (exec_cmd, exec_native_cmd,
+ get_bitbake_var, BOOTDD_EXTRA_SPACE)
+
+logger = logging.getLogger('wic')
+
+class BootimgPcbiosIsarPlugin(SourcePlugin):
+ """
+ Create MBR boot partition and install syslinux on it.
+ """
+
+ name = 'bootimg-pcbios-isar'
+
+ @classmethod
+ def _get_syslinux_dir(cls, bootimg_dir):
+ """
+ Get path to syslinux from either default bootimg_dir
+ or wic-tools STAGING_DIR.
+ """
+ for path in (bootimg_dir, get_bitbake_var("STAGING_DATADIR", "wic-tools")):
+ if not path:
+ continue
+ syslinux_dir = os.path.join(path, 'syslinux')
+ if os.path.exists(syslinux_dir):
+ return syslinux_dir
+
+ raise WicError("Couldn't find syslinux directory, exiting")
+
+ @classmethod
+ def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
+ bootimg_dir, kernel_dir, native_sysroot):
+ """
+ Called after all partitions have been prepared and assembled into a
+ disk image. In this case, we install the MBR.
+ """
+ syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
+ if creator.ptable_format == 'msdos':
+ mbrfile = os.path.join(syslinux_dir, "mbr/mbr.bin")
+ elif creator.ptable_format == 'gpt':
+ mbrfile = os.path.join(syslinux_dir, "mbr/gptmbr.bin")
+ else:
+ raise WicError("Unsupported partition table: %s" %
+ creator.ptable_format)
+
+ if not os.path.exists(mbrfile):
+ raise WicError("Couldn't find %s. If using the -e option, do you "
+ "have the right MACHINE set in local.conf? If not, "
+ "is the bootimg_dir path correct?" % mbrfile)
+
+ full_path = creator._full_path(workdir, disk_name, "direct")
+ logger.debug("Installing MBR on disk %s as %s with size %s bytes",
+ disk_name, full_path, disk.min_size)
+
+ rcode = runner.show(['dd', 'if=%s' % mbrfile,
+ 'of=%s' % full_path, 'conv=notrunc'])
+ if rcode != 0:
+ raise WicError("Unable to set MBR to %s" % full_path)
+
+ @classmethod
+ def do_configure_partition(cls, part, source_params, creator, cr_workdir,
+ oe_builddir, bootimg_dir, kernel_dir,
+ native_sysroot):
+ """
+ Called before do_prepare_partition(), creates syslinux config
+ """
+ hdddir = "%s/hdd/boot" % cr_workdir
+
+ install_cmd = "install -d %s" % hdddir
+ exec_cmd(install_cmd)
+
+ bootloader = creator.ks.bootloader
+
+ custom_cfg = None
+ if bootloader.configfile:
+ custom_cfg = get_custom_config(bootloader.configfile)
+ if custom_cfg:
+ # Use a custom configuration for grub
+ syslinux_conf = custom_cfg
+ logger.debug("Using custom configuration file %s "
+ "for syslinux.cfg", bootloader.configfile)
+ else:
+ raise WicError("configfile is specified but failed to "
+ "get it from %s." % bootloader.configfile)
+
+ if not custom_cfg:
+ # Create syslinux configuration using parameters from wks file
+ splash = os.path.join(cr_workdir, "/hdd/boot/splash.jpg")
+ if os.path.exists(splash):
+ splashline = "menu background splash.jpg"
+ else:
+ splashline = ""
+
+ syslinux_conf = ""
+ syslinux_conf += "PROMPT 0\n"
+ syslinux_conf += "TIMEOUT " + str(bootloader.timeout) + "\n"
+ syslinux_conf += "\n"
+ syslinux_conf += "ALLOWOPTIONS 1\n"
+ syslinux_conf += "SERIAL 0 115200\n"
+ syslinux_conf += "\n"
+ if splashline:
+ syslinux_conf += "%s\n" % splashline
+ syslinux_conf += "DEFAULT boot\n"
+ syslinux_conf += "LABEL boot\n"
+
+ kernel = get_bitbake_var("KERNEL_IMAGE")
+ initrd = get_bitbake_var("INITRD_IMAGE")
+ syslinux_conf += "KERNEL " + kernel + "\n"
+
+ syslinux_conf += "APPEND label=boot root=%s initrd=%s %s\n" % \
+ (creator.rootdev, initrd, bootloader.append)
+
+ logger.debug("Writing syslinux config %s/hdd/boot/syslinux.cfg",
+ cr_workdir)
+ cfg = open("%s/hdd/boot/syslinux.cfg" % cr_workdir, "w")
+ cfg.write(syslinux_conf)
+ cfg.close()
+
+ @classmethod
+ def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
+ oe_builddir, bootimg_dir, kernel_dir,
+ rootfs_dir, native_sysroot):
+ """
+ Called to do the actual content population for a partition i.e. it
+ 'prepares' the partition to be incorporated into the image.
+ In this case, prepare content for legacy bios boot partition.
+ """
+ syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
+
+ staging_kernel_dir = kernel_dir
+ kernel = get_bitbake_var("KERNEL_IMAGE")
+ initrd = get_bitbake_var("INITRD_IMAGE")
+
+ hdddir = "%s/hdd/boot" % cr_workdir
+
+ cmds = ("install -m 0644 %s/%s %s/%s" %
+ (staging_kernel_dir, kernel, hdddir, kernel),
+ "install -m 0644 %s/%s %s/%s" %
+ (staging_kernel_dir, initrd, hdddir, initrd),
+ "install -m 444 %s/modules/bios/ldlinux.c32 %s/ldlinux.c32" %
+ (syslinux_dir, hdddir),
+ "install -m 0644 %s/modules/bios/vesamenu.c32 %s/vesamenu.c32" %
+ (syslinux_dir, hdddir),
+ "install -m 444 %s/modules/bios/libcom32.c32 %s/libcom32.c32" %
+ (syslinux_dir, hdddir),
+ "install -m 444 %s/modules/bios/libutil.c32 %s/libutil.c32" %
+ (syslinux_dir, hdddir))
+
+ for install_cmd in cmds:
+ exec_cmd(install_cmd)
+
+ du_cmd = "du -bks %s" % hdddir
+ out = exec_cmd(du_cmd)
+ blocks = int(out.split()[0])
+
+ extra_blocks = part.get_extra_block_count(blocks)
+
+ if extra_blocks < BOOTDD_EXTRA_SPACE:
+ extra_blocks = BOOTDD_EXTRA_SPACE
+
+ blocks += extra_blocks
+
+ logger.debug("Added %d extra blocks to %s to get to %d total blocks",
+ extra_blocks, part.mountpoint, blocks)
+
+ # dosfs image, created by mkdosfs
+ bootimg = "%s/boot.img" % cr_workdir
+
+ dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, blocks)
+ exec_native_cmd(dosfs_cmd, native_sysroot)
+
+ mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
+ exec_cmd(mcopy_cmd, native_sysroot)
+
+ syslinux_cmd = "syslinux %s" % bootimg
+ exec_native_cmd(syslinux_cmd, native_sysroot)
+
+ chmod_cmd = "chmod 644 %s" % bootimg
+ exec_cmd(chmod_cmd)
+
+ du_cmd = "du -Lbks %s" % bootimg
+ out = exec_cmd(du_cmd)
+ bootimg_size = out.split()[0]
+
+ part.size = int(bootimg_size)
+ part.source_file = bootimg
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 12/17] wic: Add pcibios boot plugins and wks files
2018-04-13 14:19 ` [PATCH v2 12/17] wic: Add pcibios boot plugins and wks files Henning Schild
@ 2018-04-16 10:52 ` Andreas Reichel
2018-04-16 11:48 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:52 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Fri, Apr 13, 2018 at 04:19:01PM +0200, Henning Schild wrote:
> Create a modified version of directdisk.wks that enabled legacy boot
> images in Isar. "bootimg-pcbios-isar" expects the buildchroot to contain
> the packages "syslinux syslinux-common" and will create a disk using this
> bootloader.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> .../scripts/lib/wic/canned-wks/common-isar.wks.inc | 3 +
> .../scripts/lib/wic/canned-wks/directdisk-isar.wks | 7 +
> .../lib/wic/plugins/source/bootimg-pcbios-isar.py | 217 +++++++++++++++++++++
> 3 files changed, 227 insertions(+)
> create mode 100644 meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
> create mode 100644 meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
> create mode 100644 meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
>
> diff --git a/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc b/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
> new file mode 100644
> index 0000000..c8ea4c2
> --- /dev/null
> +++ b/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
> @@ -0,0 +1,3 @@
> +# This file is included into 3 canned wks files from this directory
> +part /boot --source bootimg-pcbios-isar --ondisk sda --label boot --active --align 1024
> +part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024
> diff --git a/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks b/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
> new file mode 100644
> index 0000000..2b9576d
> --- /dev/null
> +++ b/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
> @@ -0,0 +1,7 @@
> +# short-description: Create a 'pcbios' direct disk image
> +# long-description: Creates a partitioned legacy BIOS disk image that the user
> +# can directly dd to boot media.
> +
> +include common-isar.wks.inc
> +
> +bootloader --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8"
> diff --git a/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py b/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> new file mode 100644
> index 0000000..0423862
> --- /dev/null
> +++ b/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> @@ -0,0 +1,217 @@
> +# ex:ts=4:sw=4:sts=4:et
> +# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> +#
> +# Copyright (c) 2014, Intel Corporation.
> +# All rights reserved.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 2 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License along
> +# with this program; if not, write to the Free Software Foundation, Inc.,
> +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> +#
> +# DESCRIPTION
> +# This implements the 'bootimg-pcbios-isar' source plugin class for 'wic'
> +#
> +# AUTHORS
> +# Tom Zanussi <tom.zanussi (at] linux.intel.com>
> +#
> +
> +import logging
> +import os
> +
> +from wic import WicError
> +from wic.engine import get_custom_config
> +from wic.utils import runner
> +from wic.pluginbase import SourcePlugin
> +from wic.utils.misc import (exec_cmd, exec_native_cmd,
> + get_bitbake_var, BOOTDD_EXTRA_SPACE)
> +
> +logger = logging.getLogger('wic')
> +
> +class BootimgPcbiosIsarPlugin(SourcePlugin):
> + """
> + Create MBR boot partition and install syslinux on it.
> + """
> +
> + name = 'bootimg-pcbios-isar'
> +
> + @classmethod
> + def _get_syslinux_dir(cls, bootimg_dir):
> + """
> + Get path to syslinux from either default bootimg_dir
> + or wic-tools STAGING_DIR.
> + """
> + for path in (bootimg_dir, get_bitbake_var("STAGING_DATADIR", "wic-tools")):
> + if not path:
> + continue
> + syslinux_dir = os.path.join(path, 'syslinux')
> + if os.path.exists(syslinux_dir):
> + return syslinux_dir
> +
> + raise WicError("Couldn't find syslinux directory, exiting")
> +
> + @classmethod
> + def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
> + bootimg_dir, kernel_dir, native_sysroot):
> + """
> + Called after all partitions have been prepared and assembled into a
> + disk image. In this case, we install the MBR.
> + """
> + syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
> + if creator.ptable_format == 'msdos':
> + mbrfile = os.path.join(syslinux_dir, "mbr/mbr.bin")
> + elif creator.ptable_format == 'gpt':
> + mbrfile = os.path.join(syslinux_dir, "mbr/gptmbr.bin")
I know that a GPT contains an MBR for compatibility reasons. But this is
usually just to stop the system from booting of the harddisk partitioned
by GPT if it is used in an older PC accidentally. Why would one want to
install syslinux with MBR support into the protected MBR of a GPT that
should not be used for booting? Or do I get that wrong?
> + else:
> + raise WicError("Unsupported partition table: %s" %
> + creator.ptable_format)
> +
> + if not os.path.exists(mbrfile):
> + raise WicError("Couldn't find %s. If using the -e option, do you "
> + "have the right MACHINE set in local.conf? If not, "
> + "is the bootimg_dir path correct?" % mbrfile)
> +
> + full_path = creator._full_path(workdir, disk_name, "direct")
> + logger.debug("Installing MBR on disk %s as %s with size %s bytes",
> + disk_name, full_path, disk.min_size)
> +
> + rcode = runner.show(['dd', 'if=%s' % mbrfile,
> + 'of=%s' % full_path, 'conv=notrunc'])
> + if rcode != 0:
> + raise WicError("Unable to set MBR to %s" % full_path)
> +
> + @classmethod
> + def do_configure_partition(cls, part, source_params, creator, cr_workdir,
> + oe_builddir, bootimg_dir, kernel_dir,
> + native_sysroot):
> + """
> + Called before do_prepare_partition(), creates syslinux config
> + """
> + hdddir = "%s/hdd/boot" % cr_workdir
> +
> + install_cmd = "install -d %s" % hdddir
> + exec_cmd(install_cmd)
> +
> + bootloader = creator.ks.bootloader
> +
> + custom_cfg = None
> + if bootloader.configfile:
> + custom_cfg = get_custom_config(bootloader.configfile)
> + if custom_cfg:
> + # Use a custom configuration for grub
> + syslinux_conf = custom_cfg
> + logger.debug("Using custom configuration file %s "
> + "for syslinux.cfg", bootloader.configfile)
> + else:
> + raise WicError("configfile is specified but failed to "
> + "get it from %s." % bootloader.configfile)
> +
> + if not custom_cfg:
> + # Create syslinux configuration using parameters from wks file
> + splash = os.path.join(cr_workdir, "/hdd/boot/splash.jpg")
> + if os.path.exists(splash):
> + splashline = "menu background splash.jpg"
> + else:
> + splashline = ""
> +
> + syslinux_conf = ""
> + syslinux_conf += "PROMPT 0\n"
> + syslinux_conf += "TIMEOUT " + str(bootloader.timeout) + "\n"
> + syslinux_conf += "\n"
> + syslinux_conf += "ALLOWOPTIONS 1\n"
> + syslinux_conf += "SERIAL 0 115200\n"
> + syslinux_conf += "\n"
> + if splashline:
> + syslinux_conf += "%s\n" % splashline
> + syslinux_conf += "DEFAULT boot\n"
> + syslinux_conf += "LABEL boot\n"
> +
> + kernel = get_bitbake_var("KERNEL_IMAGE")
> + initrd = get_bitbake_var("INITRD_IMAGE")
> + syslinux_conf += "KERNEL " + kernel + "\n"
> +
> + syslinux_conf += "APPEND label=boot root=%s initrd=%s %s\n" % \
> + (creator.rootdev, initrd, bootloader.append)
> +
> + logger.debug("Writing syslinux config %s/hdd/boot/syslinux.cfg",
> + cr_workdir)
> + cfg = open("%s/hdd/boot/syslinux.cfg" % cr_workdir, "w")
> + cfg.write(syslinux_conf)
> + cfg.close()
> +
> + @classmethod
> + def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
> + oe_builddir, bootimg_dir, kernel_dir,
> + rootfs_dir, native_sysroot):
> + """
> + Called to do the actual content population for a partition i.e. it
> + 'prepares' the partition to be incorporated into the image.
> + In this case, prepare content for legacy bios boot partition.
> + """
> + syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
> +
> + staging_kernel_dir = kernel_dir
> + kernel = get_bitbake_var("KERNEL_IMAGE")
> + initrd = get_bitbake_var("INITRD_IMAGE")
> +
> + hdddir = "%s/hdd/boot" % cr_workdir
> +
> + cmds = ("install -m 0644 %s/%s %s/%s" %
> + (staging_kernel_dir, kernel, hdddir, kernel),
> + "install -m 0644 %s/%s %s/%s" %
> + (staging_kernel_dir, initrd, hdddir, initrd),
> + "install -m 444 %s/modules/bios/ldlinux.c32 %s/ldlinux.c32" %
> + (syslinux_dir, hdddir),
> + "install -m 0644 %s/modules/bios/vesamenu.c32 %s/vesamenu.c32" %
> + (syslinux_dir, hdddir),
> + "install -m 444 %s/modules/bios/libcom32.c32 %s/libcom32.c32" %
> + (syslinux_dir, hdddir),
> + "install -m 444 %s/modules/bios/libutil.c32 %s/libutil.c32" %
> + (syslinux_dir, hdddir))
> +
> + for install_cmd in cmds:
> + exec_cmd(install_cmd)
> +
> + du_cmd = "du -bks %s" % hdddir
> + out = exec_cmd(du_cmd)
> + blocks = int(out.split()[0])
> +
> + extra_blocks = part.get_extra_block_count(blocks)
> +
> + if extra_blocks < BOOTDD_EXTRA_SPACE:
> + extra_blocks = BOOTDD_EXTRA_SPACE
> +
> + blocks += extra_blocks
> +
> + logger.debug("Added %d extra blocks to %s to get to %d total blocks",
> + extra_blocks, part.mountpoint, blocks)
> +
> + # dosfs image, created by mkdosfs
> + bootimg = "%s/boot.img" % cr_workdir
> +
> + dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg, blocks)
> + exec_native_cmd(dosfs_cmd, native_sysroot)
> +
> + mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
> + exec_cmd(mcopy_cmd, native_sysroot)
> +
> + syslinux_cmd = "syslinux %s" % bootimg
> + exec_native_cmd(syslinux_cmd, native_sysroot)
> +
> + chmod_cmd = "chmod 644 %s" % bootimg
> + exec_cmd(chmod_cmd)
> +
> + du_cmd = "du -Lbks %s" % bootimg
> + out = exec_cmd(du_cmd)
> + bootimg_size = out.split()[0]
> +
> + part.size = int(bootimg_size)
> + part.source_file = bootimg
> --
> 2.16.1
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 12/17] wic: Add pcibios boot plugins and wks files
2018-04-16 10:52 ` Andreas Reichel
@ 2018-04-16 11:48 ` Henning Schild
0 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-16 11:48 UTC (permalink / raw)
To: Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Am Mon, 16 Apr 2018 12:52:29 +0200
schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
> On Fri, Apr 13, 2018 at 04:19:01PM +0200, Henning Schild wrote:
> > Create a modified version of directdisk.wks that enabled legacy boot
> > images in Isar. "bootimg-pcbios-isar" expects the buildchroot to
> > contain the packages "syslinux syslinux-common" and will create a
> > disk using this bootloader.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > .../scripts/lib/wic/canned-wks/common-isar.wks.inc | 3 +
> > .../scripts/lib/wic/canned-wks/directdisk-isar.wks | 7 +
> > .../lib/wic/plugins/source/bootimg-pcbios-isar.py | 217
> > +++++++++++++++++++++ 3 files changed, 227 insertions(+)
> > create mode 100644
> > meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc create
> > mode 100644
> > meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks create
> > mode 100644
> > meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> >
> > diff --git
> > a/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
> > b/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc new file
> > mode 100644 index 0000000..c8ea4c2 --- /dev/null
> > +++ b/meta-isar/scripts/lib/wic/canned-wks/common-isar.wks.inc
> > @@ -0,0 +1,3 @@
> > +# This file is included into 3 canned wks files from this directory
> > +part /boot --source bootimg-pcbios-isar --ondisk sda --label boot
> > --active --align 1024 +part / --source rootfs --ondisk sda
> > --fstype=ext4 --label platform --align 1024 diff --git
> > a/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
> > b/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks new file
> > mode 100644 index 0000000..2b9576d --- /dev/null
> > +++ b/meta-isar/scripts/lib/wic/canned-wks/directdisk-isar.wks
> > @@ -0,0 +1,7 @@
> > +# short-description: Create a 'pcbios' direct disk image
> > +# long-description: Creates a partitioned legacy BIOS disk image
> > that the user +# can directly dd to boot media.
> > +
> > +include common-isar.wks.inc
> > +
> > +bootloader --timeout=0 --append="rootwait rootfstype=ext4
> > video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" diff
> > --git
> > a/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> > b/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> > new file mode 100644 index 0000000..0423862 --- /dev/null +++
> > b/meta-isar/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> > @@ -0,0 +1,217 @@ +# ex:ts=4:sw=4:sts=4:et
> > +# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> > +#
> > +# Copyright (c) 2014, Intel Corporation.
> > +# All rights reserved.
> > +#
> > +# This program is free software; you can redistribute it and/or
> > modify +# it under the terms of the GNU General Public License
> > version 2 as +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public
> > License along +# with this program; if not, write to the Free
> > Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor,
> > Boston, MA 02110-1301 USA. +#
> > +# DESCRIPTION
> > +# This implements the 'bootimg-pcbios-isar' source plugin class
> > for 'wic' +#
> > +# AUTHORS
> > +# Tom Zanussi <tom.zanussi (at] linux.intel.com>
> > +#
> > +
> > +import logging
> > +import os
> > +
> > +from wic import WicError
> > +from wic.engine import get_custom_config
> > +from wic.utils import runner
> > +from wic.pluginbase import SourcePlugin
> > +from wic.utils.misc import (exec_cmd, exec_native_cmd,
> > + get_bitbake_var, BOOTDD_EXTRA_SPACE)
> > +
> > +logger = logging.getLogger('wic')
> > +
> > +class BootimgPcbiosIsarPlugin(SourcePlugin):
> > + """
> > + Create MBR boot partition and install syslinux on it.
> > + """
> > +
> > + name = 'bootimg-pcbios-isar'
> > +
> > + @classmethod
> > + def _get_syslinux_dir(cls, bootimg_dir):
> > + """
> > + Get path to syslinux from either default bootimg_dir
> > + or wic-tools STAGING_DIR.
> > + """
> > + for path in (bootimg_dir,
> > get_bitbake_var("STAGING_DATADIR", "wic-tools")):
> > + if not path:
> > + continue
> > + syslinux_dir = os.path.join(path, 'syslinux')
> > + if os.path.exists(syslinux_dir):
> > + return syslinux_dir
> > +
> > + raise WicError("Couldn't find syslinux directory, exiting")
> > +
> > + @classmethod
> > + def do_install_disk(cls, disk, disk_name, creator, workdir,
> > oe_builddir,
> > + bootimg_dir, kernel_dir, native_sysroot):
> > + """
> > + Called after all partitions have been prepared and
> > assembled into a
> > + disk image. In this case, we install the MBR.
> > + """
> > + syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
> > + if creator.ptable_format == 'msdos':
> > + mbrfile = os.path.join(syslinux_dir, "mbr/mbr.bin")
> > + elif creator.ptable_format == 'gpt':
> > + mbrfile = os.path.join(syslinux_dir,
> > "mbr/gptmbr.bin")
> I know that a GPT contains an MBR for compatibility reasons. But this
> is usually just to stop the system from booting of the harddisk
> partitioned by GPT if it is used in an older PC accidentally. Why
> would one want to install syslinux with MBR support into the
> protected MBR of a GPT that should not be used for booting? Or do I
> get that wrong?
This is exactly how wic does it. Maybe when i send that as fork and
later modify we can just review the isar-changes.
Henning
> > + else:
> > + raise WicError("Unsupported partition table: %s" %
> > + creator.ptable_format)
> > +
> > + if not os.path.exists(mbrfile):
> > + raise WicError("Couldn't find %s. If using the -e
> > option, do you "
> > + "have the right MACHINE set in
> > local.conf? If not, "
> > + "is the bootimg_dir path correct?" %
> > mbrfile) +
> > + full_path = creator._full_path(workdir, disk_name,
> > "direct")
> > + logger.debug("Installing MBR on disk %s as %s with size %s
> > bytes",
> > + disk_name, full_path, disk.min_size)
> > +
> > + rcode = runner.show(['dd', 'if=%s' % mbrfile,
> > + 'of=%s' % full_path, 'conv=notrunc'])
> > + if rcode != 0:
> > + raise WicError("Unable to set MBR to %s" % full_path)
> > +
> > + @classmethod
> > + def do_configure_partition(cls, part, source_params, creator,
> > cr_workdir,
> > + oe_builddir, bootimg_dir,
> > kernel_dir,
> > + native_sysroot):
> > + """
> > + Called before do_prepare_partition(), creates syslinux
> > config
> > + """
> > + hdddir = "%s/hdd/boot" % cr_workdir
> > +
> > + install_cmd = "install -d %s" % hdddir
> > + exec_cmd(install_cmd)
> > +
> > + bootloader = creator.ks.bootloader
> > +
> > + custom_cfg = None
> > + if bootloader.configfile:
> > + custom_cfg = get_custom_config(bootloader.configfile)
> > + if custom_cfg:
> > + # Use a custom configuration for grub
> > + syslinux_conf = custom_cfg
> > + logger.debug("Using custom configuration file %s "
> > + "for syslinux.cfg",
> > bootloader.configfile)
> > + else:
> > + raise WicError("configfile is specified but failed
> > to "
> > + "get it from %s." %
> > bootloader.configfile) +
> > + if not custom_cfg:
> > + # Create syslinux configuration using parameters from
> > wks file
> > + splash = os.path.join(cr_workdir,
> > "/hdd/boot/splash.jpg")
> > + if os.path.exists(splash):
> > + splashline = "menu background splash.jpg"
> > + else:
> > + splashline = ""
> > +
> > + syslinux_conf = ""
> > + syslinux_conf += "PROMPT 0\n"
> > + syslinux_conf += "TIMEOUT " + str(bootloader.timeout)
> > + "\n"
> > + syslinux_conf += "\n"
> > + syslinux_conf += "ALLOWOPTIONS 1\n"
> > + syslinux_conf += "SERIAL 0 115200\n"
> > + syslinux_conf += "\n"
> > + if splashline:
> > + syslinux_conf += "%s\n" % splashline
> > + syslinux_conf += "DEFAULT boot\n"
> > + syslinux_conf += "LABEL boot\n"
> > +
> > + kernel = get_bitbake_var("KERNEL_IMAGE")
> > + initrd = get_bitbake_var("INITRD_IMAGE")
> > + syslinux_conf += "KERNEL " + kernel + "\n"
> > +
> > + syslinux_conf += "APPEND label=boot root=%s initrd=%s
> > %s\n" % \
> > + (creator.rootdev, initrd,
> > bootloader.append) +
> > + logger.debug("Writing syslinux config
> > %s/hdd/boot/syslinux.cfg",
> > + cr_workdir)
> > + cfg = open("%s/hdd/boot/syslinux.cfg" % cr_workdir, "w")
> > + cfg.write(syslinux_conf)
> > + cfg.close()
> > +
> > + @classmethod
> > + def do_prepare_partition(cls, part, source_params, creator,
> > cr_workdir,
> > + oe_builddir, bootimg_dir, kernel_dir,
> > + rootfs_dir, native_sysroot):
> > + """
> > + Called to do the actual content population for a partition
> > i.e. it
> > + 'prepares' the partition to be incorporated into the image.
> > + In this case, prepare content for legacy bios boot
> > partition.
> > + """
> > + syslinux_dir = cls._get_syslinux_dir(bootimg_dir)
> > +
> > + staging_kernel_dir = kernel_dir
> > + kernel = get_bitbake_var("KERNEL_IMAGE")
> > + initrd = get_bitbake_var("INITRD_IMAGE")
> > +
> > + hdddir = "%s/hdd/boot" % cr_workdir
> > +
> > + cmds = ("install -m 0644 %s/%s %s/%s" %
> > + (staging_kernel_dir, kernel, hdddir, kernel),
> > + "install -m 0644 %s/%s %s/%s" %
> > + (staging_kernel_dir, initrd, hdddir, initrd),
> > + "install -m 444 %s/modules/bios/ldlinux.c32
> > %s/ldlinux.c32" %
> > + (syslinux_dir, hdddir),
> > + "install -m 0644 %s/modules/bios/vesamenu.c32
> > %s/vesamenu.c32" %
> > + (syslinux_dir, hdddir),
> > + "install -m 444 %s/modules/bios/libcom32.c32
> > %s/libcom32.c32" %
> > + (syslinux_dir, hdddir),
> > + "install -m 444 %s/modules/bios/libutil.c32
> > %s/libutil.c32" %
> > + (syslinux_dir, hdddir))
> > +
> > + for install_cmd in cmds:
> > + exec_cmd(install_cmd)
> > +
> > + du_cmd = "du -bks %s" % hdddir
> > + out = exec_cmd(du_cmd)
> > + blocks = int(out.split()[0])
> > +
> > + extra_blocks = part.get_extra_block_count(blocks)
> > +
> > + if extra_blocks < BOOTDD_EXTRA_SPACE:
> > + extra_blocks = BOOTDD_EXTRA_SPACE
> > +
> > + blocks += extra_blocks
> > +
> > + logger.debug("Added %d extra blocks to %s to get to %d
> > total blocks",
> > + extra_blocks, part.mountpoint, blocks)
> > +
> > + # dosfs image, created by mkdosfs
> > + bootimg = "%s/boot.img" % cr_workdir
> > +
> > + dosfs_cmd = "mkdosfs -n boot -S 512 -C %s %d" % (bootimg,
> > blocks)
> > + exec_native_cmd(dosfs_cmd, native_sysroot)
> > +
> > + mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
> > + exec_cmd(mcopy_cmd, native_sysroot)
> > +
> > + syslinux_cmd = "syslinux %s" % bootimg
> > + exec_native_cmd(syslinux_cmd, native_sysroot)
> > +
> > + chmod_cmd = "chmod 644 %s" % bootimg
> > + exec_cmd(chmod_cmd)
> > +
> > + du_cmd = "du -Lbks %s" % bootimg
> > + out = exec_cmd(du_cmd)
> > + bootimg_size = out.split()[0]
> > +
> > + part.size = int(bootimg_size)
> > + part.source_file = bootimg
> > --
> > 2.16.1
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (11 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 12/17] wic: Add pcibios boot plugins and wks files Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-16 10:33 ` Andreas Reichel
2018-04-13 14:19 ` [PATCH v2 14/17] multiconfig: Switch qemuamd64-stretch to using wic by default Henning Schild
` (5 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
The scripts used to support booting with kernel, initrd and rootfs.
Enable it to support the IMAGE_TYPE 'wic-img' as well.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/start_vm | 35 +++++++++++++++++++++++++----------
1 file changed, 25 insertions(+), 10 deletions(-)
diff --git a/scripts/start_vm b/scripts/start_vm
index 6c89ce9..cfb2bea 100755
--- a/scripts/start_vm
+++ b/scripts/start_vm
@@ -27,9 +27,6 @@ start_qemu() {
-M $QEMU_MACHINE \
$QCPU \
-nographic \
- -kernel $QKERNEL \
- -initrd $QINITRD \
- -append "console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV rw" \
$EXTRA_ARGS \
$root
[ $had_x -eq 0 ] && set +x
@@ -104,13 +101,6 @@ do
done
readonly IMAGE_DIR=$BUILD_DIR/tmp/deploy/images
-readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
-
-eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=")
-eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_IMAGE=")
-QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
-QINITRD=/dev/null
-[ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
readonly ISARROOT="$(dirname "$0")"/..
@@ -120,6 +110,31 @@ eval "$(egrep 'MACHINE_SERIAL' $MACHINE_CONF |bb2sh)"
readonly CONFIG_CONF=$ISARROOT/meta-isar/conf/multiconfig/qemu$ARCH-$DISTRO.conf
eval "$(egrep 'ROOTFS_DEV|QEMU_' $CONFIG_CONF |bb2sh)"
+eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^IMAGE_TYPE=")
+case "$IMAGE_TYPE" in
+ ext4-img)
+ readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
+
+ eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=")
+ eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_IMAGE=")
+ QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
+ QINITRD=/dev/null
+ [ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
+ EXTRA_ARGS="$EXTRA_ARGS \
+ -kernel $QKERNEL \
+ -initrd $QINITRD \
+ -append \"console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV rw\""
+ ;;
+ wic-img)
+ readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.wic.img
+ EXTRA_ARGS="$EXTRA_ARGS -snapshot"
+ ;;
+ *)
+ echo "IMAGE_TYPE \"$IMAGE_TYPE\" not supported"
+ exit 1
+ ;;
+esac
+
QCPU=
[ -n "$QEMU_CPU" ] && QCPU="-cpu $QEMU_CPU"
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images
2018-04-13 14:19 ` [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images Henning Schild
@ 2018-04-16 10:33 ` Andreas Reichel
2018-04-16 11:46 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:33 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Fri, Apr 13, 2018 at 04:19:02PM +0200, Henning Schild wrote:
> The scripts used to support booting with kernel, initrd and rootfs.
> Enable it to support the IMAGE_TYPE 'wic-img' as well.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> scripts/start_vm | 35 +++++++++++++++++++++++++----------
> 1 file changed, 25 insertions(+), 10 deletions(-)
>
> diff --git a/scripts/start_vm b/scripts/start_vm
> index 6c89ce9..cfb2bea 100755
> --- a/scripts/start_vm
> +++ b/scripts/start_vm
> @@ -27,9 +27,6 @@ start_qemu() {
> -M $QEMU_MACHINE \
> $QCPU \
> -nographic \
> - -kernel $QKERNEL \
> - -initrd $QINITRD \
> - -append "console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV rw" \
> $EXTRA_ARGS \
> $root
> [ $had_x -eq 0 ] && set +x
> @@ -104,13 +101,6 @@ do
> done
>
> readonly IMAGE_DIR=$BUILD_DIR/tmp/deploy/images
> -readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
> -
> -eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=")
> -eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_IMAGE=")
> -QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> -QINITRD=/dev/null
> -[ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
>
> readonly ISARROOT="$(dirname "$0")"/..
>
> @@ -120,6 +110,31 @@ eval "$(egrep 'MACHINE_SERIAL' $MACHINE_CONF |bb2sh)"
> readonly CONFIG_CONF=$ISARROOT/meta-isar/conf/multiconfig/qemu$ARCH-$DISTRO.conf
> eval "$(egrep 'ROOTFS_DEV|QEMU_' $CONFIG_CONF |bb2sh)"
>
> +eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^IMAGE_TYPE=")
> +case "$IMAGE_TYPE" in
> + ext4-img)
> + readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
> +
> + eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^KERNEL_IMAGE=")
> + eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep "^INITRD_IMAGE=")
> + QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> + QINITRD=/dev/null
> + [ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
> + EXTRA_ARGS="$EXTRA_ARGS \
> + -kernel $QKERNEL \
> + -initrd $QINITRD \
> + -append \"console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV rw\""
> + ;;
> + wic-img)
> + readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.wic.img
> + EXTRA_ARGS="$EXTRA_ARGS -snapshot"
> + ;;
> + *)
> + echo "IMAGE_TYPE \"$IMAGE_TYPE\" not supported"
> + exit 1
> + ;;
> +esac
Here we have the same problem, that only one image type can be set.
> +
> QCPU=
> [ -n "$QEMU_CPU" ] && QCPU="-cpu $QEMU_CPU"
>
> --
> 2.16.1
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images
2018-04-16 10:33 ` Andreas Reichel
@ 2018-04-16 11:46 ` Henning Schild
2018-04-16 13:24 ` Andreas Reichel
0 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-16 11:46 UTC (permalink / raw)
To: Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Am Mon, 16 Apr 2018 12:33:57 +0200
schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
> On Fri, Apr 13, 2018 at 04:19:02PM +0200, Henning Schild wrote:
> > The scripts used to support booting with kernel, initrd and rootfs.
> > Enable it to support the IMAGE_TYPE 'wic-img' as well.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > scripts/start_vm | 35 +++++++++++++++++++++++++----------
> > 1 file changed, 25 insertions(+), 10 deletions(-)
> >
> > diff --git a/scripts/start_vm b/scripts/start_vm
> > index 6c89ce9..cfb2bea 100755
> > --- a/scripts/start_vm
> > +++ b/scripts/start_vm
> > @@ -27,9 +27,6 @@ start_qemu() {
> > -M $QEMU_MACHINE \
> > $QCPU \
> > -nographic \
> > - -kernel $QKERNEL \
> > - -initrd $QINITRD \
> > - -append "console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV rw"
> > \ $EXTRA_ARGS \
> > $root
> > [ $had_x -eq 0 ] && set +x
> > @@ -104,13 +101,6 @@ do
> > done
> >
> > readonly IMAGE_DIR=$BUILD_DIR/tmp/deploy/images
> > -readonly
> > ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img -
> > -eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base |
> > grep "^KERNEL_IMAGE=") -eval $(bitbake -e
> > multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep
> > "^INITRD_IMAGE=") -QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> > -QINITRD=/dev/null -[ -n "$INITRD_IMAGE" ] &&
> > QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
> > readonly ISARROOT="$(dirname "$0")"/..
> >
> > @@ -120,6 +110,31 @@ eval "$(egrep 'MACHINE_SERIAL' $MACHINE_CONF
> > |bb2sh)" readonly
> > CONFIG_CONF=$ISARROOT/meta-isar/conf/multiconfig/qemu$ARCH-$DISTRO.conf
> > eval "$(egrep 'ROOTFS_DEV|QEMU_' $CONFIG_CONF |bb2sh)"
> > +eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base |
> > grep "^IMAGE_TYPE=") +case "$IMAGE_TYPE" in
> > + ext4-img)
> > + readonly
> > ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img +
> > + eval $(bitbake -e
> > multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep
> > "^KERNEL_IMAGE=")
> > + eval $(bitbake -e
> > multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep
> > "^INITRD_IMAGE=")
> > + QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> > + QINITRD=/dev/null
> > + [ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
> > + EXTRA_ARGS="$EXTRA_ARGS \
> > + -kernel $QKERNEL \
> > + -initrd $QINITRD \
> > + -append \"console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV
> > rw\""
> > + ;;
> > + wic-img)
> > + readonly
> > ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.wic.img
> > + EXTRA_ARGS="$EXTRA_ARGS -snapshot"
> > + ;;
> > + *)
> > + echo "IMAGE_TYPE \"$IMAGE_TYPE\" not supported"
> > + exit 1
> > + ;;
> > +esac
>
> Here we have the same problem, that only one image type can be set.
Yes, but as already discussed it is no array in Isar.
Henning
> > +
> > QCPU=
> > [ -n "$QEMU_CPU" ] && QCPU="-cpu $QEMU_CPU"
> >
> > --
> > 2.16.1
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images
2018-04-16 11:46 ` Henning Schild
@ 2018-04-16 13:24 ` Andreas Reichel
0 siblings, 0 replies; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 13:24 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Mon, Apr 16, 2018 at 01:46:28PM +0200, Henning Schild wrote:
> Am Mon, 16 Apr 2018 12:33:57 +0200
> schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
>
> > On Fri, Apr 13, 2018 at 04:19:02PM +0200, Henning Schild wrote:
> > > The scripts used to support booting with kernel, initrd and rootfs.
> > > Enable it to support the IMAGE_TYPE 'wic-img' as well.
> > >
> > > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > > ---
> > > scripts/start_vm | 35 +++++++++++++++++++++++++----------
> > > 1 file changed, 25 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/scripts/start_vm b/scripts/start_vm
> > > index 6c89ce9..cfb2bea 100755
> > > --- a/scripts/start_vm
> > > +++ b/scripts/start_vm
> > > @@ -27,9 +27,6 @@ start_qemu() {
> > > -M $QEMU_MACHINE \
> > > $QCPU \
> > > -nographic \
> > > - -kernel $QKERNEL \
> > > - -initrd $QINITRD \
> > > - -append "console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV rw"
> > > \ $EXTRA_ARGS \
> > > $root
> > > [ $had_x -eq 0 ] && set +x
> > > @@ -104,13 +101,6 @@ do
> > > done
> > >
> > > readonly IMAGE_DIR=$BUILD_DIR/tmp/deploy/images
> > > -readonly
> > > ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img -
> > > -eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base |
> > > grep "^KERNEL_IMAGE=") -eval $(bitbake -e
> > > multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep
> > > "^INITRD_IMAGE=") -QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> > > -QINITRD=/dev/null -[ -n "$INITRD_IMAGE" ] &&
> > > QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
> > > readonly ISARROOT="$(dirname "$0")"/..
> > >
> > > @@ -120,6 +110,31 @@ eval "$(egrep 'MACHINE_SERIAL' $MACHINE_CONF
> > > |bb2sh)" readonly
> > > CONFIG_CONF=$ISARROOT/meta-isar/conf/multiconfig/qemu$ARCH-$DISTRO.conf
> > > eval "$(egrep 'ROOTFS_DEV|QEMU_' $CONFIG_CONF |bb2sh)"
> > > +eval $(bitbake -e multiconfig:qemu$ARCH-$DISTRO:isar-image-base |
> > > grep "^IMAGE_TYPE=") +case "$IMAGE_TYPE" in
> > > + ext4-img)
> > > + readonly
> > > ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img +
> > > + eval $(bitbake -e
> > > multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep
> > > "^KERNEL_IMAGE=")
> > > + eval $(bitbake -e
> > > multiconfig:qemu$ARCH-$DISTRO:isar-image-base | grep
> > > "^INITRD_IMAGE=")
> > > + QKERNEL=$IMAGE_DIR/${KERNEL_IMAGE}
> > > + QINITRD=/dev/null
> > > + [ -n "$INITRD_IMAGE" ] && QINITRD=$IMAGE_DIR/${INITRD_IMAGE}
> > > + EXTRA_ARGS="$EXTRA_ARGS \
> > > + -kernel $QKERNEL \
> > > + -initrd $QINITRD \
> > > + -append \"console=$MACHINE_SERIAL root=/dev/$ROOTFS_DEV
> > > rw\""
> > > + ;;
> > > + wic-img)
> > > + readonly
> > > ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.wic.img
> > > + EXTRA_ARGS="$EXTRA_ARGS -snapshot"
> > > + ;;
> > > + *)
> > > + echo "IMAGE_TYPE \"$IMAGE_TYPE\" not supported"
> > > + exit 1
> > > + ;;
> > > +esac
> >
> > Here we have the same problem, that only one image type can be set.
>
> Yes, but as already discussed it is no array in Isar.
>
There never are arrays in bitbake, but space separated items in a
string.
But one could put a bash for loop around the case like
for imagetype in $IMAGE_TYPE; do
case ...
...
esac
done
> Henning
>
> > > +
> > > QCPU=
> > > [ -n "$QEMU_CPU" ] && QCPU="-cpu $QEMU_CPU"
> > >
> > > --
> > > 2.16.1
> > >
> >
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 14/17] multiconfig: Switch qemuamd64-stretch to using wic by default
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (12 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 13/17] scripts/start_vm: Enable booting of full disk images Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-16 10:25 ` Claudius Heine
2018-04-13 14:19 ` [PATCH v2 15/17] multiconfig: Switch qemui386-stretch " Henning Schild
` (4 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This will now create a testcase for CI and an example for users. This
uses the EFI WKS_FILE.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
index 64a62a1..356d59b 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
@@ -10,9 +10,11 @@ KERNEL_NAME ?= "amd64"
IMAGE_PREINSTALL += "init"
-ROOTFS_DEV ?= "sda"
+IMAGE_TYPE ?= "wic-img"
+
+ROOTFS_DEV ?= "sda2"
QEMU_ARCH ?= "x86_64"
-QEMU_MACHINE ?= "pc"
+QEMU_MACHINE ?= "q35"
QEMU_CPU ?= ""
-QEMU_ROOTFS_DEV ?= "-hda ##ROOTFS_IMAGE##"
+QEMU_ROOTFS_DEV ?= "-hda ##ROOTFS_IMAGE## -bios /usr/local/share/ovmf/OVMF.fd"
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 14/17] multiconfig: Switch qemuamd64-stretch to using wic by default
2018-04-13 14:19 ` [PATCH v2 14/17] multiconfig: Switch qemuamd64-stretch to using wic by default Henning Schild
@ 2018-04-16 10:25 ` Claudius Heine
2018-04-16 10:32 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Claudius Heine @ 2018-04-16 10:25 UTC (permalink / raw)
To: [ext] Henning Schild, isar-users; +Cc: Cedric Hombourger, Andreas Reichel
Hi Henning,
On 2018-04-13 16:19, [ext] Henning Schild wrote:
> This will now create a testcase for CI and an example for users. This
> uses the EFI WKS_FILE.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
> index 64a62a1..356d59b 100644
> --- a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
> +++ b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
> @@ -10,9 +10,11 @@ KERNEL_NAME ?= "amd64"
>
> IMAGE_PREINSTALL += "init"
>
> -ROOTFS_DEV ?= "sda"
> +IMAGE_TYPE ?= "wic-img"
> +
> +ROOTFS_DEV ?= "sda2"
I know that is not exactly the scope of your patchset, but does it make
sense to set the ROOTFS_DEV in a multiconf rather in the machine.conf or
inside the image recipe?
regards,
Claudius
>
> QEMU_ARCH ?= "x86_64"
> -QEMU_MACHINE ?= "pc"
> +QEMU_MACHINE ?= "q35"
> QEMU_CPU ?= ""
> -QEMU_ROOTFS_DEV ?= "-hda ##ROOTFS_IMAGE##"
> +QEMU_ROOTFS_DEV ?= "-hda ##ROOTFS_IMAGE## -bios /usr/local/share/ovmf/OVMF.fd"
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 14/17] multiconfig: Switch qemuamd64-stretch to using wic by default
2018-04-16 10:25 ` Claudius Heine
@ 2018-04-16 10:32 ` Henning Schild
0 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-16 10:32 UTC (permalink / raw)
To: Claudius Heine; +Cc: isar-users, Cedric Hombourger, Andreas Reichel
Am Mon, 16 Apr 2018 12:25:54 +0200
schrieb Claudius Heine <claudius.heine.ext@siemens.com>:
> Hi Henning,
>
> On 2018-04-13 16:19, [ext] Henning Schild wrote:
> > This will now create a testcase for CI and an example for users.
> > This uses the EFI WKS_FILE.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > meta-isar/conf/multiconfig/qemuamd64-stretch.conf | 8 +++++---
> > 1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf
> > b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf index
> > 64a62a1..356d59b 100644 ---
> > a/meta-isar/conf/multiconfig/qemuamd64-stretch.conf +++
> > b/meta-isar/conf/multiconfig/qemuamd64-stretch.conf @@ -10,9 +10,11
> > @@ KERNEL_NAME ?= "amd64"
> > IMAGE_PREINSTALL += "init"
> >
> > -ROOTFS_DEV ?= "sda"
> > +IMAGE_TYPE ?= "wic-img"
> > +
> > +ROOTFS_DEV ?= "sda2"
>
> I know that is not exactly the scope of your patchset, but does it
> make sense to set the ROOTFS_DEV in a multiconf rather in the
> machine.conf or inside the image recipe?
In fact that should not be required when using wic, same for
ROOTFS_TYPE ?= "ext4" found in meta/classes/wic-img.bbclass.
Problem is one of the scripts messing with fstab will create a broken
fstab if those values are not in sync with the .wks. On the wic-path
the messing with fstab should not be done at all. For now keeping the
two in sync was what i chose.
Henning
> regards,
> Claudius
>
> >
> > QEMU_ARCH ?= "x86_64"
> > -QEMU_MACHINE ?= "pc"
> > +QEMU_MACHINE ?= "q35"
> > QEMU_CPU ?= ""
> > -QEMU_ROOTFS_DEV ?= "-hda ##ROOTFS_IMAGE##"
> > +QEMU_ROOTFS_DEV ?= "-hda ##ROOTFS_IMAGE##
> > -bios /usr/local/share/ovmf/OVMF.fd"
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 15/17] multiconfig: Switch qemui386-stretch to using wic by default
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (13 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 14/17] multiconfig: Switch qemuamd64-stretch to using wic by default Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-16 10:35 ` Andreas Reichel
2018-04-13 14:19 ` [PATCH v2 16/17] docs: Change according to recent patches Henning Schild
` (3 subsequent siblings)
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
This time the non-EFI variant.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
meta-isar/conf/multiconfig/qemui386-stretch.conf | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/meta-isar/conf/multiconfig/qemui386-stretch.conf b/meta-isar/conf/multiconfig/qemui386-stretch.conf
index 0ba7c5b..8966c2b 100644
--- a/meta-isar/conf/multiconfig/qemui386-stretch.conf
+++ b/meta-isar/conf/multiconfig/qemui386-stretch.conf
@@ -10,7 +10,10 @@ KERNEL_NAME ?= "686-pae"
IMAGE_PREINSTALL += "init"
-ROOTFS_DEV ?= "sda"
+IMAGE_TYPE ?= "wic-img"
+WKS_FILE ?= "directdisk-isar"
+
+ROOTFS_DEV ?= "sda2"
QEMU_ARCH ?= "i386"
QEMU_MACHINE ?= "pc"
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 15/17] multiconfig: Switch qemui386-stretch to using wic by default
2018-04-13 14:19 ` [PATCH v2 15/17] multiconfig: Switch qemui386-stretch " Henning Schild
@ 2018-04-16 10:35 ` Andreas Reichel
2018-04-16 10:43 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:35 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Fri, Apr 13, 2018 at 04:19:04PM +0200, Henning Schild wrote:
> This time the non-EFI variant.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> meta-isar/conf/multiconfig/qemui386-stretch.conf | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/meta-isar/conf/multiconfig/qemui386-stretch.conf b/meta-isar/conf/multiconfig/qemui386-stretch.conf
> index 0ba7c5b..8966c2b 100644
> --- a/meta-isar/conf/multiconfig/qemui386-stretch.conf
> +++ b/meta-isar/conf/multiconfig/qemui386-stretch.conf
> @@ -10,7 +10,10 @@ KERNEL_NAME ?= "686-pae"
>
> IMAGE_PREINSTALL += "init"
>
> -ROOTFS_DEV ?= "sda"
> +IMAGE_TYPE ?= "wic-img"
> +WKS_FILE ?= "directdisk-isar"
> +
> +ROOTFS_DEV ?= "sda2"
Note:
Generally, I suggest we don't hardcode device names but use UUIDs and
PARTUUIDs. We already have that problem on multiple platforms.
>
> QEMU_ARCH ?= "i386"
> QEMU_MACHINE ?= "pc"
> --
> 2.16.1
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 15/17] multiconfig: Switch qemui386-stretch to using wic by default
2018-04-16 10:35 ` Andreas Reichel
@ 2018-04-16 10:43 ` Henning Schild
2018-04-16 10:51 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-16 10:43 UTC (permalink / raw)
To: Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Am Mon, 16 Apr 2018 12:35:54 +0200
schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
> On Fri, Apr 13, 2018 at 04:19:04PM +0200, Henning Schild wrote:
> > This time the non-EFI variant.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > meta-isar/conf/multiconfig/qemui386-stretch.conf | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta-isar/conf/multiconfig/qemui386-stretch.conf
> > b/meta-isar/conf/multiconfig/qemui386-stretch.conf index
> > 0ba7c5b..8966c2b 100644 ---
> > a/meta-isar/conf/multiconfig/qemui386-stretch.conf +++
> > b/meta-isar/conf/multiconfig/qemui386-stretch.conf @@ -10,7 +10,10
> > @@ KERNEL_NAME ?= "686-pae"
> > IMAGE_PREINSTALL += "init"
> >
> > -ROOTFS_DEV ?= "sda"
> > +IMAGE_TYPE ?= "wic-img"
> > +WKS_FILE ?= "directdisk-isar"
> > +
> > +ROOTFS_DEV ?= "sda2"
>
> Note:
> Generally, I suggest we don't hardcode device names but use UUIDs and
> PARTUUIDs. We already have that problem on multiple platforms.
See the reply i gave Claudius. This is just here to make the fstab-code
happy. Code that should not even be executed when using wic, because
then wic could do it right. If an fstab-entry exists wic will not touch
it, so we need to make sure it is correct.
Henning
> >
> > QEMU_ARCH ?= "i386"
> > QEMU_MACHINE ?= "pc"
> > --
> > 2.16.1
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 15/17] multiconfig: Switch qemui386-stretch to using wic by default
2018-04-16 10:43 ` Henning Schild
@ 2018-04-16 10:51 ` Henning Schild
2018-04-16 10:56 ` Claudius Heine
0 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-16 10:51 UTC (permalink / raw)
To: Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Am Mon, 16 Apr 2018 12:43:41 +0200
schrieb "[ext] Henning Schild" <henning.schild@siemens.com>:
> Am Mon, 16 Apr 2018 12:35:54 +0200
> schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
>
> > On Fri, Apr 13, 2018 at 04:19:04PM +0200, Henning Schild wrote:
> > > This time the non-EFI variant.
> > >
> > > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > > ---
> > > meta-isar/conf/multiconfig/qemui386-stretch.conf | 5 ++++-
> > > 1 file changed, 4 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/meta-isar/conf/multiconfig/qemui386-stretch.conf
> > > b/meta-isar/conf/multiconfig/qemui386-stretch.conf index
> > > 0ba7c5b..8966c2b 100644 ---
> > > a/meta-isar/conf/multiconfig/qemui386-stretch.conf +++
> > > b/meta-isar/conf/multiconfig/qemui386-stretch.conf @@ -10,7 +10,10
> > > @@ KERNEL_NAME ?= "686-pae"
> > > IMAGE_PREINSTALL += "init"
> > >
> > > -ROOTFS_DEV ?= "sda"
> > > +IMAGE_TYPE ?= "wic-img"
> > > +WKS_FILE ?= "directdisk-isar"
> > > +
> > > +ROOTFS_DEV ?= "sda2"
> >
> > Note:
> > Generally, I suggest we don't hardcode device names but use UUIDs
> > and PARTUUIDs. We already have that problem on multiple platforms.
I wanted to not grow this series even more. But i think it would be a
good idea to introduce a patch to not touch fstab in the configscripts.
Henning
> See the reply i gave Claudius. This is just here to make the
> fstab-code happy. Code that should not even be executed when using
> wic, because then wic could do it right. If an fstab-entry exists wic
> will not touch it, so we need to make sure it is correct.
>
> Henning
>
> > >
> > > QEMU_ARCH ?= "i386"
> > > QEMU_MACHINE ?= "pc"
> > > --
> > > 2.16.1
> > >
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 15/17] multiconfig: Switch qemui386-stretch to using wic by default
2018-04-16 10:51 ` Henning Schild
@ 2018-04-16 10:56 ` Claudius Heine
0 siblings, 0 replies; 47+ messages in thread
From: Claudius Heine @ 2018-04-16 10:56 UTC (permalink / raw)
To: [ext] Henning Schild, Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Hi,
On 2018-04-16 12:51, [ext] Henning Schild wrote:
> Am Mon, 16 Apr 2018 12:43:41 +0200
> schrieb "[ext] Henning Schild" <henning.schild@siemens.com>:
>
>> Am Mon, 16 Apr 2018 12:35:54 +0200
>> schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
>>
>>> On Fri, Apr 13, 2018 at 04:19:04PM +0200, Henning Schild wrote:
>>>> This time the non-EFI variant.
>>>>
>>>> Signed-off-by: Henning Schild <henning.schild@siemens.com>
>>>> ---
>>>> meta-isar/conf/multiconfig/qemui386-stretch.conf | 5 ++++-
>>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/meta-isar/conf/multiconfig/qemui386-stretch.conf
>>>> b/meta-isar/conf/multiconfig/qemui386-stretch.conf index
>>>> 0ba7c5b..8966c2b 100644 ---
>>>> a/meta-isar/conf/multiconfig/qemui386-stretch.conf +++
>>>> b/meta-isar/conf/multiconfig/qemui386-stretch.conf @@ -10,7 +10,10
>>>> @@ KERNEL_NAME ?= "686-pae"
>>>> IMAGE_PREINSTALL += "init"
>>>>
>>>> -ROOTFS_DEV ?= "sda"
>>>> +IMAGE_TYPE ?= "wic-img"
>>>> +WKS_FILE ?= "directdisk-isar"
>>>> +
>>>> +ROOTFS_DEV ?= "sda2"
>>>
>>> Note:
>>> Generally, I suggest we don't hardcode device names but use UUIDs
>>> and PARTUUIDs. We already have that problem on multiple platforms.
>
> I wanted to not grow this series even more. But i think it would be a
> good idea to introduce a patch to not touch fstab in the configscripts.
Yes I agree. As I said in the other discussion about the IMAGE_FEATURES
to Baurzhan, in my opinion the fstab should be rather done in a setup
package. Inclusion of this package is then controlled over the image recipe.
Claudius
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
^ permalink raw reply [flat|nested] 47+ messages in thread
* [PATCH v2 16/17] docs: Change according to recent patches
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (14 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 15/17] multiconfig: Switch qemui386-stretch " Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-13 14:19 ` [PATCH v2 17/17] scripts/vm_smoke_test: double the timeout we wait for qemus Henning Schild
` (2 subsequent siblings)
18 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
Update the section on wic to tell people how to build images using it.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
doc/technical_overview.md | 7 ++++---
doc/user_manual.md | 25 ++++++++++++-------------
2 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index 83624a5..ab62de9 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -239,10 +239,11 @@ variable. Task `do_populate` performs the following:
4. Execute dpkg command in chroot for all the copied packages.
-## 3.7 Generate Bootable Image
+## 3.7 Generate an Image
This process contains the following steps:
-1. Task `do_ext4_image`: target filesystem is packed to extfs image.
+1. Task `do_<..>_image`: creates filesystem or disk images
-2. wic tool generates bootable image for dedicated platform.
+ 1.1. `do_ext4_image`: target filesystem is packed to extfs image.
+ 1.2. `do_wic_image`: a bootable disk image gets created for the platform
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 7bd52f4..7bdfc1c 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -190,34 +190,33 @@ Created images are:
../build-2/tmp/deploy/images/isar-image-base-debian-jessie-qemuarm.ext4.img
../build-3/tmp/deploy/images/isar-image-base-debian-stretch-qemuarm.ext4.img
../build-4/tmp/deploy/images/isar-image-base-debian-jessie-qemui386.ext4.img
-../build-5/tmp/deploy/images/isar-image-base-debian-stretch-qemui386.ext4.img
+../build-5/tmp/deploy/images/isar-image-base-debian-stretch-qemui386.wic.img
../build-6/tmp/deploy/images/isar-image-base-debian-jessie-qemuamd64.ext4.img
-../build-7/tmp/deploy/images/isar-image-base-debian-stretch-qemuamd64.ext4.img
+../build-7/tmp/deploy/images/isar-image-base-debian-stretch-qemuamd64.wic.img
../build-8/tmp/deploy/images/isar-image-base.rpi-sdimg
```
-### Generate EFI disk images
+### Generate full disk image
-Once the image artifacts have been built (c.f. previous section), full EFI disk images can be generated using the `wic` utility.
-Currently, only the `i386` and `amd64` target architectures are supported:
+A bootable disk image is generated if you set IMAGE_TYPE to 'wic-img'. Behind the scenes a tool called `wic` is used to assemble the images. It is controlled by a `.wks` file which you can choose with changing WKS_FILE. Some examples in the tree use that feature already.
```
- # Generate an EFI image for the `i386` target architecture
- $ wic create -D sdimage-efi -o . -e multiconfig:qemui386-stretch:isar-image-base
- # Similarly, for the `amd64` target architecture
- $ wic create -D sdimage-efi -o . -e multiconfig:qemuamd64-stretch:isar-image-base
+ # Generate an image for the `i386` target architecture
+ $ bitbake multiconfig:qemui386-stretch:isar-image-base
+ # Similarly, for the `amd64` target architecture, in this case EFI
+ $ bitbake multiconfig:qemuamd64-stretch:isar-image-base
```
-In order to run the images with `qemu`, an EFI firmware is required and available at the following address:
+In order to run the EFI images with `qemu`, an EFI firmware is required and available at the following address:
https://github.com/tianocore/edk2/tree/3858b4a1ff09d3243fea8d07bd135478237cb8f7
Note that the `ovmf` package in Debian jessie/stretch/sid contains a pre-compiled firmware, but doesn't seem to be recent
enough to allow images to be testable under `qemu`.
```
-# AMD64 image
-qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF.fd -hda ./sdimage-*
+# AMD64 image, EFI
+qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF.fd -hda tmp/deploy/images/isar-image-base-debian-stretch-qemuamd64.wic.img
# i386 image
-qemu-system-i386 -m 256M -nographic -bios edk2/Build/OvmfIa32/RELEASE_*/FV/OVMF.fd -hda ./sdimage-*
+qemu-system-i386 -m 256M -nographic -hda tmp/deploy/images/isar-image-base-debian-stretch-qemui386.wic.img
```
---
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* [PATCH v2 17/17] scripts/vm_smoke_test: double the timeout we wait for qemus
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (15 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 16/17] docs: Change according to recent patches Henning Schild
@ 2018-04-13 14:19 ` Henning Schild
2018-04-16 10:47 ` Andreas Reichel
2018-04-25 15:53 ` [PATCH v2 00/16] wic integration Alexander Smirnov
2018-05-01 19:23 ` Alexander Smirnov
18 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-04-13 14:19 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger, Andreas Reichel, Henning Schild
If the CI server is busy or our image boots slowly we might kill qemu
too early. Wait a little longer, will not make CI much slower.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
scripts/vm_smoke_test | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
index fde0e05..274ec1d 100755
--- a/scripts/vm_smoke_test
+++ b/scripts/vm_smoke_test
@@ -51,7 +51,7 @@ run_test () {
# Start QEMU with Isar image
start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE > /dev/null 2>&1 &
- sleep 30
+ sleep 60
kill `cat $PID_FILE`
# Check login prompt
--
2.16.1
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 17/17] scripts/vm_smoke_test: double the timeout we wait for qemus
2018-04-13 14:19 ` [PATCH v2 17/17] scripts/vm_smoke_test: double the timeout we wait for qemus Henning Schild
@ 2018-04-16 10:47 ` Andreas Reichel
2018-04-16 11:45 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Andreas Reichel @ 2018-04-16 10:47 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Cedric Hombourger
On Fri, Apr 13, 2018 at 04:19:06PM +0200, Henning Schild wrote:
> If the CI server is busy or our image boots slowly we might kill qemu
> too early. Wait a little longer, will not make CI much slower.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> scripts/vm_smoke_test | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
> index fde0e05..274ec1d 100755
> --- a/scripts/vm_smoke_test
> +++ b/scripts/vm_smoke_test
> @@ -51,7 +51,7 @@ run_test () {
>
> # Start QEMU with Isar image
> start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE > /dev/null 2>&1 &
> - sleep 30
> + sleep 60
Just an idea without knowing what start_vm actually is: Why not working
with the console output and wait for a login prompt?
> kill `cat $PID_FILE`
>
> # Check login prompt
> --
> 2.16.1
>
--
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant
Andreas.Reichel@tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 17/17] scripts/vm_smoke_test: double the timeout we wait for qemus
2018-04-16 10:47 ` Andreas Reichel
@ 2018-04-16 11:45 ` Henning Schild
0 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-16 11:45 UTC (permalink / raw)
To: Andreas Reichel; +Cc: isar-users, Cedric Hombourger
Am Mon, 16 Apr 2018 12:47:57 +0200
schrieb Andreas Reichel <andreas.reichel.ext@siemens.com>:
> On Fri, Apr 13, 2018 at 04:19:06PM +0200, Henning Schild wrote:
> > If the CI server is busy or our image boots slowly we might kill
> > qemu too early. Wait a little longer, will not make CI much slower.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> > scripts/vm_smoke_test | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/vm_smoke_test b/scripts/vm_smoke_test
> > index fde0e05..274ec1d 100755
> > --- a/scripts/vm_smoke_test
> > +++ b/scripts/vm_smoke_test
> > @@ -51,7 +51,7 @@ run_test () {
> >
> > # Start QEMU with Isar image
> > start_vm -a $ARCH -d $DISTRO -o $CONSOLE_OUTPUT -p $PID_FILE
> > > /dev/null 2>&1 &
> > - sleep 30
> > + sleep 60
> Just an idea without knowing what start_vm actually is: Why not
> working with the console output and wait for a login prompt?
That is exactly what it is doing. But at some point you will have to
decide that the image does not boot and that prompt will never come.
And that is this sleep.
Henning
> > kill `cat $PID_FILE`
> >
> > # Check login prompt
> > --
> > 2.16.1
> >
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 00/16] wic integration
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (16 preceding siblings ...)
2018-04-13 14:19 ` [PATCH v2 17/17] scripts/vm_smoke_test: double the timeout we wait for qemus Henning Schild
@ 2018-04-25 15:53 ` Alexander Smirnov
2018-04-26 11:26 ` Henning Schild
2018-05-01 19:23 ` Alexander Smirnov
18 siblings, 1 reply; 47+ messages in thread
From: Alexander Smirnov @ 2018-04-25 15:53 UTC (permalink / raw)
To: Henning Schild, isar-users
Hi Henning,
On 04/13/2018 05:18 PM, Henning Schild wrote:
> This series does the follow things:
>
> 1. revert all changes that where made to wic so Isar uses an unmodified
> version of it
> 2. Introdcue an image-class that will create wic images automatically
> 3. enable testing that in CI
> 4. update docs
>
> Patches 1 to 7 just deal with getting back to an unmodified version of
> wic.
> Patch 11 - the big one - provides the real functionality.
>
> I will provide some more information on what is going on here by
> replying to p11.
>
> Major changes to the first series:
> - rebase on recent Isar
> - wic is now running in buildchroot, as root
> - calling wic manually not supported any longer
> - distros before stretch are not supported either (see p11 discussion
> for why)
> - CI can now test wic for the first time
Thank you for this contribution!
For now I have one issue with it, after applying patch #13, the
'start_vm' script doesn't work anymore:
$ start_vm -a arm -d jessie
++ qemu-system-arm -m 1024M -M virt -cpu cortex-a15 -nographic -kernel
/home/builder/isar/build/tmp/deploy/images/vmlinuz-3.16.0-5-armmp_debian-jessie-qemuarm
-initrd
/home/builder/isar/build/tmp/deploy/images/initrd.img-3.16.0-5-armmp_debian-jessie-qemuarm
-append '"console=ttyAMA0' root=/dev/vda 'rw"' -drive
file=/home/builder/isar/build/tmp/deploy/images/isar-image-base-debian-jessie-qemuarm.ext4.img,if=none,format=raw,id=hd0
-device virtio-blk-device,drive=hd0
qemu-system-arm: -append "console=ttyAMA0: Could not open
'root=/dev/vda': No such file or directory
I suppose this is a minor issue with arguments, will come back with updates.
Alex
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 00/16] wic integration
2018-04-25 15:53 ` [PATCH v2 00/16] wic integration Alexander Smirnov
@ 2018-04-26 11:26 ` Henning Schild
0 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-04-26 11:26 UTC (permalink / raw)
To: Alexander Smirnov; +Cc: isar-users
Am Wed, 25 Apr 2018 18:53:04 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> Hi Henning,
>
> On 04/13/2018 05:18 PM, Henning Schild wrote:
> > This series does the follow things:
> >
> > 1. revert all changes that where made to wic so Isar uses an
> > unmodified version of it
> > 2. Introdcue an image-class that will create wic images
> > automatically 3. enable testing that in CI
> > 4. update docs
> >
> > Patches 1 to 7 just deal with getting back to an unmodified version
> > of wic.
> > Patch 11 - the big one - provides the real functionality.
> >
> > I will provide some more information on what is going on here by
> > replying to p11.
> >
> > Major changes to the first series:
> > - rebase on recent Isar
> > - wic is now running in buildchroot, as root
> > - calling wic manually not supported any longer
> > - distros before stretch are not supported either (see p11
> > discussion for why)
> > - CI can now test wic for the first time
>
> Thank you for this contribution!
>
> For now I have one issue with it, after applying patch #13, the
> 'start_vm' script doesn't work anymore:
Ok, all of that actually worked in your CI. Except for one small
thing ... For some reason the qemu login prompt capturing fails for
the efi wic image, but the image boots and the prompt appears.
At that point i stopped because i guess the problem might be in the
qemu or something in the CI setup
see:
job/isar_henning_ilbers-ci/34/console
> $ start_vm -a arm -d jessie
> ++ qemu-system-arm -m 1024M -M virt -cpu cortex-a15 -nographic
> -kernel /home/builder/isar/build/tmp/deploy/images/vmlinuz-3.16.0-5-armmp_debian-jessie-qemuarm
> -initrd
> /home/builder/isar/build/tmp/deploy/images/initrd.img-3.16.0-5-armmp_debian-jessie-qemuarm
> -append '"console=ttyAMA0' root=/dev/vda 'rw"' -drive
The quoting looks weird. My guess is that /bin/sh on that system is not
the same as on the CI. For the testing i did locally it was a "dash"
from jessie.
Henning
> file=/home/builder/isar/build/tmp/deploy/images/isar-image-base-debian-jessie-qemuarm.ext4.img,if=none,format=raw,id=hd0
> -device virtio-blk-device,drive=hd0
> qemu-system-arm: -append "console=ttyAMA0: Could not open
> 'root=/dev/vda': No such file or directory
>
>
> I suppose this is a minor issue with arguments, will come back with
> updates.
>
> Alex
>
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 00/16] wic integration
2018-04-13 14:18 [PATCH v2 00/16] wic integration Henning Schild
` (17 preceding siblings ...)
2018-04-25 15:53 ` [PATCH v2 00/16] wic integration Alexander Smirnov
@ 2018-05-01 19:23 ` Alexander Smirnov
2018-05-03 16:32 ` Henning Schild
18 siblings, 1 reply; 47+ messages in thread
From: Alexander Smirnov @ 2018-05-01 19:23 UTC (permalink / raw)
To: Henning Schild, isar-users
On 04/13/2018 05:18 PM, Henning Schild wrote:
> This series does the follow things:
>
> 1. revert all changes that where made to wic so Isar uses an unmodified
> version of it
> 2. Introdcue an image-class that will create wic images automatically
> 3. enable testing that in CI
> 4. update docs
>
> Patches 1 to 7 just deal with getting back to an unmodified version of
> wic.
> Patch 11 - the big one - provides the real functionality.
>
> I will provide some more information on what is going on here by
> replying to p11.
>
> Major changes to the first series:
> - rebase on recent Isar
> - wic is now running in buildchroot, as root
> - calling wic manually not supported any longer
> - distros before stretch are not supported either (see p11 discussion
> for why)
> - CI can now test wic for the first time
Applied to next, thanks!
Alex
^ permalink raw reply [flat|nested] 47+ messages in thread* Re: [PATCH v2 00/16] wic integration
2018-05-01 19:23 ` Alexander Smirnov
@ 2018-05-03 16:32 ` Henning Schild
2018-05-03 16:39 ` Alexander Smirnov
0 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-05-03 16:32 UTC (permalink / raw)
To: Alexander Smirnov; +Cc: isar-users
Am Tue, 1 May 2018 22:23:01 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> On 04/13/2018 05:18 PM, Henning Schild wrote:
> > This series does the follow things:
> >
> > 1. revert all changes that where made to wic so Isar uses an
> > unmodified version of it
> > 2. Introdcue an image-class that will create wic images
> > automatically 3. enable testing that in CI
> > 4. update docs
> >
> > Patches 1 to 7 just deal with getting back to an unmodified version
> > of wic.
> > Patch 11 - the big one - provides the real functionality.
> >
> > I will provide some more information on what is going on here by
> > replying to p11.
> >
> > Major changes to the first series:
> > - rebase on recent Isar
> > - wic is now running in buildchroot, as root
> > - calling wic manually not supported any longer
> > - distros before stretch are not supported either (see p11
> > discussion for why)
> > - CI can now test wic for the first time
>
> Applied to next, thanks!
Thank you. I have one outstanding change i wanted to squash with p11,
originated from a review from Claudius. Should that get squashed or go
on top?
https://github.com/henning-schild-work/isar/commit/75bbea59fe2e4174f4eaf0f782792d208672fad6
Henning
> Alex
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 00/16] wic integration
2018-05-03 16:32 ` Henning Schild
@ 2018-05-03 16:39 ` Alexander Smirnov
2018-05-04 8:16 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Alexander Smirnov @ 2018-05-03 16:39 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users
On 05/03/2018 07:32 PM, Henning Schild wrote:
> Am Tue, 1 May 2018 22:23:01 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
>
>> On 04/13/2018 05:18 PM, Henning Schild wrote:
>>> This series does the follow things:
>>>
>>> 1. revert all changes that where made to wic so Isar uses an
>>> unmodified version of it
>>> 2. Introdcue an image-class that will create wic images
>>> automatically 3. enable testing that in CI
>>> 4. update docs
>>>
>>> Patches 1 to 7 just deal with getting back to an unmodified version
>>> of wic.
>>> Patch 11 - the big one - provides the real functionality.
>>>
>>> I will provide some more information on what is going on here by
>>> replying to p11.
>>>
>>> Major changes to the first series:
>>> - rebase on recent Isar
>>> - wic is now running in buildchroot, as root
>>> - calling wic manually not supported any longer
>>> - distros before stretch are not supported either (see p11
>>> discussion for why)
>>> - CI can now test wic for the first time
>>
>> Applied to next, thanks!
>
> Thank you. I have one outstanding change i wanted to squash with p11,
> originated from a review from Claudius. Should that get squashed or go
> on top?
> https://github.com/henning-schild-work/isar/commit/75bbea59fe2e4174f4eaf0f782792d208672fad6
>
I'd prefer on top, could you please send this patch on the list?
Alex
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 00/16] wic integration
2018-05-03 16:39 ` Alexander Smirnov
@ 2018-05-04 8:16 ` Henning Schild
2018-05-04 8:32 ` Alexander Smirnov
0 siblings, 1 reply; 47+ messages in thread
From: Henning Schild @ 2018-05-04 8:16 UTC (permalink / raw)
To: Alexander Smirnov; +Cc: isar-users, Claudius Heine
Am Thu, 3 May 2018 19:39:39 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> On 05/03/2018 07:32 PM, Henning Schild wrote:
> > Am Tue, 1 May 2018 22:23:01 +0300
> > schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> >
> >> On 04/13/2018 05:18 PM, Henning Schild wrote:
> >>> This series does the follow things:
> >>>
> >>> 1. revert all changes that where made to wic so Isar uses an
> >>> unmodified version of it
> >>> 2. Introdcue an image-class that will create wic images
> >>> automatically 3. enable testing that in CI
> >>> 4. update docs
> >>>
> >>> Patches 1 to 7 just deal with getting back to an unmodified
> >>> version of wic.
> >>> Patch 11 - the big one - provides the real functionality.
> >>>
> >>> I will provide some more information on what is going on here by
> >>> replying to p11.
> >>>
> >>> Major changes to the first series:
> >>> - rebase on recent Isar
> >>> - wic is now running in buildchroot, as root
> >>> - calling wic manually not supported any longer
> >>> - distros before stretch are not supported either (see p11
> >>> discussion for why)
> >>> - CI can now test wic for the first time
> >>
> >> Applied to next, thanks!
> >
> > Thank you. I have one outstanding change i wanted to squash with
> > p11, originated from a review from Claudius. Should that get
> > squashed or go on top?
> > https://github.com/henning-schild-work/isar/commit/75bbea59fe2e4174f4eaf0f782792d208672fad6
> >
>
> I'd prefer on top, could you please send this patch on the list?
And now i remember another "open" point. Claudius suggested to split
the forking of the plugins into two steps. One that just copies them,
and a second to apply the Isar-specific changes.
That would make it easier to see what needed to change to make them
work under Isar, which could become increasingly interesting when we
version-bump our wic some day.
I could do that on top as well, sort of two no-op patches. But in this
case a rebase would probably be the better option. Or we do not do that
at all. How would you like to proceed?
Henning
> Alex
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 00/16] wic integration
2018-05-04 8:16 ` Henning Schild
@ 2018-05-04 8:32 ` Alexander Smirnov
2018-05-04 9:30 ` Henning Schild
0 siblings, 1 reply; 47+ messages in thread
From: Alexander Smirnov @ 2018-05-04 8:32 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, Claudius Heine
Hi,
On 05/04/2018 11:16 AM, Henning Schild wrote:
> Am Thu, 3 May 2018 19:39:39 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
>
>> On 05/03/2018 07:32 PM, Henning Schild wrote:
>>> Am Tue, 1 May 2018 22:23:01 +0300
>>> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
>>>
>>>> On 04/13/2018 05:18 PM, Henning Schild wrote:
>>>>> This series does the follow things:
>>>>>
>>>>> 1. revert all changes that where made to wic so Isar uses an
>>>>> unmodified version of it
>>>>> 2. Introdcue an image-class that will create wic images
>>>>> automatically 3. enable testing that in CI
>>>>> 4. update docs
>>>>>
>>>>> Patches 1 to 7 just deal with getting back to an unmodified
>>>>> version of wic.
>>>>> Patch 11 - the big one - provides the real functionality.
>>>>>
>>>>> I will provide some more information on what is going on here by
>>>>> replying to p11.
>>>>>
>>>>> Major changes to the first series:
>>>>> - rebase on recent Isar
>>>>> - wic is now running in buildchroot, as root
>>>>> - calling wic manually not supported any longer
>>>>> - distros before stretch are not supported either (see p11
>>>>> discussion for why)
>>>>> - CI can now test wic for the first time
>>>>
>>>> Applied to next, thanks!
>>>
>>> Thank you. I have one outstanding change i wanted to squash with
>>> p11, originated from a review from Claudius. Should that get
>>> squashed or go on top?
>>> https://github.com/henning-schild-work/isar/commit/75bbea59fe2e4174f4eaf0f782792d208672fad6
>>>
>>
>> I'd prefer on top, could you please send this patch on the list?
>
> And now i remember another "open" point. Claudius suggested to split
> the forking of the plugins into two steps. One that just copies them,
> and a second to apply the Isar-specific changes.
> That would make it easier to see what needed to change to make them
> work under Isar, which could become increasingly interesting when we
> version-bump our wic some day.
>
> I could do that on top as well, sort of two no-op patches. But in this
> case a rebase would probably be the better option. Or we do not do that
> at all. How would you like to proceed?
>
If you specify the commit to drop and provide two patches, I could force
the history.
But in general I'd prefer updates-on-top, because this way has lower
probability of mistake. I know that current head is fully functional and
incrementally tested. If something is updated in the middle of history -
I have to check that this doesn't corrupt patches above.
Alex
^ permalink raw reply [flat|nested] 47+ messages in thread
* Re: [PATCH v2 00/16] wic integration
2018-05-04 8:32 ` Alexander Smirnov
@ 2018-05-04 9:30 ` Henning Schild
0 siblings, 0 replies; 47+ messages in thread
From: Henning Schild @ 2018-05-04 9:30 UTC (permalink / raw)
To: Alexander Smirnov; +Cc: isar-users, Claudius Heine
Am Fri, 4 May 2018 11:32:20 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> Hi,
>
> On 05/04/2018 11:16 AM, Henning Schild wrote:
> > Am Thu, 3 May 2018 19:39:39 +0300
> > schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> >
> >> On 05/03/2018 07:32 PM, Henning Schild wrote:
> >>> Am Tue, 1 May 2018 22:23:01 +0300
> >>> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> >>>
> >>>> On 04/13/2018 05:18 PM, Henning Schild wrote:
> >>>>> This series does the follow things:
> >>>>>
> >>>>> 1. revert all changes that where made to wic so Isar uses an
> >>>>> unmodified version of it
> >>>>> 2. Introdcue an image-class that will create wic images
> >>>>> automatically 3. enable testing that in CI
> >>>>> 4. update docs
> >>>>>
> >>>>> Patches 1 to 7 just deal with getting back to an unmodified
> >>>>> version of wic.
> >>>>> Patch 11 - the big one - provides the real functionality.
> >>>>>
> >>>>> I will provide some more information on what is going on here by
> >>>>> replying to p11.
> >>>>>
> >>>>> Major changes to the first series:
> >>>>> - rebase on recent Isar
> >>>>> - wic is now running in buildchroot, as root
> >>>>> - calling wic manually not supported any longer
> >>>>> - distros before stretch are not supported either (see p11
> >>>>> discussion for why)
> >>>>> - CI can now test wic for the first time
> >>>>
> >>>> Applied to next, thanks!
> >>>
> >>> Thank you. I have one outstanding change i wanted to squash with
> >>> p11, originated from a review from Claudius. Should that get
> >>> squashed or go on top?
> >>> https://github.com/henning-schild-work/isar/commit/75bbea59fe2e4174f4eaf0f782792d208672fad6
> >>>
> >>
> >> I'd prefer on top, could you please send this patch on the list?
> >
> > And now i remember another "open" point. Claudius suggested to split
> > the forking of the plugins into two steps. One that just copies
> > them, and a second to apply the Isar-specific changes.
> > That would make it easier to see what needed to change to make them
> > work under Isar, which could become increasingly interesting when we
> > version-bump our wic some day.
> >
> > I could do that on top as well, sort of two no-op patches. But in
> > this case a rebase would probably be the better option. Or we do
> > not do that at all. How would you like to proceed?
> >
>
> If you specify the commit to drop and provide two patches, I could
> force the history.
>
> But in general I'd prefer updates-on-top, because this way has lower
> probability of mistake. I know that current head is fully functional
> and incrementally tested. If something is updated in the middle of
> history - I have to check that this doesn't corrupt patches above.
I just sent two patches on top. Applying both is a no-op and they might
help understand what is Isar-specific in the forked files.
Henning
> Alex
^ permalink raw reply [flat|nested] 47+ messages in thread