[PATCH v2 0/6] Add support for dual bios + efi disk generation

[PATCH v2 0/6] Add support for dual bios + efi disk generation

[Felix Moessbauer]

This patch ports the bios + efi disk support from OE to ISAR.
By that, the generated disk can be booted both with EFI as well
as with PCBIOS.

Please note, that patches 2 and 3 do not work independently,
but are split for easier refactoring of the isarpluginsbase class.

Changes since v1:

- minimize changes to bootimg-biosplusefi-isar.py
- replace erronous commit message of hennings commit
- rework efi-plus-pcbios WKS file to not include any bios-only options
- update start_vm script to support modern debian distros
- test boot of both efi and bios of efi + bios test target

Best regards,
Felix

Felix Moessbauer (5):
  Add ISAR version of biosplusefi WIC plugin
  test: add target for efi-plus-pcbios image type
  add support for current debian distros in start_vm.py
  Add pcbios option to start_vm
  run vm_boot_test against EFI and PC BIOS

Henning Schild (1):
  wic: make import isarpluginbase work when plugins use each other

 meta-isar/conf/multiconfig/qemuamd64-buster.conf   |  2 ++
 .../scripts/lib/wic/canned-wks/efi-plus-pcbios.wks |  6 ++++++
 .../wic/plugins/source/bootimg-biosplusefi-isar.py |  8 ++++----
 .../lib/wic/plugins/source/bootimg-efi-isar.py     |  6 ++++--
 .../lib/wic/plugins/source/bootimg-pcbios-isar.py  |  6 ++++--
 testsuite/start_vm.py                              | 14 +++++++++-----
 testsuite/vm_boot_test/vm_boot_test.py             |  7 +++++--
 7 files changed, 34 insertions(+), 15 deletions(-)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks

-- 
2.30.2

[PATCH v2 1/6] Add ISAR version of biosplusefi WIC plugin

[Felix Moessbauer]

This patch adds support to create a disk image that works with both
EFI and legacy bios. The biosplusefi-isar is based on the original
biosplusefi WIC plugin that internally calls the bootimg-pcbios and
bootimg-efi. By that, code duplication can be avoided.

The generated image can be booted with EFI or legacy pcbios.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 .../lib/wic/plugins/source/bootimg-biosplusefi-isar.py    | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-biosplusefi-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-biosplusefi-isar.py
index 5bd73906..fb6b7fcc 100644
--- a/meta/scripts/lib/wic/plugins/source/bootimg-biosplusefi-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/bootimg-biosplusefi-isar.py
@@ -75,10 +75,10 @@ class BootimgBiosPlusEFIPlugin(SourcePlugin):
                --ondisk sda --label os_boot --active --align 1024 --use-uuid
     """
 
-    name = 'bootimg-biosplusefi'
+    name = 'bootimg-biosplusefi-isar'
 
-    __PCBIOS_MODULE_NAME = "bootimg-pcbios"
-    __EFI_MODULE_NAME = "bootimg-efi"
+    __PCBIOS_MODULE_NAME = "bootimg-pcbios-isar"
+    __EFI_MODULE_NAME = "bootimg-efi-isar"
 
     __imgEFIObj = None
     __imgBiosObj = None
@@ -106,7 +106,7 @@ class BootimgBiosPlusEFIPlugin(SourcePlugin):
         loader = SourceFileLoader(cls.__PCBIOS_MODULE_NAME, modulePath)
         mod = types.ModuleType(loader.name)
         loader.exec_module(mod)
-        cls.__imgBiosObj = mod.BootimgPcbiosPlugin()
+        cls.__imgBiosObj = mod.BootimgPcbiosIsarPlugin()
 
         # Import bootimg-efi (class name "BootimgEFIPlugin")
         modulePath = os.path.join(os.path.dirname(os.path.realpath(__file__)),
-- 
2.30.2

[PATCH v2 2/6] wic: make import isarpluginbase work when plugins use each other

[Felix Moessbauer]

From: Henning Schild <henning.schild@siemens.com>

The biosplusefi plugin calls other plugins, which in our case use
isarpluginbase and fiddle with their library paths to do so.
Make sure that path modification also works when the plugins are
called by another plugin.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Acked-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py    | 6 ++++--
 meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py | 6 ++++--
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
index d9712548..5ba0777a 100644
--- a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
@@ -23,8 +23,10 @@ from wic.pluginbase import SourcePlugin
 from wic.misc import (exec_cmd, exec_native_cmd,
                       get_bitbake_var, BOOTDD_EXTRA_SPACE)
 
-import sys
-sys.path[0] = os.path.dirname(os.path.abspath(__file__)) + "/.."
+# allow plugins to import from isarpluginbase
+if '__file__' in globals():
+    import sys
+    sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/..")
 from isarpluginbase import (isar_get_filenames, isar_populate_boot_cmd)
 
 logger = logging.getLogger('wic')
diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
index f493890f..9136d4f2 100644
--- a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
@@ -20,8 +20,10 @@ from wic.pluginbase import SourcePlugin
 from wic.misc import (exec_cmd, exec_native_cmd,
                       get_bitbake_var, BOOTDD_EXTRA_SPACE)
 
-import sys
-sys.path[0] = os.path.dirname(os.path.abspath(__file__)) + "/.."
+# allow plugins to import from isarpluginbase
+if '__file__' in globals():
+    import sys
+    sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/..")
 from isarpluginbase import (isar_get_filenames, isar_populate_boot_cmd)
 
 logger = logging.getLogger('wic')
-- 
2.30.2

[PATCH v2 3/6] test: add target for efi-plus-pcbios image type

[Felix Moessbauer]

This patch switches the qemuamd64-buster mc target to generate
a efi-plus-pcbios image.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta-isar/conf/multiconfig/qemuamd64-buster.conf         | 2 ++
 meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks | 6 ++++++
 2 files changed, 8 insertions(+)
 create mode 100644 meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks

diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
index 5615198d..44fe8af9 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
@@ -2,5 +2,7 @@
 
 MACHINE ?= "qemuamd64"
 DISTRO ?= "debian-buster"
+WKS_FILE ?= "efi-plus-pcbios.wks"
+IMAGER_INSTALL += "${SYSLINUX_BOOTLOADER_INSTALL}"
 
 IMAGE_FSTYPES ?= "wic-img ext4-img"
diff --git a/meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks b/meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks
new file mode 100644
index 00000000..03928915
--- /dev/null
+++ b/meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks
@@ -0,0 +1,6 @@
+# Example to show how to create an efi + pcbios image
+# Note, that the loader argument is mandatory. But systemd-boot also works.
+part /boot --source bootimg-biosplusefi-isar --sourceparams="loader=grub-efi" --label boot --active --align 1024
+part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --exclude-path=boot
+
+bootloader --ptable gpt --timeout 3 --append "rootwait console=ttyS0,115200 console=tty0"
-- 
2.30.2

[PATCH v2 4/6] add support for current debian distros in start_vm.py

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/start_vm.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index 4f5b2ec4..8fd3e16c 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -91,7 +91,7 @@ if __name__ == "__main__":
     parser = argparse.ArgumentParser()
     parser.add_argument('-a', '--arch', choices=['arm', 'arm64', 'amd64', 'i386', 'mipsel'], help='set isar machine architecture.', default='arm')
     parser.add_argument('-b', '--build', help='set path to build directory.', default=os.getcwd())
-    parser.add_argument('-d', '--distro', choices=['jessie', 'stretch'], help='set isar Debian distribution.', default='stretch')
+    parser.add_argument('-d', '--distro', choices=['jessie', 'stretch', 'buster', 'bullseye', 'bookworm'], help='set isar Debian distribution.', default='stretch')
     parser.add_argument('-o', '--out', help='Route QEMU console output to specified file.')
     parser.add_argument('-p', '--pid', help='Store QEMU pid to specified file.')
     args = parser.parse_args()
-- 
2.30.2

[PATCH v2 5/6] Add pcbios option to start_vm

[Felix Moessbauer]

This patch adds the --pcbios option to the start_vm script to
enforce that any -bios QEMU options are removed.
By that, QEMU is forced to use a pc bios (like seabios).

This helps to test combined efi + pcbios disks as these can be
bootet with both efi and pc bios.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/start_vm.py | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/testsuite/start_vm.py b/testsuite/start_vm.py
index 8fd3e16c..708520f7 100755
--- a/testsuite/start_vm.py
+++ b/testsuite/start_vm.py
@@ -21,7 +21,7 @@ def get_bitbake_var(output, var):
             ret = line.split('"')[1]
     return ret
 
-def format_qemu_cmdline(arch, build, distro, out, pid):
+def format_qemu_cmdline(arch, build, distro, out, pid, enforce_pcbios=False):
     bb_output = get_bitbake_env(arch, distro).decode()
 
     rootfs_image = ''
@@ -66,6 +66,9 @@ def format_qemu_cmdline(arch, build, distro, out, pid):
         extra_args.extend(['-pidfile', pid])
 
     qemu_disk_args = qemu_disk_args.replace('##ROOTFS_IMAGE##', deploy_dir_image + '/' + rootfs_image).split()
+    if enforce_pcbios and '-bios' in qemu_disk_args:
+        bios_idx = qemu_disk_args.index('-bios')
+        del qemu_disk_args[bios_idx : bios_idx+2]
 
     cmd = ['qemu-system-' + qemu_arch, '-m', '1024M']
 
@@ -80,8 +83,8 @@ def format_qemu_cmdline(arch, build, distro, out, pid):
 
     return cmd
 
-def start_qemu(arch, build, distro, out, pid):
-    cmdline = format_qemu_cmdline(arch, build, distro, out, pid)
+def start_qemu(arch, build, distro, out, pid, enforce_pcbios):
+    cmdline = format_qemu_cmdline(arch, build, distro, out, pid, enforce_pcbios)
     cmdline.insert(1, '-nographic')
 
     print(cmdline)
@@ -94,6 +97,7 @@ if __name__ == "__main__":
     parser.add_argument('-d', '--distro', choices=['jessie', 'stretch', 'buster', 'bullseye', 'bookworm'], help='set isar Debian distribution.', default='stretch')
     parser.add_argument('-o', '--out', help='Route QEMU console output to specified file.')
     parser.add_argument('-p', '--pid', help='Store QEMU pid to specified file.')
+    parser.add_argument('--pcbios', action="store_true", help='remove any bios options to enforce use of pc bios')
     args = parser.parse_args()
 
-    start_qemu(args.arch, args.build, args.distro, args.out, args.pid)
+    start_qemu(args.arch, args.build, args.distro, args.out, args.pid, args.pcbios)
-- 
2.30.2

[PATCH v2 6/6] run vm_boot_test against EFI and PC BIOS

[Felix Moessbauer]

This patch adds a test that boots the combined efi + pcbios
image with pc bios.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 testsuite/vm_boot_test/vm_boot_test.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/testsuite/vm_boot_test/vm_boot_test.py b/testsuite/vm_boot_test/vm_boot_test.py
index f8c655c9..179dad88 100644
--- a/testsuite/vm_boot_test/vm_boot_test.py
+++ b/testsuite/vm_boot_test/vm_boot_test.py
@@ -32,7 +32,7 @@ class VmBase(Test):
                         for x in output.splitlines() if x != ''))
             os.environ.update(env)
 
-    def vm_start(self, arch='amd64', distro='buster'):
+    def vm_start(self, arch='amd64', distro='buster', enforce_pcbios=False):
         build_dir = self.params.get('build_dir', default='.')
         time_to_wait = self.params.get('time_to_wait', default=60)
 
@@ -49,7 +49,7 @@ class VmBase(Test):
         os.chmod(output_file, 0o644)
 
         cmdline = start_vm.format_qemu_cmdline(arch, build_dir, distro,
-                                               output_file, None)
+                                               output_file, None, enforce_pcbios)
         cmdline.insert(1, '-nographic')
 
         self.log.info('QEMU boot line: ' + str(cmdline))
@@ -127,7 +127,10 @@ class VmBootTestFull(VmBase):
         self.vm_start('i386','buster')
 
     def test_amd64_buster(self):
+        # test efi boot
         self.vm_start('amd64','buster')
+        # test pcbios boot
+        self.vm_start('amd64', 'buster', True)
 
     def test_amd64_focal(self):
         self.vm_start('amd64','focal')
-- 
2.30.2

Re: [PATCH v2 0/6] Add support for dual bios + efi disk generation

[Henning Schild]

LGTM

Am Mon, 7 Mar 2022 07:42:45 +0100
schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:

> This patch ports the bios + efi disk support from OE to ISAR.
> By that, the generated disk can be booted both with EFI as well
> as with PCBIOS.
> 
> Please note, that patches 2 and 3 do not work independently,
> but are split for easier refactoring of the isarpluginsbase class.
> 
> Changes since v1:
> 
> - minimize changes to bootimg-biosplusefi-isar.py
> - replace erronous commit message of hennings commit
> - rework efi-plus-pcbios WKS file to not include any bios-only options
> - update start_vm script to support modern debian distros
> - test boot of both efi and bios of efi + bios test target
> 
> Best regards,
> Felix
> 
> Felix Moessbauer (5):
>   Add ISAR version of biosplusefi WIC plugin
>   test: add target for efi-plus-pcbios image type
>   add support for current debian distros in start_vm.py
>   Add pcbios option to start_vm
>   run vm_boot_test against EFI and PC BIOS
> 
> Henning Schild (1):
>   wic: make import isarpluginbase work when plugins use each other
> 
>  meta-isar/conf/multiconfig/qemuamd64-buster.conf   |  2 ++
>  .../scripts/lib/wic/canned-wks/efi-plus-pcbios.wks |  6 ++++++
>  .../wic/plugins/source/bootimg-biosplusefi-isar.py |  8 ++++----
>  .../lib/wic/plugins/source/bootimg-efi-isar.py     |  6 ++++--
>  .../lib/wic/plugins/source/bootimg-pcbios-isar.py  |  6 ++++--
>  testsuite/start_vm.py                              | 14
> +++++++++----- testsuite/vm_boot_test/vm_boot_test.py             |
> 7 +++++-- 7 files changed, 34 insertions(+), 15 deletions(-)
>  create mode 100644
> meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks
> 

Re: [PATCH v2 0/6] Add support for dual bios + efi disk generation

[Anton Mikanovich]

07.03.2022 09:42, Felix Moessbauer wrote:
> This patch ports the bios + efi disk support from OE to ISAR.
> By that, the generated disk can be booted both with EFI as well
> as with PCBIOS.
>
> Please note, that patches 2 and 3 do not work independently,
> but are split for easier refactoring of the isarpluginsbase class.
>
> Changes since v1:
>
> - minimize changes to bootimg-biosplusefi-isar.py
> - replace erronous commit message of hennings commit
> - rework efi-plus-pcbios WKS file to not include any bios-only options
> - update start_vm script to support modern debian distros
> - test boot of both efi and bios of efi + bios test target
>
> Best regards,
> Felix
>
> Felix Moessbauer (5):
>    Add ISAR version of biosplusefi WIC plugin
>    test: add target for efi-plus-pcbios image type
>    add support for current debian distros in start_vm.py
>    Add pcbios option to start_vm
>    run vm_boot_test against EFI and PC BIOS
>
> Henning Schild (1):
>    wic: make import isarpluginbase work when plugins use each other
>
>   meta-isar/conf/multiconfig/qemuamd64-buster.conf   |  2 ++
>   .../scripts/lib/wic/canned-wks/efi-plus-pcbios.wks |  6 ++++++
>   .../wic/plugins/source/bootimg-biosplusefi-isar.py |  8 ++++----
>   .../lib/wic/plugins/source/bootimg-efi-isar.py     |  6 ++++--
>   .../lib/wic/plugins/source/bootimg-pcbios-isar.py  |  6 ++++--
>   testsuite/start_vm.py                              | 14 +++++++++-----
>   testsuite/vm_boot_test/vm_boot_test.py             |  7 +++++--
>   7 files changed, 34 insertions(+), 15 deletions(-)
>   create mode 100644 meta-isar/scripts/lib/wic/canned-wks/efi-plus-pcbios.wks
>
It looks like 'Copy efi-plus-pcbios script from OE' patch was missed 
from v1.

Please also rebase CI changes on top of the latest 'Unify build and run test
cases' patchset that will probably be merged to next tomorrow.