[PATCH v15 0/2] CPIO & OVA Images

public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed

* [PATCH v15 0/2] CPIO & OVA Images
@ 2021-09-07 15:43 Q. Gylstorff
  2021-09-07 15:43 ` [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Q. Gylstorff
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Q. Gylstorff @ 2021-09-07 15:43 UTC (permalink / raw)
  To: amikan, isar-users, jan.kiszka; +Cc: Quirin Gylstorff

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

 - Generate cpio images for rescue images
 - generate ova images for VMWare & Virtualbox

Changes V2:
 - initramfs modification is now a package
 - ova-wic-img was renamed to virtual-machine-image
 - virtual-machine-image do_convert_wic supports now other image
formats
 - Add tests
 - cpiogz now use chown instead of userspec

Changes V3:
 - Add tests to build-ci
 - add postrm to initramfs-config
 - use sdimage-efi for virtualbox images
 - changed maintainer

Changes V4:
 - use templates from debhelper for postinst and postrm scripts

Changes V5:
 - add virtual-machine-template package to add the template as extra file to the build

Changes V6:
 - whitespace fix
 - initramfs-config is now a inc file instead of a recipe
 - meta isar contains a default implementation of intramfs-config

Changes V7:
 - drop initramfs-config in favour of https://groups.google.com/g/isar-users/c/47EZAUTclZs/m/npCfE6zqCwAJ
 - clarify OVA commit message
 - rebase on  origin/next f5a6fdfd9aad49202093aab2158f625429eaf7c5

Changes V8:
 - formatting
 - correct cpio commit message

Changes in v9 by Jan Kiszka:
 - renaming "virtual-machine" -> "vm"

Changes in v10 by Jan Kiszka:
 - fix incomplete renaming "virtual-machine" -> "vm"
   (the classic "posting before testing"...)

Changes in v11:
 - fix full ci build by Jan Kiszka
 - fix missing in ovf file after envsubst

Changes in v12:
 - remove var BOOTLOADER

Changes in v13:
 - add image_undo_mounts

Changes in v14:
 - remove get_disksize to reduce image_do_mounts calls

Changes in v15:
 - remove image_undo_mounts after revert

Quirin Gylstorff (2):
  meta/classes: Generate ova image for VMWare or Virtualbox
  meta/classes: add cpiogz-img

 doc/user_manual.md                            |   1 +
 meta-isar/conf/local.conf.sample              |   2 +
 meta-isar/conf/machine/virtualbox.conf        |  15 ++
 meta-isar/conf/machine/vmware.conf            |  15 ++
 .../multiconfig/qemuamd64-buster-cpiogz.conf  |   9 +
 .../multiconfig/virtualbox-ova-buster.conf    |   8 +
 meta/classes/cpiogz-img.bbclass               |  21 +++
 meta/classes/vm-img.bbclass                   | 116 +++++++++++++
 .../vm-template/files/vm-template.ovf.tmpl    | 155 ++++++++++++++++++
 .../vm-template/vm-template_0.1.bb            |  16 ++
 scripts/ci_build.sh                           |   2 +
 11 files changed, 360 insertions(+)
 create mode 100644 meta-isar/conf/machine/virtualbox.conf
 create mode 100644 meta-isar/conf/machine/vmware.conf
 create mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
 create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
 create mode 100644 meta/classes/cpiogz-img.bbclass
 create mode 100644 meta/classes/vm-img.bbclass
 create mode 100644 meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
 create mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb

-- 
2.20.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox
  2021-09-07 15:43 [PATCH v15 0/2] CPIO & OVA Images Q. Gylstorff
@ 2021-09-07 15:43 ` Q. Gylstorff
  2021-09-21  7:40   ` Henning Schild
  2021-09-07 15:43 ` [PATCH v15 2/2] meta/classes: add cpiogz-img Q. Gylstorff
  2021-09-14  9:56 ` [PATCH v15 0/2] CPIO & OVA Images Anton Mikanovich
  2 siblings, 1 reply; 5+ messages in thread
From: Q. Gylstorff @ 2021-09-07 15:43 UTC (permalink / raw)
  To: amikan, isar-users, jan.kiszka; +Cc: Quirin Gylstorff

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This allows to generate a ova file for virtualbox or vmware. The
images differ in the setting of the variable `VMDK_SUBFORMAT`.
- `streamOptimized` is used for Vmware Workstation
- `monolithicSparse` is used for Virtualbox

This is necessary as virtualbox throws an import error for a `streamOptimized`
version. The ova for Virtualbox is also bigger due the disk format.

The default machine settings are:
- 4 CPU Cores with 8GB Ram

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
[Jan: shortened named, reformatting, massaged comments, added to local.conf]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 doc/user_manual.md                            |   1 +
 meta-isar/conf/local.conf.sample              |   1 +
 meta-isar/conf/machine/virtualbox.conf        |  15 ++
 meta-isar/conf/machine/vmware.conf            |  15 ++
 .../multiconfig/virtualbox-ova-buster.conf    |   8 +
 meta/classes/vm-img.bbclass                   | 116 +++++++++++++
 .../vm-template/files/vm-template.ovf.tmpl    | 155 ++++++++++++++++++
 .../vm-template/vm-template_0.1.bb            |  16 ++
 scripts/ci_build.sh                           |   1 +
 9 files changed, 328 insertions(+)
 create mode 100644 meta-isar/conf/machine/virtualbox.conf
 create mode 100644 meta-isar/conf/machine/vmware.conf
 create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
 create mode 100644 meta/classes/vm-img.bbclass
 create mode 100644 meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
 create mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 1da0e9d..3a83735 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -454,6 +454,7 @@ Isar can generate various images types for specific machine. The type of the ima
  - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image (default option for the `rpi` machine).
  - `wic-img` - A full disk image with user-specified partitions created and populated using the wic tool.
  - `ubi-img` - A image for use on mtd nand partitions employing UBI
+ - `vm-img` - A image for use on VirtualBox or VMware
 
 ---
 
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 96a8beb..4e462b7 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -64,6 +64,7 @@ BBMULTICONFIG = " \
     nand-ubi-demo-buster \
     nanopi-neo-buster \
     stm32mp15x-buster \
+    virtualbox-ova-buster \
     rpi-stretch \
     sifive-fu540-sid-ports \
     qemuarm64-focal \
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
new file mode 100644
index 0000000..b20eaca
--- /dev/null
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -0,0 +1,15 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+DISTRO_ARCH ?= "amd64"
+
+KERNEL_NAME ?= "amd64"
+
+WKS_FILE ?= "sdimage-efi"
+
+IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+
+VMDK_SUBFORMAT = "monolithicSparse"
+IMAGE_TYPE ?= "vm-img"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
new file mode 100644
index 0000000..731559e
--- /dev/null
+++ b/meta-isar/conf/machine/vmware.conf
@@ -0,0 +1,15 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+DISTRO_ARCH ?= "amd64"
+
+KERNEL_NAME ?= "amd64"
+
+WKS_FILE ?= "sdimage-efi"
+
+IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+
+VMDK_SUBFORMAT = "streamOptimized"
+IMAGE_TYPE ?= "vm-img"
diff --git a/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
new file mode 100644
index 0000000..3042556
--- /dev/null
+++ b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
@@ -0,0 +1,8 @@
+#
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+
+MACHINE = "virtualbox"
+DISTRO = "debian-buster"
diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass
new file mode 100644
index 0000000..b230af2
--- /dev/null
+++ b/meta/classes/vm-img.bbclass
@@ -0,0 +1,116 @@
+# This software is a part of ISAR.
+# Copyright (C) 2019-2020 Siemens AG
+#
+# This class allows to generate images for VMware and VirtualBox
+#
+
+inherit buildchroot
+inherit wic-img
+
+IMAGER_BUILD_DEPS += "vm-template"
+IMAGER_INSTALL += "qemu-utils gawk uuid-runtime vm-template"
+
+# virtual machine disk settings
+SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img"
+
+# For VirtualBox, this needs to be "monolithicSparse" (default to it).
+# VMware needs this to be "streamOptimized".
+VMDK_SUBFORMAT ?= "monolithicSparse"
+
+VIRTUAL_MACHINE_IMAGE_TYPE ?= "vmdk"
+VIRTUAL_MACHINE_IMAGE_FILE = "${IMAGE_FULLNAME}-disk001.${VIRTUAL_MACHINE_IMAGE_TYPE}"
+VIRTUAL_MACHINE_DISK = "${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}"
+
+def set_convert_options(d):
+   format = d.getVar("VIRTUAL_MACHINE_IMAGE_TYPE")
+   if format == "vmdk":
+      return "-o subformat=%s" % d.getVar("VMDK_SUBFORMAT")
+   else:
+      return ""
+
+
+CONVERSION_OPTIONS = "${@set_convert_options(d)}"
+
+do_convert_wic() {
+    rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
+    image_do_mounts
+    bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${WIC_IMAGE_FILE}"
+    sudo -E  chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+    /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \
+        '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}'
+}
+
+addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template
+
+# User settings for OVA
+OVA_NAME ?= "${IMAGE_FULLNAME}"
+OVA_MEMORY ?= "8192"
+OVA_NUMBER_OF_CPU ?= "4"
+OVA_VRAM ?= "64"
+OVA_FIRMWARE ?= "efi"
+OVA_ACPI ?= "true"
+OVA_3D_ACCEL ?= "false"
+OVA_CLIPBOARD ?= "bidirectional"
+OVA_SHA_ALG = "1"
+
+# Generate random MAC addresses just as VirtualBox does, the format is
+# their assigned prefix for the first 3 bytes followed by 3 random bytes.
+VBOX_MAC_PREFIX = "080027"
+
+macgen() {
+    hexdump -n3 -e "\"${VBOX_MAC_PREFIX}%06X\n\"" /dev/urandom
+}
+
+OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \
+            OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL OVA_CLIPBOARD \
+            OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE"
+
+# the ovf template is updated with ensubst
+# this function adds the variable from OVA_VARS to the environment
+python update_environment() {
+    template_vars = (d.getVar('OVA_VARS', True) or "").split()
+    if len(template_vars) == 0:
+        return
+
+    for varname in template_vars:
+        value = d.getVar(varname, True)
+        if value:
+            os.environ.update({varname: value})
+}
+
+do_create_ova[prefuncs] += "update_environment"
+do_create_ova() {
+    if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then
+        exit 0
+    fi
+    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ova'
+    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ovf'
+    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.mf'
+
+    export PRIMARY_MAC=$(macgen)
+    export SECONDARY_MAC=$(macgen)
+    export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK})
+    export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
+
+    image_do_mounts
+
+    sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO'
+        export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \
+                                 | gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}')
+        export DISK_UUID=$(uuidgen)
+        export VM_UUID=$(uuidgen)
+        # create ovf
+        cat /usr/share/vm-template/vm-template.ovf.tmpl | envsubst > ${PP_DEPLOY}/${OVA_NAME}.ovf
+        tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.ovf
+
+        # VirtualBox needs here a manifest file. VMware does accept that format.
+        if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" ]; then
+            echo "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf
+            echo "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf
+            tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.mf
+        fi
+        tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE}
+EOSUDO
+}
+
+addtask do_create_ova after do_convert_wic before do_deploy
diff --git a/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
new file mode 100644
index 0000000..e6b5305
--- /dev/null
+++ b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
@@ -0,0 +1,155 @@
+<?xml version="1.0"?>
+<Envelope ovf:version="1.0" xml:lang="en-US" xmlns="http://schemas.dmtf.org/ovf/envelope/1" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:vbox="http://www.virtualbox.org/ovf/machine">
+  <References>
+    <File ovf:href="${VIRTUAL_MACHINE_IMAGE_FILE}" ovf:id="file1"/>
+  </References>
+  <DiskSection>
+    <Info>List of the virtual disks used in the package</Info>
+    <Disk ovf:capacity="${DISK_SIZE_BYTES}" ovf:diskId="vmdisk1" ovf:fileRef="file1" ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#${VMDK_SUBFORMAT}" vbox:uuid="${DISK_UUID}"/>
+  </DiskSection>
+  <NetworkSection>
+    <Info>Logical networks used in the package</Info>
+    <Network ovf:name="NAT">
+      <Description>Logical network used by this appliance.</Description>
+    </Network>
+  </NetworkSection>
+  <VirtualSystem ovf:id="${OVA_NAME}">
+    <Info>A virtual machine</Info>
+    <OperatingSystemSection ovf:id="96">
+      <Info>The kind of installed guest operating system</Info>
+      <Description>Debian_64</Description>
+      <vbox:OSType ovf:required="false">Debian_64</vbox:OSType>
+    </OperatingSystemSection>
+    <VirtualHardwareSection>
+      <Info>Virtual hardware requirements for a virtual machine</Info>
+      <System>
+        <vssd:ElementName>Virtual Hardware Family</vssd:ElementName>
+        <vssd:InstanceID>0</vssd:InstanceID>
+        <vssd:VirtualSystemIdentifier>${OVA_NAME}</vssd:VirtualSystemIdentifier>
+        <vssd:VirtualSystemType>virtualbox-2.2</vssd:VirtualSystemType>
+      </System>
+      <Item>
+        <rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
+        <rasd:Caption>${OVA_NUMBER_OF_CPU} virtual CPU</rasd:Caption>
+        <rasd:Description>Number of virtual CPUs</rasd:Description>
+        <rasd:ElementName>${OVA_NUMBER_OF_CPU} virtual CPU</rasd:ElementName>
+        <rasd:InstanceID>1</rasd:InstanceID>
+        <rasd:ResourceType>3</rasd:ResourceType>
+        <rasd:VirtualQuantity>${OVA_NUMBER_OF_CPU}</rasd:VirtualQuantity>
+      </Item>
+      <Item>
+        <rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits>
+        <rasd:Caption>${OVA_MEMORY} MB of memory</rasd:Caption>
+        <rasd:Description>Memory Size</rasd:Description>
+        <rasd:ElementName>${OVA_MEMORY} MB of memory</rasd:ElementName>
+        <rasd:InstanceID>2</rasd:InstanceID>
+        <rasd:ResourceType>4</rasd:ResourceType>
+        <rasd:VirtualQuantity>${OVA_MEMORY}</rasd:VirtualQuantity>
+      </Item>
+      <Item>
+        <rasd:Address>0</rasd:Address>
+        <rasd:Caption>ideController0</rasd:Caption>
+        <rasd:Description>IDE Controller</rasd:Description>
+        <rasd:ElementName>ideController0</rasd:ElementName>
+        <rasd:InstanceID>3</rasd:InstanceID>
+        <rasd:ResourceSubType>PIIX4</rasd:ResourceSubType>
+        <rasd:ResourceType>5</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:Address>1</rasd:Address>
+        <rasd:Caption>ideController1</rasd:Caption>
+        <rasd:Description>IDE Controller</rasd:Description>
+        <rasd:ElementName>ideController1</rasd:ElementName>
+        <rasd:InstanceID>4</rasd:InstanceID>
+        <rasd:ResourceSubType>PIIX4</rasd:ResourceSubType>
+        <rasd:ResourceType>5</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
+        <rasd:Caption>Ethernet adapter on 'NAT'</rasd:Caption>
+        <rasd:Connection>NAT</rasd:Connection>
+        <rasd:ElementName>Ethernet adapter on 'NAT'</rasd:ElementName>
+        <rasd:InstanceID>5</rasd:InstanceID>
+        <rasd:ResourceSubType>E1000</rasd:ResourceSubType>
+        <rasd:ResourceType>10</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:AddressOnParent>0</rasd:AddressOnParent>
+        <rasd:Caption>disk1</rasd:Caption>
+        <rasd:Description>Disk Image</rasd:Description>
+        <rasd:ElementName>disk1</rasd:ElementName>
+        <rasd:HostResource>/disk/vmdisk1</rasd:HostResource>
+        <rasd:InstanceID>6</rasd:InstanceID>
+        <rasd:Parent>3</rasd:Parent>
+        <rasd:ResourceType>17</rasd:ResourceType>
+      </Item>
+      <vmw:Config ovf:required="false" vmw:key="firmware" vmw:value="${OVA_FIRMWARE}"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.afterPowerOn" vmw:value="true"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.afterResume" vmw:value="true"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestShutdown" vmw:value="true"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestStandby" vmw:value="true"/>
+      <vmw:ExtraConfig ovf:required="false" vmw:key="virtualHW.productCompatibility" vmw:value="hosted"/>
+      </VirtualHardwareSection>
+      <vbox:Machine ovf:required="false" version="1.12-linux" uuid="{${VM_UUID}}" name="${OVA_NAME}" OSType="Debian_64" snapshotFolder="Snapshots" lastStateChange="${LAST_CHANGE}">
+        <ovf:Info>Complete VirtualBox machine configuration in VirtualBox format</ovf:Info>
+        <Hardware>
+          <CPU count="${OVA_NUMBER_OF_CPU}">
+            <PAE enabled="true"/>
+            <HardwareVirtExLargePages enabled="false"/>
+          </CPU>
+          <Memory RAMSize="${OVA_MEMORY}"/>
+          <Firmware type="${OVA_FIRMWARE_VIRTUALBOX}"/>
+          <Boot>
+            <Order position="1" device="HardDisk"/>
+            <Order position="2" device="None"/>
+            <Order position="3" device="None"/>
+            <Order position="4" device="None"/>
+          </Boot>
+          <Display VRAMSize="${OVA_VRAM}" monitorCount="1" accelerate3D="${OVA_3D_ACCEL}" accelerate2DVideo="false"/>
+          <VideoRecording enabled="false" file="Test.webm" horzRes="640" vertRes="480"/>
+          <RemoteDisplay enabled="false" authType="Null"/>
+          <BIOS>
+            <IOAPIC enabled="${OVA_ACPI}"/>
+          </BIOS>
+          <USBController enabled="false" enabledEhci="false"/>
+          <Network>
+            <Adapter slot="0" enabled="true" MACAddress="${PRIMARY_MAC}" cable="true" speed="0" type="virtio">
+              <DisabledModes/>
+              <NAT>
+                <DNS pass-domain="true" use-proxy="false" use-host-resolver="false"/>
+                <Alias logging="false" proxy-only="false" use-same-ports="false"/>
+              </NAT>
+            </Adapter>
+          </Network>
+          <LPT>
+            <Port slot="1" enabled="false" IOBase="0x378" IRQ="7"/>
+          </LPT>
+          <AudioAdapter driver="Pulse" enabled="false"/>
+          <RTC localOrUTC="local"/>
+          <SharedFolders/>
+          <Clipboard mode="Disabled"/>
+          <DragAndDrop mode="Disabled"/>
+          <IO>
+            <IoCache enabled="true" size="5"/>
+            <BandwidthGroups/>
+          </IO>
+          <HostPci>
+            <Devices/>
+          </HostPci>
+          <EmulatedUSB>
+            <CardReader enabled="false"/>
+          </EmulatedUSB>
+          <Guest memoryBalloonSize="0"/>
+          <GuestProperties/>
+        </Hardware>
+        <StorageControllers>
+          <StorageController name="IDE Controller" type="PIIX4" PortCount="2" useHostIOCache="true" Bootable="true">
+            <AttachedDevice type="HardDisk" port="0" device="0">
+              <Image uuid="{${DISK_UUID}}"/>
+            </AttachedDevice>
+          </StorageController>
+        </StorageControllers>
+      </vbox:Machine>
+  </VirtualSystem>
+</Envelope>
diff --git a/meta/recipes-devtools/vm-template/vm-template_0.1.bb b/meta/recipes-devtools/vm-template/vm-template_0.1.bb
new file mode 100644
index 0000000..1d474cd
--- /dev/null
+++ b/meta/recipes-devtools/vm-template/vm-template_0.1.bb
@@ -0,0 +1,16 @@
+# This software is a part of ISAR.
+#
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg-raw
+
+SRC_URI += "file://vm-template.ovf.tmpl"
+
+do_install() {
+    TARGET=${D}/usr/share/vm-template
+    install -m 0755 -d ${TARGET}
+    install -m 0740 ${WORKDIR}/vm-template.ovf.tmpl \
+        ${TARGET}/vm-template.ovf.tmpl
+}
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 837cd67..1d530e6 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -43,6 +43,7 @@ TARGETS_SET="\
             mc:nand-ubi-demo-buster:isar-image-ubi \
             mc:rpi-stretch:isar-image-base \
             mc:qemuamd64-focal:isar-image-base \
+            mc:virtualbox-ova-buster:isar-image-base \
             "
           # qemu-user-static of <= buster too old to build that
           # mc:qemuarm64-buster:isar-image-base
-- 
2.20.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox
  2021-09-07 15:43 ` [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Q. Gylstorff
@ 2021-09-21  7:40   ` Henning Schild
  0 siblings, 0 replies; 5+ messages in thread
From: Henning Schild @ 2021-09-21  7:40 UTC (permalink / raw)
  To: Q. Gylstorff; +Cc: amikan, isar-users, jan.kiszka

I am building VM images with that and have a couple of questions and
will likely send improving patches later.

Why is the template inside a package and not coming in via SRC_URI?
Having it a file in SRC_URI would allow overriding in a layer. To
modify bits that the current template does not offer i.e. multiple NICs.

Where does that template come from in the first place? It seems to be a
somewhat weird mix of "vmw" and "vbox" special xml mixed into one file.
Anyhow vmware does not like the result and will throw xml parser
errors.

I am tempted to make the template a file in SRC_URI and have two
variants, one for vbox one for vmware. Because of the VMDK_SUBFORMAT we
can anyways not generate one .ova that will work for both. So might as
well not do the xml mixing and base the xml templates on what comes
directly out of respective ova exports of the two VMMs.

As i said i will send patches based on that idea. Some more comments
inline to problems i will try to fix. 

Am Tue,  7 Sep 2021 17:43:34 +0200
schrieb "Q. Gylstorff" <Quirin.Gylstorff@siemens.com>:

> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> This allows to generate a ova file for virtualbox or vmware. The
> images differ in the setting of the variable `VMDK_SUBFORMAT`.
> - `streamOptimized` is used for Vmware Workstation
> - `monolithicSparse` is used for Virtualbox
> 
> This is necessary as virtualbox throws an import error for a
> `streamOptimized` version. The ova for Virtualbox is also bigger due
> the disk format.
> 
> The default machine settings are:
> - 4 CPU Cores with 8GB Ram
> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> [Jan: shortened named, reformatting, massaged comments, added to
> local.conf] Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  doc/user_manual.md                            |   1 +
>  meta-isar/conf/local.conf.sample              |   1 +
>  meta-isar/conf/machine/virtualbox.conf        |  15 ++
>  meta-isar/conf/machine/vmware.conf            |  15 ++
>  .../multiconfig/virtualbox-ova-buster.conf    |   8 +
>  meta/classes/vm-img.bbclass                   | 116 +++++++++++++
>  .../vm-template/files/vm-template.ovf.tmpl    | 155
> ++++++++++++++++++ .../vm-template/vm-template_0.1.bb            |
> 16 ++ scripts/ci_build.sh                           |   1 +
>  9 files changed, 328 insertions(+)
>  create mode 100644 meta-isar/conf/machine/virtualbox.conf
>  create mode 100644 meta-isar/conf/machine/vmware.conf
>  create mode 100644
> meta-isar/conf/multiconfig/virtualbox-ova-buster.conf create mode
> 100644 meta/classes/vm-img.bbclass create mode 100644
> meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl create
> mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 1da0e9d..3a83735 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -454,6 +454,7 @@ Isar can generate various images types for
> specific machine. The type of the ima
>   - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image
> (default option for the `rpi` machine).
>   - `wic-img` - A full disk image with user-specified partitions
> created and populated using the wic tool.
>   - `ubi-img` - A image for use on mtd nand partitions employing UBI
> + - `vm-img` - A image for use on VirtualBox or VMware
>  
>  ---
>  
> diff --git a/meta-isar/conf/local.conf.sample
> b/meta-isar/conf/local.conf.sample index 96a8beb..4e462b7 100644
> --- a/meta-isar/conf/local.conf.sample
> +++ b/meta-isar/conf/local.conf.sample
> @@ -64,6 +64,7 @@ BBMULTICONFIG = " \
>      nand-ubi-demo-buster \
>      nanopi-neo-buster \
>      stm32mp15x-buster \
> +    virtualbox-ova-buster \
>      rpi-stretch \
>      sifive-fu540-sid-ports \
>      qemuarm64-focal \
> diff --git a/meta-isar/conf/machine/virtualbox.conf
> b/meta-isar/conf/machine/virtualbox.conf new file mode 100644
> index 0000000..b20eaca
> --- /dev/null
> +++ b/meta-isar/conf/machine/virtualbox.conf
> @@ -0,0 +1,15 @@
> +# This software is a part of ISAR.
> +# Copyright (c) Siemens AG, 2020
> +#
> +# SPDX-License-Identifier: MIT
> +
> +DISTRO_ARCH ?= "amd64"
> +
> +KERNEL_NAME ?= "amd64"
> +
> +WKS_FILE ?= "sdimage-efi"
> +
> +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
> +
> +VMDK_SUBFORMAT = "monolithicSparse"
> +IMAGE_TYPE ?= "vm-img"
> diff --git a/meta-isar/conf/machine/vmware.conf
> b/meta-isar/conf/machine/vmware.conf new file mode 100644
> index 0000000..731559e
> --- /dev/null
> +++ b/meta-isar/conf/machine/vmware.conf
> @@ -0,0 +1,15 @@
> +# This software is a part of ISAR.
> +# Copyright (c) Siemens AG, 2020
> +#
> +# SPDX-License-Identifier: MIT
> +
> +DISTRO_ARCH ?= "amd64"
> +
> +KERNEL_NAME ?= "amd64"
> +
> +WKS_FILE ?= "sdimage-efi"
> +
> +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
> +
> +VMDK_SUBFORMAT = "streamOptimized"
> +IMAGE_TYPE ?= "vm-img"
> diff --git a/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
> b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf new file mode
> 100644 index 0000000..3042556
> --- /dev/null
> +++ b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
> @@ -0,0 +1,8 @@
> +#
> +# Copyright (c) Siemens AG, 2020
> +#
> +# SPDX-License-Identifier: MIT
> +
> +
> +MACHINE = "virtualbox"
> +DISTRO = "debian-buster"
> diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass
> new file mode 100644
> index 0000000..b230af2
> --- /dev/null
> +++ b/meta/classes/vm-img.bbclass
> @@ -0,0 +1,116 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2019-2020 Siemens AG
> +#
> +# This class allows to generate images for VMware and VirtualBox
> +#
> +
> +inherit buildchroot
> +inherit wic-img
> +
> +IMAGER_BUILD_DEPS += "vm-template"
> +IMAGER_INSTALL += "qemu-utils gawk uuid-runtime vm-template"
> +
> +# virtual machine disk settings
> +SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img"
> +
> +# For VirtualBox, this needs to be "monolithicSparse" (default to
> it). +# VMware needs this to be "streamOptimized".
> +VMDK_SUBFORMAT ?= "monolithicSparse"
> +
> +VIRTUAL_MACHINE_IMAGE_TYPE ?= "vmdk"
> +VIRTUAL_MACHINE_IMAGE_FILE =
> "${IMAGE_FULLNAME}-disk001.${VIRTUAL_MACHINE_IMAGE_TYPE}"
> +VIRTUAL_MACHINE_DISK = "${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}" +
> +def set_convert_options(d):
> +   format = d.getVar("VIRTUAL_MACHINE_IMAGE_TYPE")
> +   if format == "vmdk":
> +      return "-o subformat=%s" % d.getVar("VMDK_SUBFORMAT")
> +   else:
> +      return ""
> +
> +
> +CONVERSION_OPTIONS = "${@set_convert_options(d)}"
> +
> +do_convert_wic() {
> +    rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
> +    image_do_mounts
> +    bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from
> ${WIC_IMAGE_FILE}"
> +    sudo -E  chroot --userspec=$( id -u ):$( id -g )
> ${BUILDCHROOT_DIR} \
> +    /usr/bin/qemu-img convert -f raw -O
> ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \
> +        '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}'
> '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}' +}
> +
> +addtask convert_wic before do_build after do_wic_image
> do_copy_boot_files do_install_imager_deps do_transform_template +
> +# User settings for OVA
> +OVA_NAME ?= "${IMAGE_FULLNAME}"
> +OVA_MEMORY ?= "8192"
> +OVA_NUMBER_OF_CPU ?= "4"
> +OVA_VRAM ?= "64"
> +OVA_FIRMWARE ?= "efi"
> +OVA_ACPI ?= "true"
> +OVA_3D_ACCEL ?= "false"
> +OVA_CLIPBOARD ?= "bidirectional"
> +OVA_SHA_ALG = "1"
> +
> +# Generate random MAC addresses just as VirtualBox does, the format
> is +# their assigned prefix for the first 3 bytes followed by 3
> random bytes. +VBOX_MAC_PREFIX = "080027"
> +
> +macgen() {
> +    hexdump -n3 -e "\"${VBOX_MAC_PREFIX}%06X\n\"" /dev/urandom
> +}
> +
> +OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \
> +            OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL OVA_CLIPBOARD \
> +            OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE"
> +
> +# the ovf template is updated with ensubst
> +# this function adds the variable from OVA_VARS to the environment
> +python update_environment() {
> +    template_vars = (d.getVar('OVA_VARS', True) or "").split()
> +    if len(template_vars) == 0:
> +        return
> +
> +    for varname in template_vars:
> +        value = d.getVar(varname, True)
> +        if value:
> +            os.environ.update({varname: value})
> +}
> +
> +do_create_ova[prefuncs] += "update_environment"
> +do_create_ova() {
> +    if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then
> +        exit 0
> +    fi
> +    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ova'
> +    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ovf'
> +    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.mf'
> +
> +    export PRIMARY_MAC=$(macgen)
> +    export SECONDARY_MAC=$(macgen)

this variable is not used at all

> +    export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK})

also not used

> +    export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
> +
> +    image_do_mounts
> +
> +    sudo -Es chroot --userspec=$( id -u ):$( id -g )
> ${BUILDCHROOT_DIR} <<'EOSUDO'
> +        export DISK_SIZE_BYTES=$(qemu-img info -f vmdk
> "${VIRTUAL_MACHINE_DISK}" \
> +                                 | gawk 'match($0, /^virtual
> size:.*\(([0-9]+) bytes\)/, a) {print a[1]}')
> +        export DISK_UUID=$(uuidgen)
> +        export VM_UUID=$(uuidgen)
> +        # create ovf
> +        cat /usr/share/vm-template/vm-template.ovf.tmpl | envsubst >
> ${PP_DEPLOY}/${OVA_NAME}.ovf

I think that should use the templating mechanisms of isar, when the
tmpl comes from SRC_URI.

> +        tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> ${OVA_NAME}.ovf +
> +        # VirtualBox needs here a manifest file. VMware does accept
> that format.
> +        if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" ]; then
> +            echo
> "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum
> ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >>
> ${PP_DEPLOY}/${OVA_NAME}.mf
> +            echo
> "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum
> ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >>
> ${PP_DEPLOY}/${OVA_NAME}.mf
> +            tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> ${OVA_NAME}.mf
> +        fi
> +        tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY}
> ${VIRTUAL_MACHINE_IMAGE_FILE} +EOSUDO
> +}
> +
> +addtask do_create_ova after do_convert_wic before do_deploy
> diff --git
> a/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
> b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl new
> file mode 100644 index 0000000..e6b5305 --- /dev/null
> +++ b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
> @@ -0,0 +1,155 @@
> +<?xml version="1.0"?>
> +<Envelope ovf:version="1.0" xml:lang="en-US"
> xmlns="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fschemas.dmtf.org%2Fovf%2Fenvelope%2F1&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=QUK9rXrjYinEuLHMlhctHoxciU0ut2guH9wfR6qbMtU%3D&amp;reserved=0"
> xmlns:ovf="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fschemas.dmtf.org%2Fovf%2Fenvelope%2F1&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=QUK9rXrjYinEuLHMlhctHoxciU0ut2guH9wfR6qbMtU%3D&amp;reserved=0"
> xmlns:rasd="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fschemas.dmtf.org%2Fwbem%2Fwscim%2F1%2Fcim-schema%2F2%2FCIM_ResourceAllocationSettingData&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=W3tcxTg7rY5%2Fky%2Bevr%2BU59QqF3E5nONUo9MPORm%2BWwQ%3D&amp;reserved=0"
> xmlns:vssd="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fschemas.dmtf.org%2Fwbem%2Fwscim%2F1%2Fcim-schema%2F2%2FCIM_VirtualSystemSettingData&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=1zBvFYKfgMh%2FUv9RB1qsC3PuLIW9IIDRy6beCELsrmI%3D&amp;reserved=0"
> xmlns:xsi="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2F2001%2FXMLSchema-instance&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=aGeBSPKZezgUCBJnZxVKGMAucA44U9jJvumkyWgs8j4%3D&amp;reserved=0"
> xmlns:vbox="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.virtualbox.org%2Fovf%2Fmachine&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=CeVYlxjZhYqD2%2BP5KUmC6dYlTfUiz0nS98WCl0dZ3r4%3D&amp;reserved=0">
> +  <References>
> +    <File ovf:href="${VIRTUAL_MACHINE_IMAGE_FILE}" ovf:id="file1"/>
> +  </References>
> +  <DiskSection>
> +    <Info>List of the virtual disks used in the package</Info>
> +    <Disk ovf:capacity="${DISK_SIZE_BYTES}" ovf:diskId="vmdisk1"

I think the size of the disk should be a variable one can specify. For
now i solved the problem with a change to the wks to have a fixed
partition size that is much bigger than the content.

> ovf:fileRef="file1"
> ovf:format="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.vmware.com%2Finterfaces%2Fspecifications%2Fvmdk.html%23%24&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C33d43cc3f7c2404a662508d9721643aa%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637666262223847131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=YAcTjvxTtKbuDavjLbtihP1IHXlb80Pi7N1dBfY%2F7hI%3D&amp;reserved=0{VMDK_SUBFORMAT}"
> vbox:uuid="${DISK_UUID}"/>
> +  </DiskSection>
> +  <NetworkSection>
> +    <Info>Logical networks used in the package</Info>
> +    <Network ovf:name="NAT">
> +      <Description>Logical network used by this
> appliance.</Description>
> +    </Network>
> +  </NetworkSection>
> +  <VirtualSystem ovf:id="${OVA_NAME}">
> +    <Info>A virtual machine</Info>
> +    <OperatingSystemSection ovf:id="96">
> +      <Info>The kind of installed guest operating system</Info>
> +      <Description>Debian_64</Description>
> +      <vbox:OSType ovf:required="false">Debian_64</vbox:OSType>
> +    </OperatingSystemSection>
> +    <VirtualHardwareSection>
> +      <Info>Virtual hardware requirements for a virtual
> machine</Info>
> +      <System>
> +        <vssd:ElementName>Virtual Hardware Family</vssd:ElementName>
> +        <vssd:InstanceID>0</vssd:InstanceID>
> +
> <vssd:VirtualSystemIdentifier>${OVA_NAME}</vssd:VirtualSystemIdentifier>
> +
> <vssd:VirtualSystemType>virtualbox-2.2</vssd:VirtualSystemType>
> +      </System>
> +      <Item>
> +        <rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
> +        <rasd:Caption>${OVA_NUMBER_OF_CPU} virtual CPU</rasd:Caption>
> +        <rasd:Description>Number of virtual CPUs</rasd:Description>
> +        <rasd:ElementName>${OVA_NUMBER_OF_CPU} virtual
> CPU</rasd:ElementName>
> +        <rasd:InstanceID>1</rasd:InstanceID>
> +        <rasd:ResourceType>3</rasd:ResourceType>
> +
> <rasd:VirtualQuantity>${OVA_NUMBER_OF_CPU}</rasd:VirtualQuantity>
> +      </Item>
> +      <Item>
> +        <rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits>
> +        <rasd:Caption>${OVA_MEMORY} MB of memory</rasd:Caption>
> +        <rasd:Description>Memory Size</rasd:Description>
> +        <rasd:ElementName>${OVA_MEMORY} MB of
> memory</rasd:ElementName>
> +        <rasd:InstanceID>2</rasd:InstanceID>
> +        <rasd:ResourceType>4</rasd:ResourceType>
> +        <rasd:VirtualQuantity>${OVA_MEMORY}</rasd:VirtualQuantity>
> +      </Item>
> +      <Item>
> +        <rasd:Address>0</rasd:Address>
> +        <rasd:Caption>ideController0</rasd:Caption>
> +        <rasd:Description>IDE Controller</rasd:Description>
> +        <rasd:ElementName>ideController0</rasd:ElementName>
> +        <rasd:InstanceID>3</rasd:InstanceID>
> +        <rasd:ResourceSubType>PIIX4</rasd:ResourceSubType>
> +        <rasd:ResourceType>5</rasd:ResourceType>
> +      </Item>
> +      <Item>
> +        <rasd:Address>1</rasd:Address>
> +        <rasd:Caption>ideController1</rasd:Caption>
> +        <rasd:Description>IDE Controller</rasd:Description>
> +        <rasd:ElementName>ideController1</rasd:ElementName>
> +        <rasd:InstanceID>4</rasd:InstanceID>
> +        <rasd:ResourceSubType>PIIX4</rasd:ResourceSubType>
> +        <rasd:ResourceType>5</rasd:ResourceType>
> +      </Item>
> +      <Item>
> +        <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
> +        <rasd:Caption>Ethernet adapter on 'NAT'</rasd:Caption>
> +        <rasd:Connection>NAT</rasd:Connection>
> +        <rasd:ElementName>Ethernet adapter on
> 'NAT'</rasd:ElementName>
> +        <rasd:InstanceID>5</rasd:InstanceID>
> +        <rasd:ResourceSubType>E1000</rasd:ResourceSubType>
> +        <rasd:ResourceType>10</rasd:ResourceType>
> +      </Item>
> +      <Item>
> +        <rasd:AddressOnParent>0</rasd:AddressOnParent>
> +        <rasd:Caption>disk1</rasd:Caption>
> +        <rasd:Description>Disk Image</rasd:Description>
> +        <rasd:ElementName>disk1</rasd:ElementName>
> +        <rasd:HostResource>/disk/vmdisk1</rasd:HostResource>
> +        <rasd:InstanceID>6</rasd:InstanceID>
> +        <rasd:Parent>3</rasd:Parent>
> +        <rasd:ResourceType>17</rasd:ResourceType>
> +      </Item>
> +      <vmw:Config ovf:required="false" vmw:key="firmware"
> vmw:value="${OVA_FIRMWARE}"/>
> +      <vmw:Config ovf:required="false"
> vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
> +      <vmw:Config ovf:required="false" vmw:key="tools.afterPowerOn"
> vmw:value="true"/>
> +      <vmw:Config ovf:required="false" vmw:key="tools.afterResume"
> vmw:value="true"/>
> +      <vmw:Config ovf:required="false"
> vmw:key="tools.beforeGuestShutdown" vmw:value="true"/>
> +      <vmw:Config ovf:required="false"
> vmw:key="tools.beforeGuestStandby" vmw:value="true"/>
> +      <vmw:ExtraConfig ovf:required="false"
> vmw:key="virtualHW.productCompatibility" vmw:value="hosted"/>
> +      </VirtualHardwareSection>
> +      <vbox:Machine ovf:required="false" version="1.12-linux"
> uuid="{${VM_UUID}}" name="${OVA_NAME}" OSType="Debian_64"
> snapshotFolder="Snapshots" lastStateChange="${LAST_CHANGE}">
> +        <ovf:Info>Complete VirtualBox machine configuration in
> VirtualBox format</ovf:Info>
> +        <Hardware>
> +          <CPU count="${OVA_NUMBER_OF_CPU}">
> +            <PAE enabled="true"/>
> +            <HardwareVirtExLargePages enabled="false"/>
> +          </CPU>
> +          <Memory RAMSize="${OVA_MEMORY}"/>
> +          <Firmware type="${OVA_FIRMWARE_VIRTUALBOX}"/>

That variable is missing in the exports and template vars. I hope the
same value as used for vmware will work. Only difference is upper- vs
lower-case.

Henning

> +          <Boot>
> +            <Order position="1" device="HardDisk"/>
> +            <Order position="2" device="None"/>
> +            <Order position="3" device="None"/>
> +            <Order position="4" device="None"/>
> +          </Boot>
> +          <Display VRAMSize="${OVA_VRAM}" monitorCount="1"
> accelerate3D="${OVA_3D_ACCEL}" accelerate2DVideo="false"/>
> +          <VideoRecording enabled="false" file="Test.webm"
> horzRes="640" vertRes="480"/>
> +          <RemoteDisplay enabled="false" authType="Null"/>
> +          <BIOS>
> +            <IOAPIC enabled="${OVA_ACPI}"/>
> +          </BIOS>
> +          <USBController enabled="false" enabledEhci="false"/>
> +          <Network>
> +            <Adapter slot="0" enabled="true"
> MACAddress="${PRIMARY_MAC}" cable="true" speed="0" type="virtio">
> +              <DisabledModes/>
> +              <NAT>
> +                <DNS pass-domain="true" use-proxy="false"
> use-host-resolver="false"/>
> +                <Alias logging="false" proxy-only="false"
> use-same-ports="false"/>
> +              </NAT>
> +            </Adapter>
> +          </Network>
> +          <LPT>
> +            <Port slot="1" enabled="false" IOBase="0x378" IRQ="7"/>
> +          </LPT>
> +          <AudioAdapter driver="Pulse" enabled="false"/>
> +          <RTC localOrUTC="local"/>
> +          <SharedFolders/>
> +          <Clipboard mode="Disabled"/>
> +          <DragAndDrop mode="Disabled"/>
> +          <IO>
> +            <IoCache enabled="true" size="5"/>
> +            <BandwidthGroups/>
> +          </IO>
> +          <HostPci>
> +            <Devices/>
> +          </HostPci>
> +          <EmulatedUSB>
> +            <CardReader enabled="false"/>
> +          </EmulatedUSB>
> +          <Guest memoryBalloonSize="0"/>
> +          <GuestProperties/>
> +        </Hardware>
> +        <StorageControllers>
> +          <StorageController name="IDE Controller" type="PIIX4"
> PortCount="2" useHostIOCache="true" Bootable="true">
> +            <AttachedDevice type="HardDisk" port="0" device="0">
> +              <Image uuid="{${DISK_UUID}}"/>
> +            </AttachedDevice>
> +          </StorageController>
> +        </StorageControllers>
> +      </vbox:Machine>
> +  </VirtualSystem>
> +</Envelope>
> diff --git a/meta/recipes-devtools/vm-template/vm-template_0.1.bb
> b/meta/recipes-devtools/vm-template/vm-template_0.1.bb new file mode
> 100644 index 0000000..1d474cd
> --- /dev/null
> +++ b/meta/recipes-devtools/vm-template/vm-template_0.1.bb
> @@ -0,0 +1,16 @@
> +# This software is a part of ISAR.
> +#
> +# Copyright (c) Siemens AG, 2020
> +#
> +# SPDX-License-Identifier: MIT
> +
> +inherit dpkg-raw
> +
> +SRC_URI += "file://vm-template.ovf.tmpl"
> +
> +do_install() {
> +    TARGET=${D}/usr/share/vm-template
> +    install -m 0755 -d ${TARGET}
> +    install -m 0740 ${WORKDIR}/vm-template.ovf.tmpl \
> +        ${TARGET}/vm-template.ovf.tmpl
> +}
> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
> index 837cd67..1d530e6 100755
> --- a/scripts/ci_build.sh
> +++ b/scripts/ci_build.sh
> @@ -43,6 +43,7 @@ TARGETS_SET="\
>              mc:nand-ubi-demo-buster:isar-image-ubi \
>              mc:rpi-stretch:isar-image-base \
>              mc:qemuamd64-focal:isar-image-base \
> +            mc:virtualbox-ova-buster:isar-image-base \
>              "
>            # qemu-user-static of <= buster too old to build that
>            # mc:qemuarm64-buster:isar-image-base


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v15 2/2] meta/classes: add cpiogz-img
  2021-09-07 15:43 [PATCH v15 0/2] CPIO & OVA Images Q. Gylstorff
  2021-09-07 15:43 ` [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Q. Gylstorff
@ 2021-09-07 15:43 ` Q. Gylstorff
  2021-09-14  9:56 ` [PATCH v15 0/2] CPIO & OVA Images Anton Mikanovich
  2 siblings, 0 replies; 5+ messages in thread
From: Q. Gylstorff @ 2021-09-07 15:43 UTC (permalink / raw)
  To: amikan, isar-users, jan.kiszka; +Cc: Quirin Gylstorff

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This allows to create a initramfs like filesystem.

Based on original patch by Michael Adler <michael.adler@siemens.com>.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
[Jan: reformatting, dropped bullseye target, added to local.conf]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 meta-isar/conf/local.conf.sample              |  1 +
 .../multiconfig/qemuamd64-buster-cpiogz.conf  |  9 ++++++++
 meta/classes/cpiogz-img.bbclass               | 21 +++++++++++++++++++
 scripts/ci_build.sh                           |  1 +
 4 files changed, 32 insertions(+)
 create mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
 create mode 100644 meta/classes/cpiogz-img.bbclass

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 4e462b7..88a8175 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -50,6 +50,7 @@ BBMULTICONFIG = " \
     qemuamd64-stretch \
     qemuamd64-buster \
     qemuamd64-buster-tgz \
+    qemuamd64-buster-cpiogz \
     qemuamd64-bullseye \
     container-amd64-stretch \
     container-amd64-buster \
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
new file mode 100644
index 0000000..cadcdac
--- /dev/null
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
@@ -0,0 +1,9 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+MACHINE ?= "qemuamd64"
+DISTRO ?= "debian-buster"
+
+IMAGE_TYPE = "cpiogz-img"
diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
new file mode 100644
index 0000000..940e2fb
--- /dev/null
+++ b/meta/classes/cpiogz-img.bbclass
@@ -0,0 +1,21 @@
+# This software is a part of ISAR.
+# Copyright (C) 2020 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+CPIOGZ_FNAME ?= "${IMAGE_FULLNAME}.cpio.gz"
+CPIOGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${CPIOGZ_FNAME}"
+IMAGER_INSTALL += "cpio"
+CPIO_IMAGE_FORMAT ?= "newc"
+
+do_cpiogz_image() {
+    sudo rm -f ${CPIOGZ_IMAGE_FILE}
+    image_do_mounts
+    sudo chroot ${BUILDCHROOT_DIR} \
+                sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
+                       /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o | /usr/bin/gzip -9 > \
+                       ${PP_DEPLOY}/${CPIOGZ_FNAME}"
+    sudo chown $(id -u):$(id -g) ${CPIOGZ_IMAGE_FILE}
+}
+
+addtask cpiogz_image before do_image after do_image_tools
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 1d530e6..a4aa74e 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -37,6 +37,7 @@ TARGETS_SET="\
             mc:qemuamd64-stretch:isar-image-base \
             mc:qemuamd64-buster:isar-image-base \
             mc:qemuamd64-buster-tgz:isar-image-base \
+            mc:qemuamd64-buster-cpiogz:isar-image-base \
             mc:qemuamd64-buster:isar-initramfs \
             mc:qemumipsel-stretch:isar-image-base \
             mc:qemumipsel-buster:isar-image-base \
-- 
2.20.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v15 0/2] CPIO & OVA Images
  2021-09-07 15:43 [PATCH v15 0/2] CPIO & OVA Images Q. Gylstorff
  2021-09-07 15:43 ` [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Q. Gylstorff
  2021-09-07 15:43 ` [PATCH v15 2/2] meta/classes: add cpiogz-img Q. Gylstorff
@ 2021-09-14  9:56 ` Anton Mikanovich
  2 siblings, 0 replies; 5+ messages in thread
From: Anton Mikanovich @ 2021-09-14  9:56 UTC (permalink / raw)
  To: Q. Gylstorff, isar-users, jan.kiszka

07.09.2021 18:43, Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>
>   - Generate cpio images for rescue images
>   - generate ova images for VMWare & Virtualbox
>
> Changes V2:
>   - initramfs modification is now a package
>   - ova-wic-img was renamed to virtual-machine-image
>   - virtual-machine-image do_convert_wic supports now other image
> formats
>   - Add tests
>   - cpiogz now use chown instead of userspec
>
> Changes V3:
>   - Add tests to build-ci
>   - add postrm to initramfs-config
>   - use sdimage-efi for virtualbox images
>   - changed maintainer
>
> Changes V4:
>   - use templates from debhelper for postinst and postrm scripts
>
> Changes V5:
>   - add virtual-machine-template package to add the template as extra file to the build
>
> Changes V6:
>   - whitespace fix
>   - initramfs-config is now a inc file instead of a recipe
>   - meta isar contains a default implementation of intramfs-config
>
> Changes V7:
>   - drop initramfs-config in favour of https://groups.google.com/g/isar-users/c/47EZAUTclZs/m/npCfE6zqCwAJ
>   - clarify OVA commit message
>   - rebase on  origin/next f5a6fdfd9aad49202093aab2158f625429eaf7c5
>
> Changes V8:
>   - formatting
>   - correct cpio commit message
>
> Changes in v9 by Jan Kiszka:
>   - renaming "virtual-machine" -> "vm"
>
> Changes in v10 by Jan Kiszka:
>   - fix incomplete renaming "virtual-machine" -> "vm"
>     (the classic "posting before testing"...)
>
> Changes in v11:
>   - fix full ci build by Jan Kiszka
>   - fix missing in ovf file after envsubst
>
> Changes in v12:
>   - remove var BOOTLOADER
>
> Changes in v13:
>   - add image_undo_mounts
>
> Changes in v14:
>   - remove get_disksize to reduce image_do_mounts calls
>
> Changes in v15:
>   - remove image_undo_mounts after revert
>
> Quirin Gylstorff (2):
>    meta/classes: Generate ova image for VMWare or Virtualbox
>    meta/classes: add cpiogz-img
>
>   doc/user_manual.md                            |   1 +
>   meta-isar/conf/local.conf.sample              |   2 +
>   meta-isar/conf/machine/virtualbox.conf        |  15 ++
>   meta-isar/conf/machine/vmware.conf            |  15 ++
>   .../multiconfig/qemuamd64-buster-cpiogz.conf  |   9 +
>   .../multiconfig/virtualbox-ova-buster.conf    |   8 +
>   meta/classes/cpiogz-img.bbclass               |  21 +++
>   meta/classes/vm-img.bbclass                   | 116 +++++++++++++
>   .../vm-template/files/vm-template.ovf.tmpl    | 155 ++++++++++++++++++
>   .../vm-template/vm-template_0.1.bb            |  16 ++
>   scripts/ci_build.sh                           |   2 +
>   11 files changed, 360 insertions(+)
>   create mode 100644 meta-isar/conf/machine/virtualbox.conf
>   create mode 100644 meta-isar/conf/machine/vmware.conf
>   create mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
>   create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
>   create mode 100644 meta/classes/cpiogz-img.bbclass
>   create mode 100644 meta/classes/vm-img.bbclass
>   create mode 100644 meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
>   create mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb
>
Applied to next, thanks.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-09-21  7:40 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-07 15:43 [PATCH v15 0/2] CPIO & OVA Images Q. Gylstorff
2021-09-07 15:43 ` [PATCH v15 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Q. Gylstorff
2021-09-21  7:40   ` Henning Schild
2021-09-07 15:43 ` [PATCH v15 2/2] meta/classes: add cpiogz-img Q. Gylstorff
2021-09-14  9:56 ` [PATCH v15 0/2] CPIO & OVA Images Anton Mikanovich

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox