[PATCH v14 0/2] CPIO & OVA Images

[PATCH v14 0/2] CPIO & OVA Images

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

- Generate cpio images for rescue images
- generate ova images for VMWare & Virtualbox

Changes V2:
- initramfs modification is now a package
- ova-wic-img was renamed to virtual-machine-image
- virtual-machine-image do_convert_wic supports now other image
formats
- Add tests
- cpiogz now use chown instead of userspec

Changes V3:
- Add tests to build-ci
- add postrm to initramfs-config
- use sdimage-efi for virtualbox images
- changed maintainer

Changes V4:
- use templates from debhelper for postinst and postrm scripts

Changes V5:
- add virtual-machine-template package to add the template as extra file to the build

Changes V6:
- whitespace fix
- initramfs-config is now a inc file instead of a recipe
- meta isar contains a default implementation of intramfs-config

Changes V7:
- drop initramfs-config in favour of https://groups.google.com/g/isar-users/c/47EZAUTclZs/m/npCfE6zqCwAJ
- clarify OVA commit message
- rebase on  origin/next f5a6fdfd9aad49202093aab2158f625429eaf7c5

Changes V8:
- formatting
- correct cpio commit message

Changes in v9 by Jan Kiszka:
 - renaming "virtual-machine" -> "vm"

Changes in v10 by Jan Kiszka:
 - fix incomplete renaming "virtual-machine" -> "vm"
   (the classic "posting before testing"...)

Changes in v11:
 - fix full ci build by Jan Kiszka
 - fix missing in ovf file after envsubst

Changes in v12:
 - remove var BOOTLOADER

Changes in v13:
 - add image_undo_mounts

Changes in v14:
 - remove get_disksize to reduce image_do_mounts calls

Quirin Gylstorff (2):
  meta/classes: Generate ova image for VMWare or Virtualbox
  meta/classes: add cpiogz-img

 doc/user_manual.md                            |   1 +
 meta-isar/conf/local.conf.sample              |   2 +
 meta-isar/conf/machine/virtualbox.conf        |  15 ++
 meta-isar/conf/machine/vmware.conf            |  15 ++
 .../multiconfig/qemuamd64-buster-cpiogz.conf  |   9 +
 .../multiconfig/virtualbox-ova-buster.conf    |   8 +
 meta/classes/cpiogz-img.bbclass               |  22 +++
 meta/classes/vm-img.bbclass                   | 118 +++++++++++++
 .../vm-template/files/vm-template.ovf.tmpl    | 155 ++++++++++++++++++
 .../vm-template/vm-template_0.1.bb            |  16 ++
 scripts/ci_build.sh                           |   2 +
 11 files changed, 363 insertions(+)
 create mode 100644 meta-isar/conf/machine/virtualbox.conf
 create mode 100644 meta-isar/conf/machine/vmware.conf
 create mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
 create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
 create mode 100644 meta/classes/cpiogz-img.bbclass
 create mode 100644 meta/classes/vm-img.bbclass
 create mode 100644 meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
 create mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb

-- 
2.20.1

[PATCH v14 1/2] meta/classes: Generate ova image for VMWare or Virtualbox

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This allows to generate a ova file for virtualbox or vmware. The
images differ in the setting of the variable `VMDK_SUBFORMAT`.
- `streamOptimized` is used for Vmware Workstation
- `monolithicSparse` is used for Virtualbox

This is necessary as virtualbox throws an import error for a `streamOptimized`
version. The ova for Virtualbox is also bigger due the disk format.

The default machine settings are:
- 4 CPU Cores with 8GB Ram

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
[Jan: shortened named, reformatting, massaged comments, added to local.conf]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 doc/user_manual.md                            |   1 +
 meta-isar/conf/local.conf.sample              |   1 +
 meta-isar/conf/machine/virtualbox.conf        |  15 ++
 meta-isar/conf/machine/vmware.conf            |  15 ++
 .../multiconfig/virtualbox-ova-buster.conf    |   8 +
 meta/classes/vm-img.bbclass                   | 118 +++++++++++++
 .../vm-template/files/vm-template.ovf.tmpl    | 155 ++++++++++++++++++
 .../vm-template/vm-template_0.1.bb            |  16 ++
 scripts/ci_build.sh                           |   1 +
 9 files changed, 330 insertions(+)
 create mode 100644 meta-isar/conf/machine/virtualbox.conf
 create mode 100644 meta-isar/conf/machine/vmware.conf
 create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
 create mode 100644 meta/classes/vm-img.bbclass
 create mode 100644 meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
 create mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb

diff --git a/doc/user_manual.md b/doc/user_manual.md
index cf7dc2f..2f7df87 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -454,6 +454,7 @@ Isar can generate various images types for specific machine. The type of the ima
  - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image (default option for the `rpi` machine).
  - `wic-img` - A full disk image with user-specified partitions created and populated using the wic tool.
  - `ubi-img` - A image for use on mtd nand partitions employing UBI
+ - `vm-img` - A image for use on VirtualBox or VMware
 
 ---
 
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 6cf1656..7b20a62 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -64,6 +64,7 @@ BBMULTICONFIG = " \
     nand-ubi-demo-buster \
     nanopi-neo-buster \
     stm32mp15x-buster \
+    virtualbox-ova-buster \
     rpi-stretch \
     sifive-fu540-sid-ports \
     qemuarm64-focal \
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
new file mode 100644
index 0000000..b20eaca
--- /dev/null
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -0,0 +1,15 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+DISTRO_ARCH ?= "amd64"
+
+KERNEL_NAME ?= "amd64"
+
+WKS_FILE ?= "sdimage-efi"
+
+IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+
+VMDK_SUBFORMAT = "monolithicSparse"
+IMAGE_TYPE ?= "vm-img"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
new file mode 100644
index 0000000..731559e
--- /dev/null
+++ b/meta-isar/conf/machine/vmware.conf
@@ -0,0 +1,15 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+DISTRO_ARCH ?= "amd64"
+
+KERNEL_NAME ?= "amd64"
+
+WKS_FILE ?= "sdimage-efi"
+
+IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
+
+VMDK_SUBFORMAT = "streamOptimized"
+IMAGE_TYPE ?= "vm-img"
diff --git a/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
new file mode 100644
index 0000000..3042556
--- /dev/null
+++ b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf
@@ -0,0 +1,8 @@
+#
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+
+MACHINE = "virtualbox"
+DISTRO = "debian-buster"
diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass
new file mode 100644
index 0000000..7b774f4
--- /dev/null
+++ b/meta/classes/vm-img.bbclass
@@ -0,0 +1,118 @@
+# This software is a part of ISAR.
+# Copyright (C) 2019-2020 Siemens AG
+#
+# This class allows to generate images for VMware and VirtualBox
+#
+
+inherit buildchroot
+inherit wic-img
+
+IMAGER_BUILD_DEPS += "vm-template"
+IMAGER_INSTALL += "qemu-utils gawk uuid-runtime vm-template"
+
+# virtual machine disk settings
+SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img"
+
+# For VirtualBox, this needs to be "monolithicSparse" (default to it).
+# VMware needs this to be "streamOptimized".
+VMDK_SUBFORMAT ?= "monolithicSparse"
+
+VIRTUAL_MACHINE_IMAGE_TYPE ?= "vmdk"
+VIRTUAL_MACHINE_IMAGE_FILE = "${IMAGE_FULLNAME}-disk001.${VIRTUAL_MACHINE_IMAGE_TYPE}"
+VIRTUAL_MACHINE_DISK = "${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}"
+
+def set_convert_options(d):
+   format = d.getVar("VIRTUAL_MACHINE_IMAGE_TYPE")
+   if format == "vmdk":
+      return "-o subformat=%s" % d.getVar("VMDK_SUBFORMAT")
+   else:
+      return ""
+
+
+CONVERSION_OPTIONS = "${@set_convert_options(d)}"
+
+do_convert_wic() {
+    rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
+    image_do_mounts
+    bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${WIC_IMAGE_FILE}"
+    sudo -E  chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+    /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \
+        '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}'
+    image_undo_mounts
+}
+
+addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template
+
+# User settings for OVA
+OVA_NAME ?= "${IMAGE_FULLNAME}"
+OVA_MEMORY ?= "8192"
+OVA_NUMBER_OF_CPU ?= "4"
+OVA_VRAM ?= "64"
+OVA_FIRMWARE ?= "efi"
+OVA_ACPI ?= "true"
+OVA_3D_ACCEL ?= "false"
+OVA_CLIPBOARD ?= "bidirectional"
+OVA_SHA_ALG = "1"
+
+# Generate random MAC addresses just as VirtualBox does, the format is
+# their assigned prefix for the first 3 bytes followed by 3 random bytes.
+VBOX_MAC_PREFIX = "080027"
+
+macgen() {
+    hexdump -n3 -e "\"${VBOX_MAC_PREFIX}%06X\n\"" /dev/urandom
+}
+
+OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \
+            OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL OVA_CLIPBOARD \
+            OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE"
+
+# the ovf template is updated with ensubst
+# this function adds the variable from OVA_VARS to the environment
+python update_environment() {
+    template_vars = (d.getVar('OVA_VARS', True) or "").split()
+    if len(template_vars) == 0:
+        return
+
+    for varname in template_vars:
+        value = d.getVar(varname, True)
+        if value:
+            os.environ.update({varname: value})
+}
+
+do_create_ova[prefuncs] += "update_environment"
+do_create_ova() {
+    if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then
+        exit 0
+    fi
+    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ova'
+    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ovf'
+    rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.mf'
+
+    export PRIMARY_MAC=$(macgen)
+    export SECONDARY_MAC=$(macgen)
+    export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK})
+    export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
+
+    image_do_mounts
+
+    sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO'
+        export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \
+                                 | gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}')
+        export DISK_UUID=$(uuidgen)
+        export VM_UUID=$(uuidgen)
+        # create ovf
+        cat /usr/share/vm-template/vm-template.ovf.tmpl | envsubst > ${PP_DEPLOY}/${OVA_NAME}.ovf
+        tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.ovf
+
+        # VirtualBox needs here a manifest file. VMware does accept that format.
+        if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" ]; then
+            echo "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf
+            echo "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf
+            tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.mf
+        fi
+        tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE}
+EOSUDO
+    image_undo_mounts
+}
+
+addtask do_create_ova after do_convert_wic before do_deploy
diff --git a/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
new file mode 100644
index 0000000..e6b5305
--- /dev/null
+++ b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl
@@ -0,0 +1,155 @@
+<?xml version="1.0"?>
+<Envelope ovf:version="1.0" xml:lang="en-US" xmlns="http://schemas.dmtf.org/ovf/envelope/1" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:vbox="http://www.virtualbox.org/ovf/machine">
+  <References>
+    <File ovf:href="${VIRTUAL_MACHINE_IMAGE_FILE}" ovf:id="file1"/>
+  </References>
+  <DiskSection>
+    <Info>List of the virtual disks used in the package</Info>
+    <Disk ovf:capacity="${DISK_SIZE_BYTES}" ovf:diskId="vmdisk1" ovf:fileRef="file1" ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#${VMDK_SUBFORMAT}" vbox:uuid="${DISK_UUID}"/>
+  </DiskSection>
+  <NetworkSection>
+    <Info>Logical networks used in the package</Info>
+    <Network ovf:name="NAT">
+      <Description>Logical network used by this appliance.</Description>
+    </Network>
+  </NetworkSection>
+  <VirtualSystem ovf:id="${OVA_NAME}">
+    <Info>A virtual machine</Info>
+    <OperatingSystemSection ovf:id="96">
+      <Info>The kind of installed guest operating system</Info>
+      <Description>Debian_64</Description>
+      <vbox:OSType ovf:required="false">Debian_64</vbox:OSType>
+    </OperatingSystemSection>
+    <VirtualHardwareSection>
+      <Info>Virtual hardware requirements for a virtual machine</Info>
+      <System>
+        <vssd:ElementName>Virtual Hardware Family</vssd:ElementName>
+        <vssd:InstanceID>0</vssd:InstanceID>
+        <vssd:VirtualSystemIdentifier>${OVA_NAME}</vssd:VirtualSystemIdentifier>
+        <vssd:VirtualSystemType>virtualbox-2.2</vssd:VirtualSystemType>
+      </System>
+      <Item>
+        <rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
+        <rasd:Caption>${OVA_NUMBER_OF_CPU} virtual CPU</rasd:Caption>
+        <rasd:Description>Number of virtual CPUs</rasd:Description>
+        <rasd:ElementName>${OVA_NUMBER_OF_CPU} virtual CPU</rasd:ElementName>
+        <rasd:InstanceID>1</rasd:InstanceID>
+        <rasd:ResourceType>3</rasd:ResourceType>
+        <rasd:VirtualQuantity>${OVA_NUMBER_OF_CPU}</rasd:VirtualQuantity>
+      </Item>
+      <Item>
+        <rasd:AllocationUnits>MegaBytes</rasd:AllocationUnits>
+        <rasd:Caption>${OVA_MEMORY} MB of memory</rasd:Caption>
+        <rasd:Description>Memory Size</rasd:Description>
+        <rasd:ElementName>${OVA_MEMORY} MB of memory</rasd:ElementName>
+        <rasd:InstanceID>2</rasd:InstanceID>
+        <rasd:ResourceType>4</rasd:ResourceType>
+        <rasd:VirtualQuantity>${OVA_MEMORY}</rasd:VirtualQuantity>
+      </Item>
+      <Item>
+        <rasd:Address>0</rasd:Address>
+        <rasd:Caption>ideController0</rasd:Caption>
+        <rasd:Description>IDE Controller</rasd:Description>
+        <rasd:ElementName>ideController0</rasd:ElementName>
+        <rasd:InstanceID>3</rasd:InstanceID>
+        <rasd:ResourceSubType>PIIX4</rasd:ResourceSubType>
+        <rasd:ResourceType>5</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:Address>1</rasd:Address>
+        <rasd:Caption>ideController1</rasd:Caption>
+        <rasd:Description>IDE Controller</rasd:Description>
+        <rasd:ElementName>ideController1</rasd:ElementName>
+        <rasd:InstanceID>4</rasd:InstanceID>
+        <rasd:ResourceSubType>PIIX4</rasd:ResourceSubType>
+        <rasd:ResourceType>5</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
+        <rasd:Caption>Ethernet adapter on 'NAT'</rasd:Caption>
+        <rasd:Connection>NAT</rasd:Connection>
+        <rasd:ElementName>Ethernet adapter on 'NAT'</rasd:ElementName>
+        <rasd:InstanceID>5</rasd:InstanceID>
+        <rasd:ResourceSubType>E1000</rasd:ResourceSubType>
+        <rasd:ResourceType>10</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:AddressOnParent>0</rasd:AddressOnParent>
+        <rasd:Caption>disk1</rasd:Caption>
+        <rasd:Description>Disk Image</rasd:Description>
+        <rasd:ElementName>disk1</rasd:ElementName>
+        <rasd:HostResource>/disk/vmdisk1</rasd:HostResource>
+        <rasd:InstanceID>6</rasd:InstanceID>
+        <rasd:Parent>3</rasd:Parent>
+        <rasd:ResourceType>17</rasd:ResourceType>
+      </Item>
+      <vmw:Config ovf:required="false" vmw:key="firmware" vmw:value="${OVA_FIRMWARE}"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.afterPowerOn" vmw:value="true"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.afterResume" vmw:value="true"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestShutdown" vmw:value="true"/>
+      <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestStandby" vmw:value="true"/>
+      <vmw:ExtraConfig ovf:required="false" vmw:key="virtualHW.productCompatibility" vmw:value="hosted"/>
+      </VirtualHardwareSection>
+      <vbox:Machine ovf:required="false" version="1.12-linux" uuid="{${VM_UUID}}" name="${OVA_NAME}" OSType="Debian_64" snapshotFolder="Snapshots" lastStateChange="${LAST_CHANGE}">
+        <ovf:Info>Complete VirtualBox machine configuration in VirtualBox format</ovf:Info>
+        <Hardware>
+          <CPU count="${OVA_NUMBER_OF_CPU}">
+            <PAE enabled="true"/>
+            <HardwareVirtExLargePages enabled="false"/>
+          </CPU>
+          <Memory RAMSize="${OVA_MEMORY}"/>
+          <Firmware type="${OVA_FIRMWARE_VIRTUALBOX}"/>
+          <Boot>
+            <Order position="1" device="HardDisk"/>
+            <Order position="2" device="None"/>
+            <Order position="3" device="None"/>
+            <Order position="4" device="None"/>
+          </Boot>
+          <Display VRAMSize="${OVA_VRAM}" monitorCount="1" accelerate3D="${OVA_3D_ACCEL}" accelerate2DVideo="false"/>
+          <VideoRecording enabled="false" file="Test.webm" horzRes="640" vertRes="480"/>
+          <RemoteDisplay enabled="false" authType="Null"/>
+          <BIOS>
+            <IOAPIC enabled="${OVA_ACPI}"/>
+          </BIOS>
+          <USBController enabled="false" enabledEhci="false"/>
+          <Network>
+            <Adapter slot="0" enabled="true" MACAddress="${PRIMARY_MAC}" cable="true" speed="0" type="virtio">
+              <DisabledModes/>
+              <NAT>
+                <DNS pass-domain="true" use-proxy="false" use-host-resolver="false"/>
+                <Alias logging="false" proxy-only="false" use-same-ports="false"/>
+              </NAT>
+            </Adapter>
+          </Network>
+          <LPT>
+            <Port slot="1" enabled="false" IOBase="0x378" IRQ="7"/>
+          </LPT>
+          <AudioAdapter driver="Pulse" enabled="false"/>
+          <RTC localOrUTC="local"/>
+          <SharedFolders/>
+          <Clipboard mode="Disabled"/>
+          <DragAndDrop mode="Disabled"/>
+          <IO>
+            <IoCache enabled="true" size="5"/>
+            <BandwidthGroups/>
+          </IO>
+          <HostPci>
+            <Devices/>
+          </HostPci>
+          <EmulatedUSB>
+            <CardReader enabled="false"/>
+          </EmulatedUSB>
+          <Guest memoryBalloonSize="0"/>
+          <GuestProperties/>
+        </Hardware>
+        <StorageControllers>
+          <StorageController name="IDE Controller" type="PIIX4" PortCount="2" useHostIOCache="true" Bootable="true">
+            <AttachedDevice type="HardDisk" port="0" device="0">
+              <Image uuid="{${DISK_UUID}}"/>
+            </AttachedDevice>
+          </StorageController>
+        </StorageControllers>
+      </vbox:Machine>
+  </VirtualSystem>
+</Envelope>
diff --git a/meta/recipes-devtools/vm-template/vm-template_0.1.bb b/meta/recipes-devtools/vm-template/vm-template_0.1.bb
new file mode 100644
index 0000000..1d474cd
--- /dev/null
+++ b/meta/recipes-devtools/vm-template/vm-template_0.1.bb
@@ -0,0 +1,16 @@
+# This software is a part of ISAR.
+#
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg-raw
+
+SRC_URI += "file://vm-template.ovf.tmpl"
+
+do_install() {
+    TARGET=${D}/usr/share/vm-template
+    install -m 0755 -d ${TARGET}
+    install -m 0740 ${WORKDIR}/vm-template.ovf.tmpl \
+        ${TARGET}/vm-template.ovf.tmpl
+}
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 49d6aa8..b1bbe51 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -43,6 +43,7 @@ TARGETS_SET="\
             mc:nand-ubi-demo-buster:isar-image-ubi \
             mc:rpi-stretch:isar-image-base \
             mc:qemuamd64-focal:isar-image-base \
+            mc:virtualbox-ova-buster:isar-image-base \
             "
           # qemu-user-static of <= buster too old to build that
           # mc:qemuarm64-buster:isar-image-base
-- 
2.20.1

[PATCH v14 2/2] meta/classes: add cpiogz-img

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This allows to create a initramfs like filesystem.

Based on original patch by Michael Adler <michael.adler@siemens.com>.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
[Jan: reformatting, dropped bullseye target, added to local.conf]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 meta-isar/conf/local.conf.sample              |  1 +
 .../multiconfig/qemuamd64-buster-cpiogz.conf  |  9 ++++++++
 meta/classes/cpiogz-img.bbclass               | 22 +++++++++++++++++++
 scripts/ci_build.sh                           |  1 +
 4 files changed, 33 insertions(+)
 create mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
 create mode 100644 meta/classes/cpiogz-img.bbclass

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 7b20a62..fcdc277 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -50,6 +50,7 @@ BBMULTICONFIG = " \
     qemuamd64-stretch \
     qemuamd64-buster \
     qemuamd64-buster-tgz \
+    qemuamd64-buster-cpiogz \
     qemuamd64-bullseye \
     container-amd64-stretch \
     container-amd64-buster \
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
new file mode 100644
index 0000000..cadcdac
--- /dev/null
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
@@ -0,0 +1,9 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2020
+#
+# SPDX-License-Identifier: MIT
+
+MACHINE ?= "qemuamd64"
+DISTRO ?= "debian-buster"
+
+IMAGE_TYPE = "cpiogz-img"
diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
new file mode 100644
index 0000000..8e90acb
--- /dev/null
+++ b/meta/classes/cpiogz-img.bbclass
@@ -0,0 +1,22 @@
+# This software is a part of ISAR.
+# Copyright (C) 2020 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+CPIOGZ_FNAME ?= "${IMAGE_FULLNAME}.cpio.gz"
+CPIOGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${CPIOGZ_FNAME}"
+IMAGER_INSTALL += "cpio"
+CPIO_IMAGE_FORMAT ?= "newc"
+
+do_cpiogz_image() {
+    sudo rm -f ${CPIOGZ_IMAGE_FILE}
+    image_do_mounts
+    sudo chroot ${BUILDCHROOT_DIR} \
+                sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
+                       /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o | /usr/bin/gzip -9 > \
+                       ${PP_DEPLOY}/${CPIOGZ_FNAME}"
+    sudo chown $(id -u):$(id -g) ${CPIOGZ_IMAGE_FILE}
+    image_undo_mounts
+}
+
+addtask cpiogz_image before do_image after do_image_tools
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index b1bbe51..a2895f6 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -37,6 +37,7 @@ TARGETS_SET="\
             mc:qemuamd64-stretch:isar-image-base \
             mc:qemuamd64-buster:isar-image-base \
             mc:qemuamd64-buster-tgz:isar-image-base \
+            mc:qemuamd64-buster-cpiogz:isar-image-base \
             mc:qemuamd64-buster:isar-initramfs \
             mc:qemumipsel-stretch:isar-image-base \
             mc:qemumipsel-buster:isar-image-base \
-- 
2.20.1

Re: [PATCH v14 0/2] CPIO & OVA Images

[Anton Mikanovich]

08.07.2021 11:42, Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>
> - Generate cpio images for rescue images
> - generate ova images for VMWare & Virtualbox
>
We've experienced floating issue during full CI testing if include this 
patchset:

00:14:20 INFO: Creating image(s)...
00:14:20
00:14:20 ERROR: _exec_cmd: export 
PATH=//sbin://usr/sbin://usr/bin://bin:None:$PATH;export 
PSEUDO_PREFIX=//usr;export 
PSEUDO_LOCALSTATEDIR=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs/../pseudo;export 
PSEUDO_PASSWD=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs;export 
PSEUDO_NOSYMLINKEXP=1;export 
PSEUDO_IGNORE_PATHS=/tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4,/;/workspace/build/isar_am_devel/110/scripts/wic_fakeroot 
fsck.ext4 -pvfD 
/tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4 
returned '127' instead of 0
00:14:20 output: /bin/sh: 1: 
/workspace/build/isar_am_devel/110/scripts/wic_fakeroot: not found
00:14:20
00:14:20 WARNING: exit code 1 from a shell command.

It doesn't looks like something virtual images related, so not sure if 
it was caused by this patchset.
But for some reason it was happened only if CPIO&OVA v14 was included.

The failed builds are:
http://ci.isar-build.org:8080/job/isar_am_devel/110/consoleFull
http://ci.isar-build.org:8080/job/isar_am_devel/111/consoleFull
http://ci.isar-build.org:8080/job/isar_am_devel/115/consoleFull
http://ci.isar-build.org:8080/job/isar_am_devel/123/consoleFull

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v14 0/2] CPIO & OVA Images

[Gylstorff Quirin]

I will test it on our site. with the patchset applied to next( 
e274130b870b7e31532fe191b9932cde1d819b4b).

Quirin

On 8/20/21 11:07 AM, Anton Mikanovich wrote:
> 08.07.2021 11:42, Q. Gylstorff wrote:
>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>>
>> - Generate cpio images for rescue images
>> - generate ova images for VMWare & Virtualbox
>>
> We've experienced floating issue during full CI testing if include this 
> patchset:
> 
> 00:14:20 INFO: Creating image(s)...
> 00:14:20
> 00:14:20 ERROR: _exec_cmd: export 
> PATH=//sbin://usr/sbin://usr/bin://bin:None:$PATH;export 
> PSEUDO_PREFIX=//usr;export 
> PSEUDO_LOCALSTATEDIR=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs/../pseudo;export 
> PSEUDO_PASSWD=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs;export 
> PSEUDO_NOSYMLINKEXP=1;export 
> PSEUDO_IGNORE_PATHS=/tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4,/;/workspace/build/isar_am_devel/110/scripts/wic_fakeroot 
> fsck.ext4 -pvfD 
> /tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4 
> returned '127' instead of 0
> 00:14:20 output: /bin/sh: 1: 
> /workspace/build/isar_am_devel/110/scripts/wic_fakeroot: not found
> 00:14:20
> 00:14:20 WARNING: exit code 1 from a shell command.
> 
> It doesn't looks like something virtual images related, so not sure if 
> it was caused by this patchset.
> But for some reason it was happened only if CPIO&OVA v14 was included.
> 
> The failed builds are:
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F110%2FconsoleFull&amp;data=04%7C01%7C6549bcd2-981c-4c06-8e1b-b5c6cc3441b4%40ad011.siemens.com%7C442a9219ee3b41688f2d08d963ba04f1%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650472859000759%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=pQ8aWLsweh%2Fkml4q%2BxtAVbjLj6HpuonxhQe7HXgkc5k%3D&amp;reserved=0 
> 
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F111%2FconsoleFull&amp;data=04%7C01%7C6549bcd2-981c-4c06-8e1b-b5c6cc3441b4%40ad011.siemens.com%7C442a9219ee3b41688f2d08d963ba04f1%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650472859000759%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=si7qQ2PIB1nvCsNtTlAKVQ6zgpgoMwSA9z%2FObQV9Tok%3D&amp;reserved=0 
> 
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F115%2FconsoleFull&amp;data=04%7C01%7C6549bcd2-981c-4c06-8e1b-b5c6cc3441b4%40ad011.siemens.com%7C442a9219ee3b41688f2d08d963ba04f1%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650472859010754%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=KuV4mhnc8piuKaAfYImmx7VHzRykEyvTn6aCwWfnde8%3D&amp;reserved=0 
> 
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F123%2FconsoleFull&amp;data=04%7C01%7C6549bcd2-981c-4c06-8e1b-b5c6cc3441b4%40ad011.siemens.com%7C442a9219ee3b41688f2d08d963ba04f1%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650472859010754%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=1cDZUp%2BcwQ30sTqD2qYGKdMXpk9pN6EvF3ldB9TvrjM%3D&amp;reserved=0 
> 
> 

Re: [PATCH v14 0/2] CPIO & OVA Images

[Jan Kiszka]

On 07.09.21 12:48, Gylstorff Quirin wrote:
> I will test it on our site. with the patchset applied to next(
> e274130b870b7e31532fe191b9932cde1d819b4b).
> 

Full CI with your patches applied passed our nightly run today.

Jan

> Quirin
> 
> On 8/20/21 11:07 AM, Anton Mikanovich wrote:
>> 08.07.2021 11:42, Q. Gylstorff wrote:
>>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>>>
>>> - Generate cpio images for rescue images
>>> - generate ova images for VMWare & Virtualbox
>>>
>> We've experienced floating issue during full CI testing if include
>> this patchset:
>>
>> 00:14:20 INFO: Creating image(s)...
>> 00:14:20
>> 00:14:20 ERROR: _exec_cmd: export
>> PATH=//sbin://usr/sbin://usr/bin://bin:None:$PATH;export
>> PSEUDO_PREFIX=//usr;export
>> PSEUDO_LOCALSTATEDIR=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs/../pseudo;export
>> PSEUDO_PASSWD=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs;export
>> PSEUDO_NOSYMLINKEXP=1;export
>> PSEUDO_IGNORE_PATHS=/tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4,/;/workspace/build/isar_am_devel/110/scripts/wic_fakeroot
>> fsck.ext4 -pvfD
>> /tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4
>> returned '127' instead of 0
>> 00:14:20 output: /bin/sh: 1:
>> /workspace/build/isar_am_devel/110/scripts/wic_fakeroot: not found
>> 00:14:20
>> 00:14:20 WARNING: exit code 1 from a shell command.
>>
>> It doesn't looks like something virtual images related, so not sure if
>> it was caused by this patchset.
>> But for some reason it was happened only if CPIO&OVA v14 was included.
>>
>> The failed builds are:
>> http://ci.isar-build.org:8080/job/isar_am_devel/110/consoleFull
>>
>> http://ci.isar-build.org:8080/job/isar_am_devel/111/consoleFull
>>
>> http://ci.isar-build.org:8080/job/isar_am_devel/115/consoleFull
>>
>> http://ci.isar-build.org:8080/job/isar_am_devel/123/consoleFull
>>
>>

Re: [PATCH v14 0/2] CPIO & OVA Images

[Anton Mikanovich]

07.09.2021 16:06, Jan Kiszka wrote:
> On 07.09.21 12:48, Gylstorff Quirin wrote:
>> I will test it on our site. with the patchset applied to next(
>> e274130b870b7e31532fe191b9932cde1d819b4b).
>>
> Full CI with your patches applied passed our nightly run today.
>
> Jan
>
As I've already mentioned, the issue is floating.
We have at least 4 failed builds in our CI, but even more without any 
issues (with v14 in both groups). And none builds with the same issue 
without v14 applied.
The last time it was seen at 2021-08-12, but I'm not able to reproduce 
the issue after rebase on next.

Anyway now we need to rebase again to remove image_undo_mounts that was 
gone after the last revert.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v14 0/2] CPIO & OVA Images

[Anton Mikanovich]

07.09.2021 16:06, Jan Kiszka wrote:
> On 07.09.21 12:48, Gylstorff Quirin wrote:
>> I will test it on our site. with the patchset applied to next(
>> e274130b870b7e31532fe191b9932cde1d819b4b).
>>
> Full CI with your patches applied passed our nightly run today.
>
> Jan
>
I was testing v15 on top of current next and the issue was not reproduced.
So propose to merge v15 but keep in mind this thread in case the issue 
will happen again.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v14 0/2] CPIO & OVA Images

[Henning Schild]

I kind of want to bet this has to do with mounting again.
generate_wic_image does quite a bit of layer mounting and umounting. If
mounting the base isar layer fails that script can probably "go
missing". And i am not sure those mounts will be cleaned up when the
task does not finish correctly.

regards,
Henning

Am Fri, 20 Aug 2021 12:07:53 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:

> 08.07.2021 11:42, Q. Gylstorff wrote:
> > From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> >
> > - Generate cpio images for rescue images
> > - generate ova images for VMWare & Virtualbox
> >  
> We've experienced floating issue during full CI testing if include
> this patchset:
> 
> 00:14:20 INFO: Creating image(s)...
> 00:14:20
> 00:14:20 ERROR: _exec_cmd: export 
> PATH=//sbin://usr/sbin://usr/bin://bin:None:$PATH;export 
> PSEUDO_PREFIX=//usr;export 
> PSEUDO_LOCALSTATEDIR=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs/../pseudo;export 
> PSEUDO_PASSWD=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs;export 
> PSEUDO_NOSYMLINKEXP=1;export 
> PSEUDO_IGNORE_PATHS=/tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4,/;/workspace/build/isar_am_devel/110/scripts/wic_fakeroot 
> fsck.ext4 -pvfD 
> /tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4 
> returned '127' instead of 0
> 00:14:20 output: /bin/sh: 1: 
> /workspace/build/isar_am_devel/110/scripts/wic_fakeroot: not found
> 00:14:20
> 00:14:20 WARNING: exit code 1 from a shell command.
> 
> It doesn't looks like something virtual images related, so not sure
> if it was caused by this patchset.
> But for some reason it was happened only if CPIO&OVA v14 was included.
> 
> The failed builds are:
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F110%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333123853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=N%2Bwdk5CdmnSnxZSYawz9V9N5Doo5lBsWrU5bUodPNfY%3D&amp;reserved=0
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F111%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333123853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=AOlB1MufUym42Fx9MxA9i75lM%2FmgDYuH6pqYI93Ibek%3D&amp;reserved=0
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F115%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333133842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=W7WUqLyo0VrLMGiJMEUc%2Bpvoqzj7xutMRo2wjhmBPnE%3D&amp;reserved=0
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F123%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333133842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=4z1JroTZd%2Fdwf7T%2F4ptKvV30AfdEYvQqgu0p861EbSw%3D&amp;reserved=0
> 

Re: [PATCH v14 0/2] CPIO & OVA Images

[Henning Schild]

I just read some logs where three wic images are created in the same
chroot.

The first one failed to umount tmp

> umount: /builds/.../build/tmp: target is busy.

Followed by the other two not being able to find wic_fakeroot.

I guess we will have to see how to react on mount/umount issues to make
them fail ... and not some follow-up.

And roughly looking at the code i guess we will have to grab a
buildchroot lock in do_wic_image. In fact i guess the locking of the
buildchroot should probably be implemented inside buildchroot_do_mounts.

Henning

Am Tue, 14 Sep 2021 12:54:45 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> I kind of want to bet this has to do with mounting again.
> generate_wic_image does quite a bit of layer mounting and umounting.
> If mounting the base isar layer fails that script can probably "go
> missing". And i am not sure those mounts will be cleaned up when the
> task does not finish correctly.
> 
> regards,
> Henning
> 
> Am Fri, 20 Aug 2021 12:07:53 +0300
> schrieb Anton Mikanovich <amikan@ilbers.de>:
> 
> > 08.07.2021 11:42, Q. Gylstorff wrote:  
> > > From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> > >
> > > - Generate cpio images for rescue images
> > > - generate ova images for VMWare & Virtualbox
> > >    
> > We've experienced floating issue during full CI testing if include
> > this patchset:
> > 
> > 00:14:20 INFO: Creating image(s)...
> > 00:14:20
> > 00:14:20 ERROR: _exec_cmd: export 
> > PATH=//sbin://usr/sbin://usr/bin://bin:None:$PATH;export 
> > PSEUDO_PREFIX=//usr;export 
> > PSEUDO_LOCALSTATEDIR=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs/../pseudo;export 
> > PSEUDO_PASSWD=/workspace/build/isar_am_devel/110/build/tmp/work/debian-buster-amd64/isar-image-base-qemuamd64-wic-img/1.0-r0/rootfs;export 
> > PSEUDO_NOSYMLINKEXP=1;export 
> > PSEUDO_IGNORE_PATHS=/tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4,/;/workspace/build/isar_am_devel/110/scripts/wic_fakeroot 
> > fsck.ext4 -pvfD 
> > /tmp/tmp.nSYNbkMHKq/isar-image-base-debian-buster-qemuamd64.wic/tmp.wic.4t1qlax6/rootfs_platform.2.ext4 
> > returned '127' instead of 0
> > 00:14:20 output: /bin/sh: 1: 
> > /workspace/build/isar_am_devel/110/scripts/wic_fakeroot: not found
> > 00:14:20
> > 00:14:20 WARNING: exit code 1 from a shell command.
> > 
> > It doesn't looks like something virtual images related, so not sure
> > if it was caused by this patchset.
> > But for some reason it was happened only if CPIO&OVA v14 was
> > included.
> > 
> > The failed builds are:
> > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F110%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333123853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=N%2Bwdk5CdmnSnxZSYawz9V9N5Doo5lBsWrU5bUodPNfY%3D&amp;reserved=0
> > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F111%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333123853%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=AOlB1MufUym42Fx9MxA9i75lM%2FmgDYuH6pqYI93Ibek%3D&amp;reserved=0
> > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F115%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333133842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=W7WUqLyo0VrLMGiJMEUc%2Bpvoqzj7xutMRo2wjhmBPnE%3D&amp;reserved=0
> > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel%2F123%2FconsoleFull&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C11c07c50cd7c46b53c8d08d963ba0572%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637650473333133842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=4z1JroTZd%2Fdwf7T%2F4ptKvV30AfdEYvQqgu0p861EbSw%3D&amp;reserved=0
> >   
>