From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6952976920918097920 X-Received: by 2002:a05:6512:308a:: with SMTP id z10mr17746963lfd.15.1619543995140; Tue, 27 Apr 2021 10:19:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:7c0c:: with SMTP id x12ls6644640ljc.0.gmail; Tue, 27 Apr 2021 10:19:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxeWoXZqT31F57xVt+OKbVWU3qU3Jj1pxJMT+rk3A4jPkL0+BiTON7HgSjBq5aI0QdbJUVR X-Received: by 2002:a2e:9495:: with SMTP id c21mr17346947ljh.287.1619543993896; Tue, 27 Apr 2021 10:19:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619543993; cv=none; d=google.com; s=arc-20160816; b=pItzZMfmOuT+7uV8Oq1sUKnw7C3DrnA2sK0HArJlAlcg//pRKyVg9DUeq+iOd+q3rA U8qQcGRJnbzPesKK1iqUo0arKVMutXxJZISBLxoI2Fc55o7Dc+pbE15RaNYCaS4dK/wB 6m8ARWNYXcwGg0jyVFMDBjHYn/jLAM5mn3aElM13VuT042V1EB1zTQuQ+Qffjy9slxrg DCznpd1oKGd9BbrBvwDIVcZFf3NyZlgBkisbPBvqozdH2yFkQzmlhvg+GjuPLBELev/D DdgNduxVHOiU4xjJ5Wj328h29OAiBwn9BJZkgQYOJPP0ZvzrXvadFeEWOsWP5ViyGvjH bbVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=vVpQ72dEWzYz09sOs3rPH/hFe3VcIWRZBrajqWoBk+8=; b=TKumWquup7JmydjMt/YPcQ8NglpdoHgfBTAsQ/apEIB/I33/xbBQHpObW23S3cD/l1 TU8gVo8bOXncBHY1PnM22MoUQCQAsyVBCLa/RcxWKet6UmYHVsbqljU25huOuJPPDouM +We8qW2HhGomUATcQGNHxw5cuZ78VFwy26guwt2iL5uvlnYNqeOALPTggzj1YW7/CTYs BqOzVtBfcVZoosXTMl7T6YDJ5ePJHc2M5w2wLZJmqWvgVKchfWJo+dUjbAyN0iSj0n7S z1OJ9uq7IA+Qd5e1uYxx5lLM4+9sig6wuD44PXO6MvUp/pwkHT9NheGBIb78LyllamGt I4Ag== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id c11si1368851lfh.4.2021.04.27.10.19.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Apr 2021 10:19:53 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 13RHJqmu013126 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 27 Apr 2021 19:19:52 +0200 Received: from md1za8fc.ad001.siemens.net ([167.87.26.73]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 13RHGEqh027973; Tue, 27 Apr 2021 19:16:14 +0200 Date: Tue, 27 Apr 2021 19:16:13 +0200 From: Henning Schild To: "[ext] Q. Gylstorff" Cc: Jan.Kiszka@siemens.com, isar-users@googlegroups.com Subject: Re: [PATCH v11 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Message-ID: <20210427191613.5f04147e@md1za8fc.ad001.siemens.net> In-Reply-To: <20210422083223.2751-2-Quirin.Gylstorff@siemens.com> References: <20210422083223.2751-1-Quirin.Gylstorff@siemens.com> <20210422083223.2751-2-Quirin.Gylstorff@siemens.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 9ech1umQCIFr Am Thu, 22 Apr 2021 10:32:22 +0200 schrieb "[ext] Q. Gylstorff" : > From: Quirin Gylstorff > > This allows to generate a ova file for virtualbox or vmware. The > images differ in the setting of the variable `VMDK_SUBFORMAT`. > - `streamOptimized` is used for Vmware Workstation > - `monolithicSparse` is used for Virtualbox > > This is necessary as virtualbox throws an import error for a > `streamOptimized` version. The ova for Virtualbox is also bigger due > the disk format. > > The default machine settings are: > - 4 CPU Cores with 8GB Ram > > Signed-off-by: Quirin Gylstorff > [Jan: shortened named, reformatting, massaged comments, added to > local.conf] Signed-off-by: Jan Kiszka > --- > doc/user_manual.md | 1 + > meta-isar/conf/local.conf.sample | 1 + > meta-isar/conf/machine/virtualbox.conf | 16 ++ > meta-isar/conf/machine/vmware.conf | 16 ++ > .../multiconfig/virtualbox-ova-buster.conf | 8 + > meta/classes/vm-img.bbclass | 122 ++++++++++++++ > .../vm-template/files/vm-template.ovf.tmpl | 155 > ++++++++++++++++++ .../vm-template/vm-template_0.1.bb | > 16 ++ scripts/ci_build.sh | 1 + > 9 files changed, 336 insertions(+) > create mode 100644 meta-isar/conf/machine/virtualbox.conf > create mode 100644 meta-isar/conf/machine/vmware.conf > create mode 100644 > meta-isar/conf/multiconfig/virtualbox-ova-buster.conf create mode > 100644 meta/classes/vm-img.bbclass create mode 100644 > meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl create > mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index fec9896..2749ef0 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -454,6 +454,7 @@ Isar can generate various images types for > specific machine. The type of the ima > - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image > (default option for the `rpi` machine). > - `wic-img` - A full disk image with user-specified partitions > created and populated using the wic tool. > - `ubi-img` - A image for use on mtd nand partitions employing UBI > + - `vm-img` - A image for use on VirtualBox or VMware > > --- > > diff --git a/meta-isar/conf/local.conf.sample > b/meta-isar/conf/local.conf.sample index 77585ec..6c1d299 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -64,6 +64,7 @@ BBMULTICONFIG = " \ > nand-ubi-demo-buster \ > nanopi-neo-buster \ > stm32mp15x-buster \ > + virtualbox-ova-buster \ > rpi-stretch \ > sifive-fu540-sid-ports \ > qemuarm64-focal \ > diff --git a/meta-isar/conf/machine/virtualbox.conf > b/meta-isar/conf/machine/virtualbox.conf new file mode 100644 > index 0000000..8f33ae4 > --- /dev/null > +++ b/meta-isar/conf/machine/virtualbox.conf > @@ -0,0 +1,16 @@ > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > +DISTRO_ARCH ?= "amd64" > + > +KERNEL_NAME ?= "amd64" > +BOOTLOADER ?= "grub" What is BOOTLOADER? i think that is not needed because WKF_FILE decides that. Henning > + > +WKS_FILE ?= "sdimage-efi" > + > +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" > + > +VMDK_SUBFORMAT = "monolithicSparse" > +IMAGE_TYPE ?= "vm-img" > diff --git a/meta-isar/conf/machine/vmware.conf > b/meta-isar/conf/machine/vmware.conf new file mode 100644 > index 0000000..02a349a > --- /dev/null > +++ b/meta-isar/conf/machine/vmware.conf > @@ -0,0 +1,16 @@ > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > +DISTRO_ARCH ?= "amd64" > + > +KERNEL_NAME ?= "amd64" > +BOOTLOADER ?= "grub" > + > +WKS_FILE ?= "sdimage-efi" > + > +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" > + > +VMDK_SUBFORMAT = "streamOptimized" > +IMAGE_TYPE ?= "vm-img" > diff --git a/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf > b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf new file mode > 100644 index 0000000..3042556 > --- /dev/null > +++ b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf > @@ -0,0 +1,8 @@ > +# > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > + > +MACHINE = "virtualbox" > +DISTRO = "debian-buster" > diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass > new file mode 100644 > index 0000000..d5fc3ac > --- /dev/null > +++ b/meta/classes/vm-img.bbclass > @@ -0,0 +1,122 @@ > +# This software is a part of ISAR. > +# Copyright (C) 2019-2020 Siemens AG > +# > +# This class allows to generate images for VMware and VirtualBox > +# > + > +inherit buildchroot > +inherit wic-img > + > +IMAGER_BUILD_DEPS += "vm-template" > +IMAGER_INSTALL += "qemu-utils gawk uuid-runtime vm-template" > + > +# virtual machine disk settings > +SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img" > + > +# For VirtualBox, this needs to be "monolithicSparse" (default to > it). +# VMware needs this to be "streamOptimized". > +VMDK_SUBFORMAT ?= "monolithicSparse" > + > +VIRTUAL_MACHINE_IMAGE_TYPE ?= "vmdk" > +VIRTUAL_MACHINE_IMAGE_FILE = > "${IMAGE_FULLNAME}-disk001.${VIRTUAL_MACHINE_IMAGE_TYPE}" > +VIRTUAL_MACHINE_DISK = "${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}" + > +def set_convert_options(d): > + format = d.getVar("VIRTUAL_MACHINE_IMAGE_TYPE") > + if format == "vmdk": > + return "-o subformat=%s" % d.getVar("VMDK_SUBFORMAT") > + else: > + return "" > + > + > +CONVERSION_OPTIONS = "${@set_convert_options(d)}" > + > +do_convert_wic() { > + rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}' > + image_do_mounts > + bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from > ${WIC_IMAGE_FILE}" > + sudo -E chroot --userspec=$( id -u ):$( id -g ) > ${BUILDCHROOT_DIR} \ > + /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} > ${CONVERSION_OPTIONS} \ > + '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' > '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}' +} > + > +addtask convert_wic before do_build after do_wic_image > do_copy_boot_files do_install_imager_deps do_transform_template + > +# User settings for OVA > +OVA_NAME ?= "${IMAGE_FULLNAME}" > +OVA_MEMORY ?= "8192" > +OVA_NUMBER_OF_CPU ?= "4" > +OVA_VRAM ?= "64" > +OVA_FIRMWARE ?= "efi" > +OVA_ACPI ?= "true" > +OVA_3D_ACCEL ?= "false" > +OVA_CLIPBOARD ?= "bidirectional" > +OVA_SHA_ALG = "1" > + > +# Generate random MAC addresses just as VirtualBox does, the format > is +# their assigned prefix for the first 3 bytes followed by 3 > random bytes. +VBOX_MAC_PREFIX = "080027" > + > +macgen() { > + hexdump -n3 -e "\"${VBOX_MAC_PREFIX}%06X\n\"" /dev/urandom > +} > + > +get_disksize() { > + image_do_mounts > + sudo -E chroot --userspec=$( id -u ):$( id -g ) > ${BUILDCHROOT_DIR} \ > + qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" | gawk > 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}' +} > + > +OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \ > + OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL OVA_CLIPBOARD \ > + OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE" > + > +# the ovf template is updated with ensubst > +# this function adds the variable from OVA_VARS to the environment > +python update_environment() { > + template_vars = (d.getVar('OVA_VARS', True) or "").split() > + if len(template_vars) == 0: > + return > + > + for varname in template_vars: > + value = d.getVar(varname, True) > + if value: > + os.environ.update({varname: value}) > +} > + > +do_create_ova[prefuncs] += "update_environment" > +do_create_ova() { > + if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then > + exit 0 > + fi > + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ova' > + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ovf' > + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.mf' > + > + export PRIMARY_MAC=$(macgen) > + export SECONDARY_MAC=$(macgen) > + export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK}) > + export DISK_SIZE_BYTES=$(get_disksize) > + export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ") > + export OVA_FIRMWARE_VIRTUALBOX=$(echo ${OVA_FIRMWARE} | tr > '[a-z]' '[A-Z]') + > + image_do_mounts > + > + sudo -Es chroot --userspec=$( id -u ):$( id -g ) > ${BUILDCHROOT_DIR} <<'EOSUDO' > + export DISK_UUID=$(uuidgen) > + export VM_UUID=$(uuidgen) > + # create ovf > + cat /usr/share/vm-template/vm-template.ovf.tmpl | envsubst > > ${PP_DEPLOY}/${OVA_NAME}.ovf > + tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > ${OVA_NAME}.ovf + > + # VirtualBox needs here a manifest file. VMware does accept > that format. > + if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" ]; then > + echo > "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum > ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> > ${PP_DEPLOY}/${OVA_NAME}.mf > + echo > "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum > ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> > ${PP_DEPLOY}/${OVA_NAME}.mf > + tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > ${OVA_NAME}.mf > + fi > + tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > ${VIRTUAL_MACHINE_IMAGE_FILE} +EOSUDO > +} > + > +addtask do_create_ova after do_convert_wic before do_deploy > diff --git > a/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl > b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl new > file mode 100644 index 0000000..e6b5305 --- /dev/null > +++ b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl > @@ -0,0 +1,155 @@ > + > + xmlns="http://schemas.dmtf.org/ovf/envelope/1" > xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" > xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" > xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > xmlns:vbox="http://www.virtualbox.org/ovf/machine"> > + > + > + > + > + List of the virtual disks used in the package > + ovf:fileRef="file1" > ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#${VMDK_SUBFORMAT}" > vbox:uuid="${DISK_UUID}"/> > + > + > + Logical networks used in the package > + > + Logical network used by this > appliance. > + > + > + > + A virtual machine > + > + The kind of installed guest operating system > + Debian_64 > + Debian_64 > + > + > + Virtual hardware requirements for a virtual > machine > + > + Virtual Hardware Family > + 0 > + > ${OVA_NAME} > + > virtualbox-2.2 > + > + > + hertz * 10^6 > + ${OVA_NUMBER_OF_CPU} virtual CPU > + Number of virtual CPUs > + ${OVA_NUMBER_OF_CPU} virtual > CPU > + 1 > + 3 > + > ${OVA_NUMBER_OF_CPU} > + > + > + MegaBytes > + ${OVA_MEMORY} MB of memory > + Memory Size > + ${OVA_MEMORY} MB of > memory > + 2 > + 4 > + ${OVA_MEMORY} > + > + > + 0 > + ideController0 > + IDE Controller > + ideController0 > + 3 > + PIIX4 > + 5 > + > + > + 1 > + ideController1 > + IDE Controller > + ideController1 > + 4 > + PIIX4 > + 5 > + > + > + true > + Ethernet adapter on 'NAT' > + NAT > + Ethernet adapter on > 'NAT' > + 5 > + E1000 > + 10 > + > + > + 0 > + disk1 > + Disk Image > + disk1 > + /disk/vmdisk1 > + 6 > + 3 > + 17 > + > + vmw:value="${OVA_FIRMWARE}"/> > + vmw:key="tools.syncTimeWithHost" vmw:value="false"/> > + vmw:value="true"/> > + vmw:value="true"/> > + vmw:key="tools.beforeGuestShutdown" vmw:value="true"/> > + vmw:key="tools.beforeGuestStandby" vmw:value="true"/> > + vmw:key="virtualHW.productCompatibility" vmw:value="hosted"/> > + > + uuid="{${VM_UUID}}" name="${OVA_NAME}" OSType="Debian_64" > snapshotFolder="Snapshots" lastStateChange="${LAST_CHANGE}"> > + Complete VirtualBox machine configuration in > VirtualBox format > + > + > + > + > + > + > + > + > + > + > + > + > + > + accelerate3D="${OVA_3D_ACCEL}" accelerate2DVideo="false"/> > + horzRes="640" vertRes="480"/> > + > + > + > + > + > + > + MACAddress="${PRIMARY_MAC}" cable="true" speed="0" type="virtio"> > + > + > + use-host-resolver="false"/> > + use-same-ports="false"/> > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + PortCount="2" useHostIOCache="true" Bootable="true"> > + > + > + > + > + > + > + > + > diff --git a/meta/recipes-devtools/vm-template/vm-template_0.1.bb > b/meta/recipes-devtools/vm-template/vm-template_0.1.bb new file mode > 100644 index 0000000..1d474cd > --- /dev/null > +++ b/meta/recipes-devtools/vm-template/vm-template_0.1.bb > @@ -0,0 +1,16 @@ > +# This software is a part of ISAR. > +# > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > +inherit dpkg-raw > + > +SRC_URI += "file://vm-template.ovf.tmpl" > + > +do_install() { > + TARGET=${D}/usr/share/vm-template > + install -m 0755 -d ${TARGET} > + install -m 0740 ${WORKDIR}/vm-template.ovf.tmpl \ > + ${TARGET}/vm-template.ovf.tmpl > +} > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh > index adc22e4..a77cf5a 100755 > --- a/scripts/ci_build.sh > +++ b/scripts/ci_build.sh > @@ -43,6 +43,7 @@ TARGETS_SET="\ > mc:nand-ubi-demo-buster:isar-image-ubi \ > mc:rpi-stretch:isar-image-base \ > mc:qemuamd64-focal:isar-image-base \ > + mc:virtualbox-ova-buster:isar-image-base \ > " > # qemu-user-static of <= buster too old to build that > # mc:qemuarm64-buster:isar-image-base