From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6808879013169201152 X-Received: by 2002:a17:906:447:: with SMTP id e7mr3573863eja.172.1612523773412; Fri, 05 Feb 2021 03:16:13 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:6a87:: with SMTP id p7ls4415448ejr.10.gmail; Fri, 05 Feb 2021 03:16:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJyJpKo1M8imrhZLtPOMPkQSn69nNlQtEaweLdsFFqr1bs0y7UTAP9AzYconCsvZw5HaVzpK X-Received: by 2002:a17:907:98d7:: with SMTP id kd23mr3561667ejc.283.1612523772399; Fri, 05 Feb 2021 03:16:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612523772; cv=none; d=google.com; s=arc-20160816; b=k5rFU0AYbizgusFcqqDidJzVkyTdVAh4oeXbFD1E8pwZ6GyJEYAj81Gyza3LnOjLsG nD+mjc3GAbw68cQzcX5GCVcdHPyWs1KHNjhd8Nl5yz38jLukdGDZett9MlzoK6p2eXAr yVFpzr1kAiHkLkK9DJbzQ04wRcCFeyiHmNnfMbwjn/CApxRRDkdwgEvjY7vTzAAnantu GO4n8qpeAEdrsG4nSpy2xSlm8nOhmiBAW6fG59tXZkd7+wktG/LCQmYSpQdr0+W6IZ/q TsqFr4/GyxEle8jAMb3e7hWrXirnnRaC+irmCN1Q80NvnEj8dOpBo81/QiOi9GohUbl4 RMqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=LM8oLU4t4FtLggujBEYqHIS8rGf+4TIJRO+Khy6VONM=; b=hDNLnznqFYQNbruW1QfTgmoJKQKLJ6B3H14KA7BDjBfFbNROAYkJvC8bx6dzttJ93R OJGLNO8QvmN/OeW4dR9ihtaLnsTFJ+mPMJRgR8VN4EIDV0qFAu/Tqd6O5YyZLJB2XOw9 wqbL9PbsMTVKxvRLD07M3sq9JMdak35hGrjU0vDpSNUMlHtn1Ja3Qk+GSVQPub9or1kQ AwTSBw/ybqL0D9VSNdiBkx/IABjnxKNGlF8PFLbr+fRI1Ey/+66bwwW+7j1iiyLTtuMh iouWSWBW0LZW5oOa+lH8BZLglzvZW20fCuGUD+9dMSPoBxR9OGsXbZyX2ioCbZpV6gvF T0oQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id jz19si644081ejb.0.2021.02.05.03.16.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Feb 2021 03:16:12 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 115BGCij022371 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 5 Feb 2021 12:16:12 +0100 Received: from [167.87.72.79] ([167.87.72.79]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 115BGBbn000903; Fri, 5 Feb 2021 12:16:11 +0100 Subject: Re: [PATCH v7 1/2] meta/classes: Generate ova image for vmware or virtualbox To: "Q. Gylstorff" , isar-users@googlegroups.com References: <20200423094636.24511-2-Quirin.Gylstorff@siemens.com> <20210205105235.29169-1-Quirin.Gylstorff@siemens.com> <20210205105235.29169-2-Quirin.Gylstorff@siemens.com> From: Jan Kiszka Message-ID: <8ec5bfd3-ab8a-69c5-adb5-f28730d895a5@siemens.com> Date: Fri, 5 Feb 2021 12:16:11 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <20210205105235.29169-2-Quirin.Gylstorff@siemens.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: co/s/MNOp1lO On 05.02.21 11:52, Q. Gylstorff wrote: > From: Quirin Gylstorff > > This allows to generate a ova file for virtualbox or vmware. The > images differ in the setting of the variable `VMDK_SUBFORMAT`. > - `streamOptimized` is used for Vmware Workstation > - `monolithicSparse` is used for Virtualbox > > This is necessary as virtualbox throws an import error for a `streamOptimized` > version. The ova for Virtualbox is also bigger due the disk format. > > The default machine settings are: > - 4 CPU Cores with 8GB RAM > > Signed-off-by: Quirin Gylstorff > --- > doc/user_manual.md | 1 + > meta-isar/conf/machine/virtualbox.conf | 16 ++ > meta-isar/conf/machine/vmware.conf | 16 ++ > .../multiconfig/virtualbox-ova-buster.conf | 8 + > meta/classes/virtual-machine-image.bbclass | 92 +++++++++++ > .../files/virtual-machine-template.ovf.tmpl | 155 ++++++++++++++++++ > .../virtual-machine-template_0.1.bb | 17 ++ > scripts/ci_build.sh | 3 +- > 8 files changed, 307 insertions(+), 1 deletion(-) > create mode 100644 meta-isar/conf/machine/virtualbox.conf > create mode 100644 meta-isar/conf/machine/vmware.conf > create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf > create mode 100644 meta/classes/virtual-machine-image.bbclass > create mode 100644 meta/recipes-devtools/virtual-machine-template/files/virtual-machine-template.ovf.tmpl > create mode 100644 meta/recipes-devtools/virtual-machine-template/virtual-machine-template_0.1.bb > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index a4f3d1d..8b8ac7c 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -367,6 +367,7 @@ Isar can generate various images types for specific machine. The type of the ima > - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image (default option for the `rpi` machine). > - `wic-img` - A full disk image with user-specified partitions created and populated using the wic tool. > - `ubi-img` - A image for use on mtd nand partitions employing UBI > + - `virtual-machine-image` - A image for use on VirtualBox or VMware > > --- > > diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf > new file mode 100644 > index 0000000..9b823ff > --- /dev/null > +++ b/meta-isar/conf/machine/virtualbox.conf > @@ -0,0 +1,16 @@ > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > +DISTRO_ARCH ?= "amd64" > + > +KERNEL_NAME ?= "amd64" > +BOOTLOADER ?= "grub" > + > +WKS_FILE ?= "sdimage-efi" > + > +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" > + > +VMDK_SUBFORMAT = "monolithicSparse" > +IMAGE_TYPE ?= "virtual-machine-image" > diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf > new file mode 100644 > index 0000000..b6b4cc7 > --- /dev/null > +++ b/meta-isar/conf/machine/vmware.conf > @@ -0,0 +1,16 @@ > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > +DISTRO_ARCH ?= "amd64" > + > +KERNEL_NAME ?= "amd64" > +BOOTLOADER ?= "grub" > + > +WKS_FILE ?= "sdimage-efi" > + > +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" > + > +VMDK_SUBFORMAT = "streamOptimized" > +IMAGE_TYPE ?= "virtual-machine-image" > diff --git a/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf > new file mode 100644 > index 0000000..3042556 > --- /dev/null > +++ b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf > @@ -0,0 +1,8 @@ > +# > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > + > +MACHINE = "virtualbox" > +DISTRO = "debian-buster" > diff --git a/meta/classes/virtual-machine-image.bbclass b/meta/classes/virtual-machine-image.bbclass > new file mode 100644 > index 0000000..10e86a1 > --- /dev/null > +++ b/meta/classes/virtual-machine-image.bbclass > @@ -0,0 +1,92 @@ > +# This software is a part of ISAR. > +# Copyright (C) 2019-2020 Siemens AG > +# > +# This class allows to generate images for vmware and virtualbox > +# > + > +inherit buildchroot > +inherit wic-img Blank line > +IMAGER_BUILD_DEPS += "virtual-machine-template" > +IMAGER_INSTALL += "qemu-utils gawk uuid-runtime virtual-machine-template" Blank line > +export OVA_NAME ?= "${IMAGE_FULLNAME}" > +export OVA_MEMORY ?= "8192" > +export OVA_NUMBER_OF_CPU ?= "4" > +export OVA_VRAM ?= "64" > +export OVA_FIRMWARE ?= "efi" > +export OVA_ACPI ?= "true" > +export OVA_3D_ACCEL ?= "false" > +export OVA_CLIPBOARD ?= "bidirectional" > +SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img" > +OVA_SHA_ALG ?= "1" > +VIRTUAL_MACHINE_IMAGE_TYPE ?= "vmdk" > +export VIRTUAL_MACHINE_IMAGE_FILE ?= "${IMAGE_FULLNAME}-disk001.${VIRTUAL_MACHINE_IMAGE_TYPE}" > +VIRTUAL_MACHINE_DISK ?= "${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}" Please more structure (blank-line separation e.g.) if those are of different types. Are all of them user-tunable vars? > +# for virtualbox this needs to be monolithicSparse > +# for vmware this needs to be streamOptimized > +#VMDK_SUBFORMAT ?= "streamOptimized" Likely "export VMDK..." then. Why do we need all those exports? > +export VMDK_SUBFORMAT ?= "monolithicSparse" Blank line > +def set_convert_options(d): > + format = d.getVar("VIRTUAL_MACHINE_IMAGE_TYPE") > + if format == "vmdk": > + return "-o subformat=%s" % d.getVar("VMDK_SUBFORMAT") > + else: > + return "" > + > + > +CONVERSION_OPTIONS = "${@set_convert_options(d)}" > + > +do_convert_wic() { > + rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}' > + image_do_mounts > + bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${WIC_IMAGE_FILE}" > + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > + /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \ > + '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}' > +} > + > +addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template > + > +# Generate random MAC addresses just as VirtualBox does, the format is > +# their assigned prefix for the first 3 bytes followed by 3 random bytes. > +VBOX_MAC_PREFIX = "080027" > +macgen() { > + hexdump -n3 -e "\"${VBOX_MAC_PREFIX}%06X\n\"" /dev/urandom > +} Blank line > +get_disksize() { > + image_do_mounts > + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ > + qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" | gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}' > +} Blank line > +do_create_ova() { > + if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then > + exit 0 > + fi > + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ova' > + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ovf' > + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.mf' > + > + export PRIMARY_MAC=$(macgen) > + export SECONDARY_MAC=$(macgen) > + export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK}) > + export DISK_SIZE_BYTES=$(get_disksize) > + export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ") > + export OVA_FIRMWARE_VIRTUALBOX=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]') > + image_do_mounts > + sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO' > + export DISK_UUID=$(uuidgen) > + export VM_UUID=$(uuidgen) > + # create ovf > + cat /usr/share/virtual-machine-template/virtual-machine-template.ovf.tmpl | envsubst > ${PP_DEPLOY}/${OVA_NAME}.ovf > + tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.ovf > + > + # virtual box needs here a manifest file vmware does not want to accept the format > + if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" ]; then > + echo "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf > + echo "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf > + tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.mf > + fi > + tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE} > +EOSUDO > +} > + > +addtask do_create_ova after do_convert_wic before do_deploy > diff --git a/meta/recipes-devtools/virtual-machine-template/files/virtual-machine-template.ovf.tmpl b/meta/recipes-devtools/virtual-machine-template/files/virtual-machine-template.ovf.tmpl > new file mode 100644 > index 0000000..e6b5305 > --- /dev/null > +++ b/meta/recipes-devtools/virtual-machine-template/files/virtual-machine-template.ovf.tmpl > @@ -0,0 +1,155 @@ > + > + > + > + > + > + > + List of the virtual disks used in the package > + > + > + > + Logical networks used in the package > + > + Logical network used by this appliance. > + > + > + > + A virtual machine > + > + The kind of installed guest operating system > + Debian_64 > + Debian_64 > + > + > + Virtual hardware requirements for a virtual machine > + > + Virtual Hardware Family > + 0 > + ${OVA_NAME} > + virtualbox-2.2 > + > + > + hertz * 10^6 > + ${OVA_NUMBER_OF_CPU} virtual CPU > + Number of virtual CPUs > + ${OVA_NUMBER_OF_CPU} virtual CPU > + 1 > + 3 > + ${OVA_NUMBER_OF_CPU} > + > + > + MegaBytes > + ${OVA_MEMORY} MB of memory > + Memory Size > + ${OVA_MEMORY} MB of memory > + 2 > + 4 > + ${OVA_MEMORY} > + > + > + 0 > + ideController0 > + IDE Controller > + ideController0 > + 3 > + PIIX4 > + 5 > + > + > + 1 > + ideController1 > + IDE Controller > + ideController1 > + 4 > + PIIX4 > + 5 > + > + > + true > + Ethernet adapter on 'NAT' > + NAT > + Ethernet adapter on 'NAT' > + 5 > + E1000 > + 10 > + > + > + 0 > + disk1 > + Disk Image > + disk1 > + /disk/vmdisk1 > + 6 > + 3 > + 17 > + > + > + > + > + > + > + > + > + > + > + Complete VirtualBox machine configuration in VirtualBox format > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > diff --git a/meta/recipes-devtools/virtual-machine-template/virtual-machine-template_0.1.bb b/meta/recipes-devtools/virtual-machine-template/virtual-machine-template_0.1.bb > new file mode 100644 > index 0000000..dc67355 > --- /dev/null > +++ b/meta/recipes-devtools/virtual-machine-template/virtual-machine-template_0.1.bb > @@ -0,0 +1,17 @@ > +# This software is a part of ISAR. > +# > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > +inherit dpkg-raw > + > + > +SRC_URI += "file://virtual-machine-template.ovf.tmpl" > + > +do_install() { > + TARGET=${D}/usr/share/virtual-machine-template > + install -m 0755 -d ${TARGET} > + install -m 0740 ${WORKDIR}/virtual-machine-template.ovf.tmpl \ > + ${TARGET}/virtual-machine-template.ovf.tmpl > +} > diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh > index a8d9ad9..005bda5 100755 > --- a/scripts/ci_build.sh > +++ b/scripts/ci_build.sh > @@ -32,7 +32,8 @@ TARGETS_SET="\ > mc:qemumipsel-buster:isar-image-base \ > mc:nand-ubi-demo-buster:isar-image-ubi \ > mc:rpi-stretch:isar-image-base \ > - mc:qemuamd64-focal:isar-image-base" > + mc:qemuamd64-focal:isar-image-base \ > + mc:virtualbox-ova-buster:isar-image-base" > # qemu-user-static of <= buster too old to build that > # mc:qemuarm64-buster:isar-image-base > # mc:qemuarm64-bullseye:isar-image-base > -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux