From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6952975998838112256 X-Received: by 2002:a7b:c157:: with SMTP id z23mr929132wmi.146.1618865877705; Mon, 19 Apr 2021 13:57:57 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:24d4:: with SMTP id k203ls288334wmk.0.canary-gmail; Mon, 19 Apr 2021 13:57:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz+ypKJPy3MMZNen//xWE87I9Fd4I7sM0I/mGvglsg3FC55o49LSh9gNE5E52/2oGuxdxB+ X-Received: by 2002:a1c:e407:: with SMTP id b7mr876236wmh.75.1618865876629; Mon, 19 Apr 2021 13:57:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618865876; cv=none; d=google.com; s=arc-20160816; b=ETZlH2gJp7vl5QRADb7xjlgaQuyFT4vGXfKrcp6SuBDcCZaGZx0c4IDBhp/ye7S3Zi k5wWLTGlnnhQzDqpUKAXm4ZvTPpNXLGNjVvjGTzSLjSuOWwNulKT+u41AnsGRViQnBB5 9cePNPcNxVyonqPCIDJc0xAqXLlCF8ANravhwu403loh4wUV4+NXuUcUkC37IucddXcf da8Wp08SvQrdxdvtwTX7KT2b6sj/Bor1x4JPLlV/q/a/w6QXXOn9ZSORwKp27mM30l74 b2jmVn0kkkU9c+iu2hPuo7sKM+NLB6+5HL2Y4VRzYSGYbFdSlTjKRis5GsayEgwUaetI Dk7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=J2ukh0+juRzRVDbJMeZnek+SbTvGpB/rkR7h98OdY0Q=; b=htrbOIMy4dZjwcOqvdoLbcTq0A4rj4IL+6HwhYFdBPCLL3bHu6W1r4QVUfpRWpytUX Cw9d2G6p8mSJ0K3dzyEUQATGdhbUU5F4DZ3JiQoyt/TGL2ss4RVVzjrZ3xZM2rHvCBoc XlrbfCIJkqrBKBD7XsxRuDfGhtKLycfOE6gvXLYA3H6MslEGOtMmK1WJtNof8OgZRMr0 eTqVoPGZ7PbeRpo0eS2n5+UN/zYxEu1ZbF5OI+zM+MmE5o97g6vu3WHy3D7A3dLRDBtk 8ViM2XSSWZk6H6MHKlCjjCCodAGk+VS31MVCrO8baGmcA1QBW9Uh1lfe2u+qN90/rqaM g49g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id 5si991679wrj.4.2021.04.19.13.57.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Apr 2021 13:57:56 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 13JKvu0d013484 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 19 Apr 2021 22:57:56 +0200 Received: from md1f2u6c.ad001.siemens.net ([167.87.240.116]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 13JKoaBN032525; Mon, 19 Apr 2021 22:50:37 +0200 From: Jan Kiszka To: isar-users Cc: Quirin Gylstorff Subject: [PATCH v9 1/2] meta/classes: Generate ova image for VMWare or Virtualbox Date: Mon, 19 Apr 2021 22:50:35 +0200 Message-Id: <8d45f0720fe9d377639015ebfae6759883978002.1618865436.git.jan.kiszka@siemens.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: 6854UtuDLxLf From: Quirin Gylstorff This allows to generate a ova file for virtualbox or vmware. The images differ in the setting of the variable `VMDK_SUBFORMAT`. - `streamOptimized` is used for Vmware Workstation - `monolithicSparse` is used for Virtualbox This is necessary as virtualbox throws an import error for a `streamOptimized` version. The ova for Virtualbox is also bigger due the disk format. The default machine settings are: - 4 CPU Cores with 8GB Ram Signed-off-by: Quirin Gylstorff [Jan: shortened named, reformatting, massaged comments] Signed-off-by: Jan Kiszka --- doc/user_manual.md | 1 + meta-isar/conf/machine/virtualbox.conf | 16 ++ meta-isar/conf/machine/vmware.conf | 16 ++ .../multiconfig/virtualbox-ova-buster.conf | 8 + meta/classes/vm-img.bbclass | 104 ++++++++++++ .../vm-template/files/vm-template.ovf.tmpl | 155 ++++++++++++++++++ .../vm-template/vm-template_0.1.bb | 16 ++ scripts/ci_build.sh | 1 + 8 files changed, 317 insertions(+) create mode 100644 meta-isar/conf/machine/virtualbox.conf create mode 100644 meta-isar/conf/machine/vmware.conf create mode 100644 meta-isar/conf/multiconfig/virtualbox-ova-buster.conf create mode 100644 meta/classes/vm-img.bbclass create mode 100644 meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl create mode 100644 meta/recipes-devtools/vm-template/vm-template_0.1.bb diff --git a/doc/user_manual.md b/doc/user_manual.md index fec9896..2749ef0 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -454,6 +454,7 @@ Isar can generate various images types for specific machine. The type of the ima - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image (default option for the `rpi` machine). - `wic-img` - A full disk image with user-specified partitions created and populated using the wic tool. - `ubi-img` - A image for use on mtd nand partitions employing UBI + - `vm-img` - A image for use on VirtualBox or VMware --- diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf new file mode 100644 index 0000000..8f33ae4 --- /dev/null +++ b/meta-isar/conf/machine/virtualbox.conf @@ -0,0 +1,16 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2020 +# +# SPDX-License-Identifier: MIT + +DISTRO_ARCH ?= "amd64" + +KERNEL_NAME ?= "amd64" +BOOTLOADER ?= "grub" + +WKS_FILE ?= "sdimage-efi" + +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" + +VMDK_SUBFORMAT = "monolithicSparse" +IMAGE_TYPE ?= "vm-img" diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf new file mode 100644 index 0000000..02a349a --- /dev/null +++ b/meta-isar/conf/machine/vmware.conf @@ -0,0 +1,16 @@ +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2020 +# +# SPDX-License-Identifier: MIT + +DISTRO_ARCH ?= "amd64" + +KERNEL_NAME ?= "amd64" +BOOTLOADER ?= "grub" + +WKS_FILE ?= "sdimage-efi" + +IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" + +VMDK_SUBFORMAT = "streamOptimized" +IMAGE_TYPE ?= "vm-img" diff --git a/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf new file mode 100644 index 0000000..3042556 --- /dev/null +++ b/meta-isar/conf/multiconfig/virtualbox-ova-buster.conf @@ -0,0 +1,8 @@ +# +# Copyright (c) Siemens AG, 2020 +# +# SPDX-License-Identifier: MIT + + +MACHINE = "virtualbox" +DISTRO = "debian-buster" diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass new file mode 100644 index 0000000..8a6b2da --- /dev/null +++ b/meta/classes/vm-img.bbclass @@ -0,0 +1,104 @@ +# This software is a part of ISAR. +# Copyright (C) 2019-2020 Siemens AG +# +# This class allows to generate images for VMware and VirtualBox +# + +inherit buildchroot +inherit wic-img + +IMAGER_BUILD_DEPS += "vm-template" +IMAGER_INSTALL += "qemu-utils gawk uuid-runtime vm-template" + +# virtual machine disk settings +SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img" + +# For VirtualBox, this needs to be "monolithicSparse" (default to it). +# VMware needs this to be "streamOptimized". +VMDK_SUBFORMAT ?= "monolithicSparse" + +VIRTUAL_MACHINE_IMAGE_TYPE ?= "vmdk" +VIRTUAL_MACHINE_IMAGE_FILE = "${IMAGE_FULLNAME}-disk001.${VIRTUAL_MACHINE_IMAGE_TYPE}" +VIRTUAL_MACHINE_DISK = "${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}" + +def set_convert_options(d): + format = d.getVar("VIRTUAL_MACHINE_IMAGE_TYPE") + if format == "vmdk": + return "-o subformat=%s" % d.getVar("VMDK_SUBFORMAT") + else: + return "" + + +CONVERSION_OPTIONS = "${@set_convert_options(d)}" + +do_convert_wic() { + rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}' + image_do_mounts + bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${WIC_IMAGE_FILE}" + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \ + '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE}' +} + +addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template + +# User settings for OVA +OVA_NAME ?= "${IMAGE_FULLNAME}" +OVA_MEMORY ?= "8192" +OVA_NUMBER_OF_CPU ?= "4" +OVA_VRAM ?= "64" +OVA_FIRMWARE ?= "efi" +OVA_ACPI ?= "true" +OVA_3D_ACCEL ?= "false" +OVA_CLIPBOARD ?= "bidirectional" +OVA_SHA_ALG = "1" + +# Generate random MAC addresses just as VirtualBox does, the format is +# their assigned prefix for the first 3 bytes followed by 3 random bytes. +VBOX_MAC_PREFIX = "080027" + +macgen() { + hexdump -n3 -e "\"${VBOX_MAC_PREFIX}%06X\n\"" /dev/urandom +} + +get_disksize() { + image_do_mounts + sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" | gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}' +} + +do_create_ova() { + if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then + exit 0 + fi + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ova' + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.ovf' + rm -f '${DEPLOY_DIR_IMAGE}/${OVA_NAME}.mf' + + export PRIMARY_MAC=$(macgen) + export SECONDARY_MAC=$(macgen) + export DISK_NAME=$(basename -s .vmdk ${VIRTUAL_MACHINE_DISK}) + export DISK_SIZE_BYTES=$(get_disksize) + export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ") + export OVA_FIRMWARE_VIRTUALBOX=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]') + + image_do_mounts + + sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO' + export DISK_UUID=$(uuidgen) + export VM_UUID=$(uuidgen) + # create ovf + cat /usr/share/vm-template/vm-template.ovf.tmpl | envsubst > ${PP_DEPLOY}/${OVA_NAME}.ovf + tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.ovf + + # VirtualBox needs here a manifest file. VMware does accept that format. + if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" ]; then + echo "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf + echo "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> ${PP_DEPLOY}/${OVA_NAME}.mf + tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.mf + fi + tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE} +EOSUDO +} + +addtask do_create_ova after do_convert_wic before do_deploy diff --git a/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl new file mode 100644 index 0000000..e6b5305 --- /dev/null +++ b/meta/recipes-devtools/vm-template/files/vm-template.ovf.tmpl @@ -0,0 +1,155 @@ + + + + + + + List of the virtual disks used in the package + + + + Logical networks used in the package + + Logical network used by this appliance. + + + + A virtual machine + + The kind of installed guest operating system + Debian_64 + Debian_64 + + + Virtual hardware requirements for a virtual machine + + Virtual Hardware Family + 0 + ${OVA_NAME} + virtualbox-2.2 + + + hertz * 10^6 + ${OVA_NUMBER_OF_CPU} virtual CPU + Number of virtual CPUs + ${OVA_NUMBER_OF_CPU} virtual CPU + 1 + 3 + ${OVA_NUMBER_OF_CPU} + + + MegaBytes + ${OVA_MEMORY} MB of memory + Memory Size + ${OVA_MEMORY} MB of memory + 2 + 4 + ${OVA_MEMORY} + + + 0 + ideController0 + IDE Controller + ideController0 + 3 + PIIX4 + 5 + + + 1 + ideController1 + IDE Controller + ideController1 + 4 + PIIX4 + 5 + + + true + Ethernet adapter on 'NAT' + NAT + Ethernet adapter on 'NAT' + 5 + E1000 + 10 + + + 0 + disk1 + Disk Image + disk1 + /disk/vmdisk1 + 6 + 3 + 17 + + + + + + + + + + + Complete VirtualBox machine configuration in VirtualBox format + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/meta/recipes-devtools/vm-template/vm-template_0.1.bb b/meta/recipes-devtools/vm-template/vm-template_0.1.bb new file mode 100644 index 0000000..ccb1280 --- /dev/null +++ b/meta/recipes-devtools/vm-template/vm-template_0.1.bb @@ -0,0 +1,16 @@ +# This software is a part of ISAR. +# +# Copyright (c) Siemens AG, 2020 +# +# SPDX-License-Identifier: MIT + +inherit dpkg-raw + +SRC_URI += "file://virtual-machine-template.ovf.tmpl" + +do_install() { + TARGET=${D}/usr/share/virtual-machine-template + install -m 0755 -d ${TARGET} + install -m 0740 ${WORKDIR}/virtual-machine-template.ovf.tmpl \ + ${TARGET}/virtual-machine-template.ovf.tmpl +} diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index adc22e4..a77cf5a 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -43,6 +43,7 @@ TARGETS_SET="\ mc:nand-ubi-demo-buster:isar-image-ubi \ mc:rpi-stretch:isar-image-base \ mc:qemuamd64-focal:isar-image-base \ + mc:virtualbox-ova-buster:isar-image-base \ " # qemu-user-static of <= buster too old to build that # mc:qemuarm64-buster:isar-image-base -- 2.26.2