* [PATCH v2 1/7] image: remove IMAGE_SUFFIX
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 2/7] image: refactor SDK Adriaan Schmidt
` (5 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
It is no longer required, and removing it enables sharing of rootfs
data between image types.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta/classes/image.bbclass | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index eb879ffe..8899d1e9 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -1,15 +1,13 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2017 ilbers GmbH
-# Replace possible multiple spaces with single underscores
-IMAGE_SUFFIX = "${@'_'.join(d.getVar("IMAGE_FSTYPES", True).split())}"
# Make workdir and stamps machine-specific without changing common PN target
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_SUFFIX}/${PV}-${PR}"
-STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_SUFFIX}/${PV}-${PR}"
-STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}-${IMAGE_SUFFIX}/*-*"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
+STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/${PV}-${PR}"
+STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
# Sstate also needs to be machine-specific
-SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}-${IMAGE_SUFFIX}"
+SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
IMAGE_INSTALL ?= ""
IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4-img"}"
@@ -29,7 +27,7 @@ INITRD_IMAGE ?= "${IMAGE_FULLNAME}-initrd.img"
DTB_FILES ?= ""
# Useful variables for imager implementations:
-PP = "/home/builder/${PN}-${MACHINE}-${IMAGE_SUFFIX}"
+PP = "/home/builder/${PN}-${MACHINE}"
PP_DEPLOY = "${PP}/deploy"
PP_ROOTFS = "${PP}/rootfs"
PP_WORK = "${PP}/work"
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 2/7] image: refactor SDK
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 1/7] image: remove IMAGE_SUFFIX Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 3/7] meta: introduce IMAGE_CMD_* Adriaan Schmidt
` (4 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
- turn the image-sdk-extension into an image recipe variant (requires
renaming of the class to `sdk` to use BBCLASSEXTEND).
- put the logic from sdkchroot into that variant
- turn the remainder of sdkcroot into the package sdk-files that
that is then installed into SDKs
API change: SDK_FORMATS now has the same semantics (and knows the same
formats) as IMAGE_FSTYPES. Because there is no image type to generate
tar.xz images, for now the default SDK_FORMAT is `targz-img`.
NOTE: this patch does not work stand-alone. It requires changes to the
way image types are handled.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta/classes/image-sdk-extension.bbclass | 87 -----------
meta/classes/image.bbclass | 2 +-
meta/classes/sdk.bbclass | 137 ++++++++++++++++++
.../{sdkchroot => sdk-files}/files/README.sdk | 0
.../files/configscript.sh | 0
.../files/gcc-sysroot-wrapper.sh | 0
.../files/relocate-sdk.sh | 0
meta/recipes-devtools/sdk-files/sdk-files.bb | 26 ++++
meta/recipes-devtools/sdkchroot/sdkchroot.bb | 78 ----------
9 files changed, 164 insertions(+), 166 deletions(-)
delete mode 100644 meta/classes/image-sdk-extension.bbclass
create mode 100644 meta/classes/sdk.bbclass
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/README.sdk (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/configscript.sh (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/gcc-sysroot-wrapper.sh (100%)
rename meta/recipes-devtools/{sdkchroot => sdk-files}/files/relocate-sdk.sh (100%)
create mode 100644 meta/recipes-devtools/sdk-files/sdk-files.bb
delete mode 100644 meta/recipes-devtools/sdkchroot/sdkchroot.bb
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
deleted file mode 100644
index 052c1b58..00000000
--- a/meta/classes/image-sdk-extension.bbclass
+++ /dev/null
@@ -1,87 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-#
-# This class extends the image.bbclass to supply the creation of a sdk
-
-SDK_INCLUDE_ISAR_APT ?= "0"
-SDK_FORMATS ?= "tar-xz"
-
-sdk_tar_xz() {
- # Copy mount_chroot.sh for convenience
- sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
-
- # Create SDK archive
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
- bbdebug 1 "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz"
-}
-
-do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-do_populate_sdk[depends] = "sdkchroot:do_build"
-do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS"
-do_populate_sdk() {
- local sdk_container_formats=""
-
- if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then
- # Copy isar-apt with deployed Isar packages
- sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt
- else
- # Remove isar-apt repo entry
- sudo rm -f ${SDKCHROOT_DIR}/etc/apt/sources.list.d/isar-apt.list
- fi
-
- sudo umount -R ${SDKCHROOT_DIR}/dev || true
- sudo umount ${SDKCHROOT_DIR}/proc || true
- sudo umount -R ${SDKCHROOT_DIR}/sys || true
-
- # Remove setup scripts
- sudo rm -f ${SDKCHROOT_DIR}/chroot-setup.sh ${SDKCHROOT_DIR}/configscript.sh
-
- # Make all links relative
- for link in $(find ${SDKCHROOT_DIR}/ -type l); do
- target=$(readlink $link)
-
- if [ "${target#/}" != "${target}" ]; then
- basedir=$(dirname $link)
- new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${SDKCHROOT_DIR}/${target})
-
- # remove first to allow rewriting directory links
- sudo rm $link
- sudo ln -s $new_target $link
- fi
- done
-
- # Set up sysroot wrapper
- for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
- for tool in $(find ${SDKCHROOT_DIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
- sudo mv "${tool}" "${tool}.bin"
- sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
- done
- done
-
- # separate SDK formats: TAR and container formats
- for sdk_format in ${SDK_FORMATS} ; do
- case ${sdk_format} in
- "tar-xz")
- sdk_tar_xz
- ;;
- "docker-archive" | "oci" | "oci-archive" | "docker-daemon" | "containers-storage")
- sdk_container_formats="${sdk_container_formats} ${sdk_format}"
- ;;
- *)
- die "unsupported SDK format specified: ${sdk_format}"
- ;;
- esac
- done
-
- # generate the SDK in all the desired container formats
- if [ -n "${sdk_container_formats}" ] ; then
- bbnote "Generating SDK container in ${sdk_container_formats} format"
- containerize_rootfs "${SDKCHROOT_DIR}" "${sdk_container_formats}" "sdk-"
- fi
-}
-
-addtask populate_sdk after do_rootfs
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8899d1e9..f87b76e7 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -78,7 +78,7 @@ ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${MACHINE}"
ROOTFS_POSTPROCESS_COMMAND_prepend = "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache-deb-src', 'cache_deb_src', '', d)} "
inherit rootfs
-inherit image-sdk-extension
+inherit sdk
inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
new file mode 100644
index 00000000..adf9a1fe
--- /dev/null
+++ b/meta/classes/sdk.bbclass
@@ -0,0 +1,137 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass to supply the creation of a sdk
+
+# hook up the -sdk image variant
+BBCLASSEXTEND = "sdk"
+BPN = "${PN}"
+
+python sdk_virtclass_handler() {
+ pn = e.data.getVar('PN')
+ if pn.endswith('-sdk'):
+ e.data.setVar('BPN', pn[:-len('-sdk')])
+ e.data.appendVar('OVERRIDES', ':class-sdk')
+ # sdkchroot deploy only for sdk image
+ bb.build.addtask('deploy_sdkchroot', 'do_build', 'do_rootfs', d)
+ else:
+ # add do_populate_sdk only to the non-sdk variant
+ # it only exists to preserve the interface...
+ bb.build.addtask('populate_sdk', '', '', e.data)
+ e.data.appendVarFlag('do_populate_sdk', 'depends', '${BPN}-sdk:do_build')
+ e.data.appendVarFlag('do_clean', 'depends', '${BPN}-sdk:do_clean')
+}
+addhandler sdk_virtclass_handler
+sdk_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
+
+# SDK is image-specific:
+SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${MACHINE}"
+
+# SDK settings
+SDK_INCLUDE_ISAR_APT ?= "0"
+SDK_FORMATS ?= "targz-img"
+SDK_INSTALL ?= ""
+SDK_PREINSTALL += " \
+ debhelper \
+ autotools-dev \
+ dpkg \
+ locales \
+ docbook-to-man \
+ apt \
+ automake \
+ devscripts \
+ equivs"
+
+TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
+TOOLCHAIN_${HOST_ARCH} = "build-essential"
+TOOLCHAIN_i386 = "build-essential"
+TOOLCHAIN_append_compat-arch = " crossbuild-essential-${COMPAT_DISTRO_ARCH}"
+
+# rootfs/image overrides for the SDK
+ROOTFS_ARCH_class-sdk = "${HOST_ARCH}"
+ROOTFS_DISTRO_class-sdk = "${HOST_DISTRO}"
+ROOTFS_PACKAGES_class-sdk = "sdk-files ${TOOLCHAIN} ${SDK_PREINSTALL} ${SDK_INSTALL}"
+ROOTFS_FEATURES_append_class-sdk = " clean-package-cache generate-manifest export-dpkg-status"
+ROOTFS_MANIFEST_DEPLOY_DIR_class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
+ROOTFS_DPKGSTATUS_DEPLOY_DIR_class-sdk = "${DEPLOY_DIR_SDKCHROOT}"
+
+IMAGE_FSTYPES_class-sdk = "${SDK_FORMATS}"
+
+# bitbake dependencies
+SDKDEPENDS += "sdk-files ${SDK_INSTALL}"
+SDKDEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
+DEPENDS_class-sdk = "${SDKDEPENDS}"
+
+SDKROOTFSDEPENDS = ""
+SDKROOTFSDEPENDS_class-sdk = "${BPN}:do_rootfs"
+do_rootfs_install[depends] += "${SDKROOTFSDEPENDS}"
+
+SDKROOTFSVARDEPS = ""
+SDKROOTFSVARDEPS_class-sdk = "SDK_INCLUDE_ISAR_APT"
+do_rootfs_install[vardeps] += "${SDKROOTFSVARDEPS}"
+
+# additional SDK steps
+ROOTFS_CONFIGURE_COMMAND_append_class-sdk = " ${@'rootfs_configure_isar_apt_dir' if d.getVar('SDK_INCLUDE_ISAR_APT') == '1' else ''}"
+rootfs_configure_isar_apt_dir() {
+ # Copy isar-apt instead of mounting:
+ sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
+}
+
+ROOTFS_POSTPROCESS_COMMAND_prepend_class-sdk = "sdkchroot_configscript "
+sdkchroot_configscript () {
+ sudo chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+}
+
+ROOTFS_POSTPROCESS_COMMAND_append_class-sdk = " sdkchroot_finalize"
+sdkchroot_finalize() {
+ if [ "${SDK_INCLUDE_ISAR_APT}" = "0" ]; then
+ # Remove isar-apt repo entry
+ sudo rm -f ${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list
+ fi
+
+ sudo umount -R ${ROOTFSDIR}/dev || true
+ sudo umount ${ROOTFSDIR}/proc || true
+ sudo umount -R ${ROOTFSDIR}/sys || true
+
+ # Remove setup scripts
+ sudo rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
+
+ # Make all links relative
+ for link in $(find ${ROOTFSDIR}/ -type l); do
+ target=$(readlink $link)
+
+ if [ "${target#/}" != "${target}" ]; then
+ basedir=$(dirname $link)
+ new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${ROOTFSDIR}/${target})
+
+ # remove first to allow rewriting directory links
+ sudo rm $link
+ sudo ln -s $new_target $link
+ fi
+ done
+
+ # Set up sysroot wrapper
+ for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
+ for tool in $(find ${ROOTFSDIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
+ sudo mv "${tool}" "${tool}.bin"
+ sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+ done
+ done
+}
+
+do_deploy_sdkchroot[dirs] = "${DEPLOY_DIR_SDKCHROOT}"
+do_deploy_sdkchroot() {
+ ln -Tfsr "${ROOTFSDIR}" "${SDKCHROOT_DIR}"
+}
+
+CLEANFUNCS_class-sdk = "clean_deploy"
+clean_deploy() {
+ rm -f "${SDKCHROOT_DIR}"
+}
+
+do_populate_sdk[noexec] = "1"
+do_populate_sdk() {
+ :
+}
diff --git a/meta/recipes-devtools/sdkchroot/files/README.sdk b/meta/recipes-devtools/sdk-files/files/README.sdk
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/README.sdk
rename to meta/recipes-devtools/sdk-files/files/README.sdk
diff --git a/meta/recipes-devtools/sdkchroot/files/configscript.sh b/meta/recipes-devtools/sdk-files/files/configscript.sh
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/configscript.sh
rename to meta/recipes-devtools/sdk-files/files/configscript.sh
diff --git a/meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh b/meta/recipes-devtools/sdk-files/files/gcc-sysroot-wrapper.sh
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/gcc-sysroot-wrapper.sh
rename to meta/recipes-devtools/sdk-files/files/gcc-sysroot-wrapper.sh
diff --git a/meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh b/meta/recipes-devtools/sdk-files/files/relocate-sdk.sh
similarity index 100%
rename from meta/recipes-devtools/sdkchroot/files/relocate-sdk.sh
rename to meta/recipes-devtools/sdk-files/files/relocate-sdk.sh
diff --git a/meta/recipes-devtools/sdk-files/sdk-files.bb b/meta/recipes-devtools/sdk-files/sdk-files.bb
new file mode 100644
index 00000000..36cdb319
--- /dev/null
+++ b/meta/recipes-devtools/sdk-files/sdk-files.bb
@@ -0,0 +1,26 @@
+# SDK Root filesystem
+#
+# This software is a part of ISAR.
+# Copyright (C) 2015-2018 ilbers GmbH
+
+DESCRIPTION = "Isar SDK Root filesystem"
+
+LICENSE = "gpl-2.0"
+LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
+
+inherit dpkg-raw
+
+SRC_URI = " \
+ file://configscript.sh \
+ file://relocate-sdk.sh \
+ file://gcc-sysroot-wrapper.sh \
+ file://README.sdk"
+PV = "0.1"
+
+do_install() {
+ install -m 644 ${WORKDIR}/README.sdk ${D}
+ install -m 755 ${WORKDIR}/relocate-sdk.sh ${D}
+ install -m 755 -d ${D}/usr/bin
+ install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${D}/usr/bin
+ install -m 755 ${WORKDIR}/configscript.sh ${D}
+}
diff --git a/meta/recipes-devtools/sdkchroot/sdkchroot.bb b/meta/recipes-devtools/sdkchroot/sdkchroot.bb
deleted file mode 100644
index e367eae2..00000000
--- a/meta/recipes-devtools/sdkchroot/sdkchroot.bb
+++ /dev/null
@@ -1,78 +0,0 @@
-# SDK Root filesystem
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2018 ilbers GmbH
-
-DESCRIPTION = "Isar SDK Root filesystem"
-
-LICENSE = "gpl-2.0"
-LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
-
-SRC_URI = " \
- file://configscript.sh \
- file://relocate-sdk.sh \
- file://gcc-sysroot-wrapper.sh \
- file://README.sdk"
-PV = "0.1"
-
-SDK_INSTALL ?= ""
-
-DEPENDS += "${SDK_INSTALL}"
-
-DEPENDS_append_riscv64 = "${@' crossbuild-essential-riscv64' if d.getVar('ISAR_CROSS_COMPILE', True) == '1' and d.getVar('PN') != 'crossbuild-essential-riscv64' else ''}"
-
-TOOLCHAIN = "crossbuild-essential-${DISTRO_ARCH}"
-TOOLCHAIN_${HOST_ARCH} = "build-essential"
-TOOLCHAIN_i386 = "build-essential"
-TOOLCHAIN_append_compat-arch = " crossbuild-essential-${COMPAT_DISTRO_ARCH}"
-
-inherit rootfs
-ROOTFS_ARCH = "${HOST_ARCH}"
-ROOTFS_DISTRO = "${HOST_DISTRO}"
-ROOTFSDIR = "${S}"
-ROOTFS_PACKAGES = "${SDK_PREINSTALL} ${SDK_INSTALL} ${TOOLCHAIN}"
-ROOTFS_FEATURES += "clean-package-cache generate-manifest export-dpkg-status"
-ROOTFS_MANIFEST_DEPLOY_DIR = "${DEPLOY_DIR_SDKCHROOT}"
-ROOTFS_DPKGSTATUS_DEPLOY_DIR = "${DEPLOY_DIR_SDKCHROOT}"
-
-SDK_PREINSTALL += " \
- debhelper \
- autotools-dev \
- dpkg \
- locales \
- docbook-to-man \
- apt \
- automake \
- devscripts \
- equivs"
-
-SDK_INCLUDE_ISAR_APT ?= "0"
-
-S = "${WORKDIR}/rootfs"
-
-ROOTFS_CONFIGURE_COMMAND += "${@'rootfs_configure_isar_apt_dir' if d.getVar('SDK_INCLUDE_ISAR_APT') == '1' else ''}"
-rootfs_configure_isar_apt_dir() {
- # Copy isar-apt instead of mounting:
- sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
-}
-
-ROOTFS_POSTPROCESS_COMMAND =+ "sdkchroot_install_files"
-sdkchroot_install_files() {
- # Configure root filesystem
- sudo install -m 644 ${WORKDIR}/README.sdk ${S}
- sudo install -m 755 ${WORKDIR}/relocate-sdk.sh ${S}
- sudo install -m 755 ${WORKDIR}/gcc-sysroot-wrapper.sh ${S}/usr/bin
- sudo install -m 755 ${WORKDIR}/configscript.sh ${S}
- sudo chroot ${S} /configscript.sh ${DISTRO_ARCH}
-}
-
-do_sdkchroot_deploy[dirs] = "${DEPLOY_DIR_SDKCHROOT}"
-do_sdkchroot_deploy() {
- ln -Tfsr "${ROOTFSDIR}" "${SDKCHROOT_DIR}"
-}
-addtask sdkchroot_deploy before do_build after do_rootfs
-
-CLEANFUNCS = "clean_deploy"
-clean_deploy() {
- rm -f "${SDKCHROOT_DIR}"
-}
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 3/7] meta: introduce IMAGE_CMD_*
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 1/7] image: remove IMAGE_SUFFIX Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 2/7] image: refactor SDK Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
2022-04-14 17:46 ` Anton Mikanovich
2022-04-05 13:24 ` [PATCH v2 4/7] imagetypes: restructure files Adriaan Schmidt
` (3 subsequent siblings)
6 siblings, 1 reply; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
This makes a number of changes:
- Replace all do_*_image tasks with IMAGE_CMD_* definitions
- Remove all implicit conversions (gz, xz)
- Add IMAGE_CONVERSION_* commands
- Create tasks (do_image_*) on demand in image.bbclass for
all requested IMAGE_FSTYPES
- When creating the tasks:
- image_do_mounts is inserted automatically
- a final chown is inserted automatically
- variables IMAGE_FILE_HOST and IMAGE_FILE_CHROOT are
set to reference the image
- variable SUDO_CHROOT contains the chroot command
- Create conversions on demand based on IMAGE_FSTYPES.
A conversion is defined by CONVERSION_CMD_type, and its
dependencies given as CONVERSION_DEPS_type
- In conversion commands
- the input file is named ${IMAGE_FULLNAME}.${type}
- the conversions appends its own type, e.g. the output file
would be ${IMAGE_FULLNAME}.${type}.xz
- a final chown is appended automatically
- Image types now longer have a -img suffix, e.g., ext4 instead
of ext4-img, and conversions are appended like tar.gz instead
of targz-img
- Imager dependencies are set as IMAGER_INSTALL_type
- Dependencies between image types are modelled like
IMAGE_TYPEDEP_ova = "wic"
- Required arguments/variables are modelled by
IMAGE_CMD_REQUIRED_ARGUMENTS = "A B C"
- Container types (previously CONTAINER_IMAGE_FORMATS) are now
first class image types (oci, oci-archive, docker-archive,
docker-daemon, containers.storage)
- The logic of image-container-extension has moved to
container-img
- The VM image now has type ova (instead of vm-img)
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta/classes/container-img.bbclass | 94 ++++++++-
meta/classes/cpiogz-img.bbclass | 20 +-
meta/classes/ext4-img.bbclass | 21 +-
meta/classes/fit-img.bbclass | 17 +-
.../classes/image-container-extension.bbclass | 83 --------
meta/classes/image.bbclass | 184 +++++++++++++++++-
meta/classes/rootfs.bbclass | 1 +
meta/classes/sdk.bbclass | 2 +-
meta/classes/targz-img.bbclass | 11 +-
meta/classes/ubi-img.bbclass | 24 +--
meta/classes/ubifs-img.bbclass | 29 +--
meta/classes/vm-img.bbclass | 24 +--
meta/classes/wic-img.bbclass | 45 ++---
13 files changed, 331 insertions(+), 224 deletions(-)
delete mode 100644 meta/classes/image-container-extension.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
index 322889a9..436a0051 100644
--- a/meta/classes/container-img.bbclass
+++ b/meta/classes/container-img.bbclass
@@ -3,15 +3,93 @@
#
# SPDX-License-Identifier: MIT
#
-# This class provides the task 'container_image'
+# This class provides the task 'containerize'
# to create container images containing the target rootfs.
-do_container_image[dirs] = "${DEPLOY_DIR_IMAGE}"
-do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-do_container_image[vardeps] += "CONTAINER_IMAGE_FORMATS"
-do_container_image(){
- bbdebug 1 "Generate container image in these formats: ${CONTAINER_IMAGE_FORMATS}"
- containerize_rootfs "${IMAGE_ROOTFS}" "${CONTAINER_IMAGE_FORMATS}"
+CONTAINER_TYPES = "oci oci-archive docker-archive docker-daemon containers-storage"
+USING_CONTAINER = "${@bb.utils.contains_any('IMAGE_BASETYPES', d.getVar('CONTAINER_TYPES').split(), '1', '0', d)}"
+
+CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
+CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
+
+python() {
+ if not d.getVar('USING_CONTAINER') == '1':
+ return
+ for t in d.getVar('CONTAINER_TYPES').split():
+ t_clean = t.replace('-', '_').replace('.', '_')
+ d.setVar('IMAGE_CMD_' + t_clean, 'convert_container %s "${CONTAINER_IMAGE_NAME}" "${IMAGE_FILE_HOST}"' % t)
+ d.setVar('IMAGE_FULLNAME_' + t_clean, '${PN}-${DISTRO}-${DISTRO_ARCH}')
+ bb.build.addtask('containerize', 'do_image_' + t_clean, 'do_image_tools', d)
+}
+
+do_containerize() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local tag="${CONTAINER_IMAGE_TAG}"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="${IMAGE_ROOTFS}"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ # clean-up temporary files
+ sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
+
+ # pack container image
+ bbdebug 1 "pack container image"
+ sudo umoci repack --image "${oci_img_dir}:${tag}" \
+ "${oci_img_dir}_unpacked"
+ sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
+ sudo rm -rf "${oci_img_dir}_unpacked"
+
+ # no root needed anymore
+ sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
}
-addtask container_image before do_image after do_image_tools
+convert_container() {
+ local tag="${CONTAINER_IMAGE_TAG}"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local container_type="$1"
+ local image_name="$2"
+ local image_archive="$3"
+
+ # convert the OCI container image to the desired format
+ bbdebug 1 "Creating container image type: ${container_type}"
+ case "${container_type}" in
+ "docker-archive" | "oci-archive")
+ if [ "${container_type}" = "oci-archive" ] ; then
+ target="${container_type}:${image_archive}:${tag}"
+ else
+ target="${container_type}:${image_archive}:${image_name}:${tag}"
+ fi
+ rm -f "${image_archive}"
+ bbdebug 2 "Converting OCI image to ${container_type}"
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${tag}" "${target}"
+ ;;
+ "oci")
+ tar --create --directory "${oci_img_dir}" \
+ --file "${image_archive}" .
+ ;;
+ "docker-daemon" | "containers-storage")
+ if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
+ die "Adding the container image to a container runtime (${container_type}) not supported if running from a container (e.g. 'kas-container')"
+ fi
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${tag}" \
+ "${container_type}:${image_name}:${tag}"
+ ;;
+ *)
+ die "Unsupported format for convert_container: ${container_type}"
+ ;;
+ esac
+}
diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
index 2a49456b..f4c33bd9 100644
--- a/meta/classes/cpiogz-img.bbclass
+++ b/meta/classes/cpiogz-img.bbclass
@@ -3,20 +3,12 @@
#
# SPDX-License-Identifier: MIT
-CPIOGZ_FNAME ?= "${IMAGE_FULLNAME}.cpio.gz"
-CPIOGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${CPIOGZ_FNAME}"
-IMAGER_INSTALL += "cpio"
+IMAGER_INSTALL_cpio += "cpio"
CPIO_IMAGE_FORMAT ?= "newc"
-do_cpiogz_image() {
- sudo rm -f ${CPIOGZ_IMAGE_FILE}
- image_do_mounts
- sudo chroot ${BUILDCHROOT_DIR} \
- sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
- /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o | /usr/bin/gzip -9 > \
- ${PP_DEPLOY}/${CPIOGZ_FNAME}"
- sudo chown $(id -u):$(id -g) ${CPIOGZ_IMAGE_FILE}
+IMAGE_CMD_cpio() {
+ ${SUDO_CHROOT} \
+ sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
+ /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
+ ${IMAGE_FILE_CHROOT}"
}
-
-addtask cpiogz_image before do_image after do_image_tools
-do_cpiogz_image[dirs] = "${DEPLOY_DIR_IMAGE}"
diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
index 5085afcc..73d1bb57 100644
--- a/meta/classes/ext4-img.bbclass
+++ b/meta/classes/ext4-img.bbclass
@@ -1,24 +1,15 @@
# This software is a part of ISAR.
# Copyright (C) 2015-2017 ilbers GmbH
-EXT4_IMAGE_FILE = "${IMAGE_FULLNAME}.ext4.img"
-
-IMAGER_INSTALL += "e2fsprogs"
+IMAGER_INSTALL_ext4 += "e2fsprogs"
MKE2FS_ARGS ?= "-t ext4"
# Generate ext4 filesystem image
-do_ext4_image() {
- rm -f '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}'
-
- truncate -s ${ROOTFS_SIZE}K '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}'
+IMAGE_CMD_ext4() {
+ truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'
- image_do_mounts
-
- sudo chroot ${BUILDCHROOT_DIR} /sbin/mke2fs ${MKE2FS_ARGS} \
- -F -d '${PP_ROOTFS}' '${PP_DEPLOY}/${EXT4_IMAGE_FILE}'
+ ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
+ -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
}
-
-addtask ext4_image before do_image after do_image_tools
-do_ext4_image[prefuncs] = 'set_image_size'
-do_ext4_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+#IMAGE_CMD_ext4[vardepsexclude] = "ROOTFS_SIZE ROOTFS_EXTRA"
diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass
index 1ad0c5b8..ef65af88 100644
--- a/meta/classes/fit-img.bbclass
+++ b/meta/classes/fit-img.bbclass
@@ -6,24 +6,17 @@
MKIMAGE_ARGS ??= ""
FIT_IMAGE_SOURCE ??= "fitimage.its"
-FIT_IMAGE_FILE ?= "${IMAGE_FULLNAME}.fit.img"
-IMAGER_INSTALL += "u-boot-tools device-tree-compiler"
+IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"
# Generate fit image
-do_fit_image() {
+IMAGE_CMD_fit() {
if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
die "FIT_IMAGE_SOURCE does not contain fitimage source file"
fi
- rm -f '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}'
-
- image_do_mounts
-
# Create fit image using buildchroot tools
- sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \
- -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${PP_DEPLOY}/${FIT_IMAGE_FILE}'
- sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}'
+ ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
+ -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
}
-addtask fit_image before do_image after do_image_tools do_transform_template
-do_fit_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
deleted file mode 100644
index cdec4633..00000000
--- a/meta/classes/image-container-extension.bbclass
+++ /dev/null
@@ -1,83 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2021
-#
-# SPDX-License-Identifier: MIT
-#
-# This class extends the image.bbclass for containerizing the root filesystem.
-
-CONTAINER_IMAGE_FORMATS ?= "docker-archive"
-CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
-CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
-
-containerize_rootfs() {
- local cmd="/bin/dash"
- local empty_tag="empty"
- local tag="${CONTAINER_IMAGE_TAG}"
- local oci_img_dir="${WORKDIR}/oci-image"
- local rootfs="$1"
- local container_formats="$2"
- local container_name_prefix="$3"
-
- # prepare OCI container image skeleton
- bbdebug 1 "prepare OCI container image skeleton"
- sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
- sudo umoci init --layout "${oci_img_dir}"
- sudo umoci new --image "${oci_img_dir}:${empty_tag}"
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
- --config.cmd="${cmd}"
- sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
- "${oci_img_dir}_unpacked"
-
- # add root filesystem as the flesh of the skeleton
- sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
- # clean-up temporary files
- sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
-
- # pack container image
- bbdebug 1 "pack container image"
- sudo umoci repack --image "${oci_img_dir}:${tag}" \
- "${oci_img_dir}_unpacked"
- sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
- sudo rm -rf "${oci_img_dir}_unpacked"
-
- # no root needed anymore
- sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
-
- # convert the OCI container image to the desired format
- image_name="${container_name_prefix}${CONTAINER_IMAGE_NAME}"
- for image_type in ${CONTAINER_IMAGE_FORMATS} ; do
- image_archive="${DEPLOY_DIR_IMAGE}/${image_name}-${tag}-${image_type}.tar"
- bbdebug 1 "Creating container image type: ${image_type}"
- case "${image_type}" in
- "docker-archive" | "oci-archive")
- if [ "${image_type}" = "oci-archive" ] ; then
- target="${image_type}:${image_archive}:${tag}"
- else
- target="${image_type}:${image_archive}:${image_name}:${tag}"
- fi
- rm -f "${image_archive}" "${image_archive}.xz"
- bbdebug 2 "Converting OCI image to ${image_type}"
- skopeo --insecure-policy copy \
- "oci:${oci_img_dir}:${tag}" "${target}"
- bbdebug 2 "Compressing image"
- xz -T0 "${image_archive}"
- ;;
- "oci")
- tar --create --xz --directory "${oci_img_dir}" \
- --file "${image_archive}.xz" .
- ;;
- "docker-daemon" | "containers-storage")
- if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
- die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')"
- fi
- skopeo --insecure-policy copy \
- "oci:${oci_img_dir}:${tag}" \
- "${image_type}:${image_name}:${tag}"
- ;;
- *)
- die "Unsupported format for containerize_rootfs: ${image_type}"
- ;;
- esac
- done
-}
-
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index f87b76e7..4413a7d6 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -10,7 +10,8 @@ STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
IMAGE_INSTALL ?= ""
-IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4-img"}"
+IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4"}"
+IMAGE_CONVERSIONS = "gz xz"
IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
@@ -83,7 +84,183 @@ inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
-inherit image-container-extension
+
+def get_base_type(t, d):
+ bt = t
+ for c in d.getVar('IMAGE_CONVERSIONS').split():
+ if t.endswith('.' + c):
+ bt = t[:-len('.' + c)]
+ break
+ return bt if bt == t else get_base_type(bt, d)
+
+# determine image basetypes, just so we can use it in imagetypes* classes
+python() {
+ basetypes = set()
+ for t in (d.getVar('IMAGE_FSTYPES') or '').split():
+ bt = get_base_type(t, d)
+ if bt.endswith('-img'):
+ # be backwards-compatible
+ bt = bt[:-len('-img')]
+ bb.warn("IMAGE_TYPE '{0}-img' is deprecated. Please use '{0}' instead.".format(bt))
+ basetypes.add(bt)
+ deps = (d.getVar('IMAGE_TYPEDEP_' + bt.replace('-', '_').replace('.', '_')) or '').split()
+ basetypes |= set([get_base_type(t, d) for t in deps])
+ d.setVar('IMAGE_BASETYPES', ' '.join(basetypes))
+}
+
+# image types
+IMAGE_CLASSES ??= ""
+IMGCLASSES = "container-img cpiogz-img ext4-img fit-img targz-img ubi-img ubifs-img vm-img wic-img"
+IMGCLASSES += "${IMAGE_CLASSES}"
+inherit ${IMGCLASSES}
+
+# image conversions
+CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
+CONVERSION_DEPS_gz = "gzip"
+
+XZ_OPTIONS ?= ""
+CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'cat ${IMAGE_FILE_CHROOT} | xz ${XZ_OPTIONS} > ${IMAGE_FILE_CHROOT}.xz'"
+CONVERSION_DEPS_xz = "xz-utils"
+
+# hook up IMAGE_CMD_*
+python() {
+ image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
+ conversions = set(d.getVar('IMAGE_CONVERSIONS').split())
+
+ basetypes = {}
+ typedeps = {}
+ vardeps = set()
+
+ def collect_image_type(t):
+ bt = get_base_type(t, d)
+ if bt.endswith('-img'):
+ # be backwards-compatible
+ bt = bt[:-len('-img')]
+ bb.warn("IMAGE_TYPE '{0}-img' is deprecated. Please use '{0}' instead.".format(bt))
+
+ if bt not in basetypes:
+ basetypes[bt] = []
+ if t not in basetypes[bt]:
+ basetypes[bt].append(t)
+ t_clean = t.replace('-', '_').replace('.', '_')
+ deps = (d.getVar('IMAGE_TYPEDEP_' + t_clean) or '').split()
+ vardeps.add('IMAGE_TYPEDEP_' + t_clean)
+ if bt not in typedeps:
+ typedeps[bt] = set()
+ for dep in deps:
+ if dep not in image_types:
+ image_types.append(dep)
+ collect_image_type(dep)
+ typedeps[bt].add(get_base_type(dep, d))
+ if bt != t:
+ collect_image_type(bt)
+
+ for t in image_types[:]:
+ collect_image_type(t)
+
+ # TODO: OE uses do_image, but Isar is different...
+ d.appendVarFlag('do_image_tools', 'vardeps', ' '.join(vardeps))
+
+ imager_install = set()
+ imager_build_deps = set()
+ conversion_install = set()
+ for bt in basetypes:
+ vardeps = set()
+ cmds = []
+ bt_clean = bt.replace('-', '_').replace('.', '_')
+
+ # prepare local environment
+ localdata = bb.data.createCopy(d)
+ localdata.setVar('OVERRIDES', bt_clean + ':' + d.getVar('OVERRIDES', False))
+ localdata.setVar('PV', d.getVar('PV'))
+ localdata.delVar('DATETIME')
+ localdata.delVar('DATE')
+ localdata.delVar('TMPDIR')
+ vardepsexclude = (d.getVarFlag('IMAGE_CMD_' + bt_clean, 'vardepsexclude', True) or '').split()
+ for dep in vardepsexclude:
+ localdata.delVar(dep)
+
+ # check if required args are set
+ required_args = (localdata.getVar('IMAGE_CMD_REQUIRED_ARGS') or '').split()
+ if any([d.getVar(arg) is None for arg in required_args]):
+ bb.fatal("IMAGE_TYPE '%s' requires these arguments: %s" % (image_type, ', '.join(required_args)))
+
+ # convenience variables to be used by CMDs
+ localdata.setVar('IMAGE_FILE_HOST', '${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.${type}')
+ #bb.warn("FULLNAME is %s -> %s" % (localdata.getVar('IMAGE_FULLNAME', False), localdata.getVar('IMAGE_FULLNAME', True)))
+ localdata.setVar('IMAGE_FILE_CHROOT', '${PP_DEPLOY}/${IMAGE_FULLNAME}.${type}')
+ localdata.setVar('SUDO_CHROOT', localdata.expand('sudo chroot ${BUILDCHROOT_DIR}'))
+
+ # imager install
+ for dep in (d.getVar('IMAGER_INSTALL_' + bt_clean) or '').split():
+ imager_install.add(dep)
+ for dep in (d.getVar('IMAGER_BUILD_DEPS_' + bt_clean) or '').split():
+ imager_build_deps.add(dep)
+
+ # construct image command
+ cmds.append('\timage_do_mounts')
+ image_cmd = localdata.getVar('IMAGE_CMD_' + bt_clean)
+ if image_cmd:
+ localdata.setVar('type', bt)
+ cmds.append(localdata.expand(image_cmd))
+ #bb.warn("IMAGE_CMD\n*** %s\n*** %s" % (image_cmd, localdata.expand(image_cmd)))
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
+ else:
+ bb.fatal("No IMAGE_CMD for %s" % bt)
+ vardeps.add('IMAGE_CMD_' + bt_clean)
+ d.delVarFlag('IMAGE_CMD_' + bt_clean, 'func')
+ task_deps = d.getVarFlag('IMAGE_CMD_' + bt_clean, 'depends')
+
+ # add conversions
+ conversion_depends = set()
+ rm_images = set()
+ def create_conversions(t):
+ for c in sorted(conversions):
+ if t.endswith('.' + c):
+ t = t[:-len(c) - 1]
+ create_conversions(t)
+ localdata.setVar('type', t)
+ cmd = '\t' + localdata.getVar('CONVERSION_CMD_' + c)
+ if cmd not in cmds:
+ cmds.append(cmd)
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
+ vardeps.add('CONVERSION_CMD_' + c)
+ for dep in (localdata.getVar('CONVERSION_DEPS_' + c) or '').split():
+ conversion_install.add(dep)
+ # remove temporary image files
+ if t not in image_types:
+ rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
+
+ for t in basetypes[bt]:
+ create_conversions(t)
+
+ if bt not in image_types:
+ localdata.setVar('type', t)
+ rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
+
+ for image in rm_images:
+ cmds.append('\trm ' + image)
+
+ # image type dependencies
+ after = 'do_image_tools'
+ for dep in typedeps[bt]:
+ after += ' do_image_%s' % dep.replace('-', '_').replace('.', '_')
+
+ # create the task
+ task = 'do_image_%s' % bt_clean
+ d.setVar(task, '\n'.join(cmds))
+ d.setVarFlag(task, 'func', '1')
+ d.appendVarFlag(task, 'prefuncs', ' set_image_size')
+ d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps))
+ d.appendVarFlag(task, 'vardepsexclude', ' ' + ' '.join(vardepsexclude))
+ d.appendVarFlag(task, 'dirs', localdata.expand(' ${DEPLOY_DIR_IMAGE}'))
+ if task_deps:
+ d.appendVarFlag(task, 'depends', task_deps)
+ bb.build.addtask(task, 'do_image', after, d)
+
+ d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
+ d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
+}
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
@@ -256,6 +433,3 @@ do_rootfs_quality_check() {
}
addtask rootfs_quality_check after do_rootfs_finalize before do_rootfs
-
-# Last so that the image type can overwrite tasks if needed
-inherit ${IMAGE_FSTYPES}
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index b021e728..7e9fb0f3 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -172,6 +172,7 @@ rootfs_install_pkgs_install() {
do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}"
+do_rootfs_install[vardepsexclude] += "IMAGE_ROOTFS"
do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build"
do_rootfs_install[recrdeptask] = "do_deploy_deb"
python do_rootfs_install() {
diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
index adf9a1fe..477dff70 100644
--- a/meta/classes/sdk.bbclass
+++ b/meta/classes/sdk.bbclass
@@ -31,7 +31,7 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${MACHINE}"
# SDK settings
SDK_INCLUDE_ISAR_APT ?= "0"
-SDK_FORMATS ?= "targz-img"
+SDK_FORMATS ?= "tar.xz"
SDK_INSTALL ?= ""
SDK_PREINSTALL += " \
debhelper \
diff --git a/meta/classes/targz-img.bbclass b/meta/classes/targz-img.bbclass
index bf94af02..74d34e29 100644
--- a/meta/classes/targz-img.bbclass
+++ b/meta/classes/targz-img.bbclass
@@ -3,13 +3,6 @@
#
# SPDX-License-Identifier: MIT
-TARGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz"
-
-do_targz_image() {
- rm -f ${TARGZ_IMAGE_FILE}
- sudo tar -cvzf ${TARGZ_IMAGE_FILE} --one-file-system -C ${IMAGE_ROOTFS} .
- sudo chown $(id -u):$(id -g) ${TARGZ_IMAGE_FILE}
+IMAGE_CMD_tar() {
+ sudo tar -cvzf ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz --one-file-system -C ${IMAGE_ROOTFS} .
}
-
-addtask targz_image before do_image after do_image_tools
-do_targz_image[dirs] = "${DEPLOY_DIR_IMAGE}"
diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass
index efaf058e..92acb6f8 100644
--- a/meta/classes/ubi-img.bbclass
+++ b/meta/classes/ubi-img.bbclass
@@ -3,30 +3,18 @@
#
# SPDX-License-Identifier: MIT
-python() {
- if not d.getVar("UBINIZE_ARGS"):
- raise bb.parse.SkipRecipe("UBINIZE_ARGS must be set")
-}
-
UBINIZE_CFG ??= "ubinize.cfg"
-UBI_IMAGE_FILE ?= "${IMAGE_FULLNAME}.ubi.img"
-IMAGER_INSTALL += "mtd-utils"
+IMAGER_INSTALL_ubi += "mtd-utils"
# Generate ubi filesystem image
-do_ubi_image() {
+IMAGE_CMD_ubi() {
if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
die "UBINIZE_CFG does not contain ubinize config file."
fi
- rm -f '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}'
-
- image_do_mounts
-
- # Create ubi image using buildchroot tools
- sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \
- -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}' '${PP_WORK}/${UBINIZE_CFG}'
- sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}'
+ ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
+ -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
}
-addtask ubi_image before do_image after do_image_tools do_transform_template
-do_ubi_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
+IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
diff --git a/meta/classes/ubifs-img.bbclass b/meta/classes/ubifs-img.bbclass
index 229eb3ef..60c3bf14 100644
--- a/meta/classes/ubifs-img.bbclass
+++ b/meta/classes/ubifs-img.bbclass
@@ -3,30 +3,19 @@
#
# SPDX-License-Identifier: MIT
-python() {
- if not d.getVar("MKUBIFS_ARGS"):
- raise bb.parse.SkipRecipe("mkubifs_args must be set")
-}
-
-UBIFS_IMAGE_FILE ?= "${IMAGE_FULLNAME}.ubifs.img"
-
-IMAGER_INSTALL += "mtd-utils"
+IMAGER_INSTALL_ubifs += "mtd-utils"
# glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
# should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
-ISAR_CROSS_COMPILE_armhf = "1"
+python() {
+ if 'ubifs' in (d.getVar('IMAGE_BASETYPES') or '').split():
+ d.setVar('ISAR_CROSS_COMPILE_armhf', '1')
+}
# Generate ubifs filesystem image
-do_ubifs_image() {
- rm -f '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}'
-
- image_do_mounts
-
+IMAGE_CMD_ubifs() {
# Create ubifs image using buildchroot tools
- sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
- -r '${PP_ROOTFS}' '${PP_DEPLOY}/${UBIFS_IMAGE_FILE}'
- sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}'
+ ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
+ -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
}
-
-addtask ubifs_image before do_image after do_image_tools
-do_ubifs_image[dirs] = "${DEPLOY_DIR_IMAGE}"
+IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass
index 4bc977b9..8a676aca 100644
--- a/meta/classes/vm-img.bbclass
+++ b/meta/classes/vm-img.bbclass
@@ -5,16 +5,18 @@
#
inherit buildchroot
-inherit wic-img
+
+USING_OVA = "${@bb.utils.contains('IMAGE_BASETYPES', 'ova', '1', '0', d)}"
FILESEXTRAPATHS_prepend := "${LAYERDIR_core}/classes/vm-img:"
OVF_TEMPLATE_FILE ?= "vm-img-virtualbox.ovf.tmpl"
-SRC_URI += "file://${OVF_TEMPLATE_FILE}"
+SRC_URI += "${@'file://${OVF_TEMPLATE_FILE}' if d.getVar('USING_OVA') == '1' else ''}"
-IMAGER_INSTALL += "qemu-utils gawk uuid-runtime"
+IMAGE_TYPEDEP_ova = "wic"
+IMAGER_INSTALL_ova += "qemu-utils gawk uuid-runtime"
# virtual machine disk settings
-SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img"
+SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic"
# For VirtualBox, this needs to be "monolithicSparse" (default to it).
# VMware needs this to be "streamOptimized".
@@ -34,7 +36,7 @@ def set_convert_options(d):
CONVERSION_OPTIONS = "${@set_convert_options(d)}"
-do_convert_wic() {
+convert_wic() {
rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
image_do_mounts
bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${SOURCE_IMAGE_FILE}"
@@ -43,8 +45,6 @@ do_convert_wic() {
'${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${VIRTUAL_MACHINE_DISK}'
}
-addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template
-
# User settings for OVA
OVA_NAME ?= "${IMAGE_FULLNAME}"
OVA_MEMORY ?= "8192"
@@ -67,10 +67,11 @@ OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \
OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL \
OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE"
-TEMPLATE_FILES += "${OVF_TEMPLATE_FILE}"
+TEMPLATE_FILES += "${@'${OVF_TEMPLATE_FILE}' if d.getVar('USING_OVA') == '1' else ''}"
TEMPLATE_VARS += "${OVA_VARS}"
-do_create_ova() {
+do_image_ova[prefuncs] += "convert_wic"
+IMAGE_CMD_ova() {
if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then
exit 0
fi
@@ -81,10 +82,7 @@ do_create_ova() {
export PRIMARY_MAC=$(macgen)
export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
export OVA_FIRMWARE_UPPERCASE=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]')
-
export OVF_TEMPLATE_STAGE2=$(echo ${OVF_TEMPLATE_FILE} | sed 's/.tmpl$//' )
- image_do_mounts
-
sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO'
set -e
export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \
@@ -104,5 +102,3 @@ do_create_ova() {
tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE}
EOSUDO
}
-
-addtask do_create_ova after do_convert_wic before do_deploy
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 7537a27b..cfcc94c7 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -4,7 +4,8 @@
# this class is heavily inspired by OEs ./meta/classes/image_types_wic.bbclass
#
-WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}')}"
+USING_WIC = "${@bb.utils.contains('IMAGE_BASETYPES', 'wic', '1', '0', d)}"
+WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if d.getVar('USING_WIC') == '1' else ''}"
WKS_FILE ??= "sdimage-efi"
@@ -14,6 +15,9 @@ do_copy_wks_template () {
}
python () {
+ if not d.getVar('USING_WIC') == '1':
+ return
+
import itertools
import re
@@ -74,13 +78,13 @@ python () {
except (IOError, OSError) as exc:
pass
else:
- bb.build.addtask('do_copy_wks_template', 'do_transform_template do_wic_image', None, d)
- bb.build.addtask('do_transform_template', 'do_wic_image', None, d)
+ bb.build.addtask('do_copy_wks_template', 'do_transform_template do_image_wic', None, d)
+ bb.build.addtask('do_transform_template', 'do_image_wic', None, d)
}
inherit buildchroot
-IMAGER_INSTALL += "${WIC_IMAGER_INSTALL}"
+IMAGER_INSTALL_wic += "${WIC_IMAGER_INSTALL}"
# wic comes with reasonable defaults, and the proper interface is the wks file
ROOTFS_EXTRA ?= "0"
@@ -125,32 +129,23 @@ python do_rootfs_wicenv () {
}
-addtask do_rootfs_wicenv after do_rootfs before do_wic_image
+addtask do_rootfs_wicenv after do_rootfs before do_image_wic
do_rootfs_wicenv[vardeps] += "${WICVARS}"
do_rootfs_wicenv[prefuncs] = 'set_image_size'
-WIC_IMAGE_FILE ="${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.img"
-
-python check_for_wic_warnings() {
- with open("{}/log.do_wic_image".format(d.getVar("T"))) as f:
- for line in f.readlines():
- if line.startswith("WARNING"):
- bb.warn(line.strip())
+check_for_wic_warnings() {
+ WARN="$(grep -e '^WARNING' ${T}/log.do_image_wic || true)"
+ if [ -n "$WARN" ]; then
+ bbwarn "$WARN"
+ fi
}
-do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-do_wic_image[dirs] = "${DEPLOY_DIR_IMAGE}"
-python do_wic_image() {
- cmds = ['wic_do_mounts', 'generate_wic_image', 'check_for_wic_warnings']
- weights = [5, 90, 5]
- progress_reporter = bb.progress.MultiStageProgressReporter(d, weights)
-
- for cmd in cmds:
- progress_reporter.next_stage()
- bb.build.exec_func(cmd, d)
- progress_reporter.finish()
+do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
+IMAGE_CMD_wic() {
+ wic_do_mounts
+ generate_wic_image
+ check_for_wic_warnings
}
-addtask wic_image before do_image after do_image_tools
wic_do_mounts() {
buildchroot_do_mounts
@@ -209,7 +204,7 @@ generate_wic_image() {
sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
find ${BUILDCHROOT_DIR}/${WICTMP} -type f -name "*.direct*" | while read f; do
suffix=$(basename $f | sed 's/\(.*\)\(\.direct\)\(.*\)/\3/')
- mv -f ${f} ${WIC_IMAGE_FILE}${suffix}
+ mv -f ${f} "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic${suffix}"
done
rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
rm -rf ${IMAGE_ROOTFS}/../pseudo
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 3/7] meta: introduce IMAGE_CMD_*
2022-04-05 13:24 ` [PATCH v2 3/7] meta: introduce IMAGE_CMD_* Adriaan Schmidt
@ 2022-04-14 17:46 ` Anton Mikanovich
2022-04-15 6:55 ` Schmidt, Adriaan
0 siblings, 1 reply; 10+ messages in thread
From: Anton Mikanovich @ 2022-04-14 17:46 UTC (permalink / raw)
To: Adriaan Schmidt, isar-users
Hello, Adriaan
05.04.2022 16:24, Adriaan Schmidt wrote:
> This makes a number of changes:
> - Replace all do_*_image tasks with IMAGE_CMD_* definitions
> - Remove all implicit conversions (gz, xz)
> - Add IMAGE_CONVERSION_* commands
> - Create tasks (do_image_*) on demand in image.bbclass for
> all requested IMAGE_FSTYPES
> - When creating the tasks:
> - image_do_mounts is inserted automatically
> - a final chown is inserted automatically
> - variables IMAGE_FILE_HOST and IMAGE_FILE_CHROOT are
> set to reference the image
> - variable SUDO_CHROOT contains the chroot command
> - Create conversions on demand based on IMAGE_FSTYPES.
> A conversion is defined by CONVERSION_CMD_type, and its
> dependencies given as CONVERSION_DEPS_type
> - In conversion commands
> - the input file is named ${IMAGE_FULLNAME}.${type}
> - the conversions appends its own type, e.g. the output file
> would be ${IMAGE_FULLNAME}.${type}.xz
> - a final chown is appended automatically
> - Image types now longer have a -img suffix, e.g., ext4 instead
> of ext4-img, and conversions are appended like tar.gz instead
> of targz-img
> - Imager dependencies are set as IMAGER_INSTALL_type
> - Dependencies between image types are modelled like
> IMAGE_TYPEDEP_ova = "wic"
> - Required arguments/variables are modelled by
> IMAGE_CMD_REQUIRED_ARGUMENTS = "A B C"
> - Container types (previously CONTAINER_IMAGE_FORMATS) are now
> first class image types (oci, oci-archive, docker-archive,
> docker-daemon, containers.storage)
> - The logic of image-container-extension has moved to
> container-img
> - The VM image now has type ova (instead of vm-img)
>
> Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
> ---
> meta/classes/container-img.bbclass | 94 ++++++++-
> meta/classes/cpiogz-img.bbclass | 20 +-
> meta/classes/ext4-img.bbclass | 21 +-
> meta/classes/fit-img.bbclass | 17 +-
> .../classes/image-container-extension.bbclass | 83 --------
> meta/classes/image.bbclass | 184 +++++++++++++++++-
> meta/classes/rootfs.bbclass | 1 +
> meta/classes/sdk.bbclass | 2 +-
> meta/classes/targz-img.bbclass | 11 +-
> meta/classes/ubi-img.bbclass | 24 +--
> meta/classes/ubifs-img.bbclass | 29 +--
> meta/classes/vm-img.bbclass | 24 +--
> meta/classes/wic-img.bbclass | 45 ++---
> 13 files changed, 331 insertions(+), 224 deletions(-)
> delete mode 100644 meta/classes/image-container-extension.bbclass
>
> diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
> index 322889a9..436a0051 100644
> --- a/meta/classes/container-img.bbclass
> +++ b/meta/classes/container-img.bbclass
> @@ -3,15 +3,93 @@
> #
> # SPDX-License-Identifier: MIT
> #
> -# This class provides the task 'container_image'
> +# This class provides the task 'containerize'
> # to create container images containing the target rootfs.
>
> -do_container_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> -do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> -do_container_image[vardeps] += "CONTAINER_IMAGE_FORMATS"
> -do_container_image(){
> - bbdebug 1 "Generate container image in these formats: ${CONTAINER_IMAGE_FORMATS}"
> - containerize_rootfs "${IMAGE_ROOTFS}" "${CONTAINER_IMAGE_FORMATS}"
> +CONTAINER_TYPES = "oci oci-archive docker-archive docker-daemon containers-storage"
> +USING_CONTAINER = "${@bb.utils.contains_any('IMAGE_BASETYPES', d.getVar('CONTAINER_TYPES').split(), '1', '0', d)}"
> +
> +CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
> +CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
> +
> +python() {
> + if not d.getVar('USING_CONTAINER') == '1':
> + return
> + for t in d.getVar('CONTAINER_TYPES').split():
> + t_clean = t.replace('-', '_').replace('.', '_')
> + d.setVar('IMAGE_CMD_' + t_clean, 'convert_container %s "${CONTAINER_IMAGE_NAME}" "${IMAGE_FILE_HOST}"' % t)
> + d.setVar('IMAGE_FULLNAME_' + t_clean, '${PN}-${DISTRO}-${DISTRO_ARCH}')
> + bb.build.addtask('containerize', 'do_image_' + t_clean, 'do_image_tools', d)
> +}
> +
> +do_containerize() {
> + local cmd="/bin/dash"
> + local empty_tag="empty"
> + local tag="${CONTAINER_IMAGE_TAG}"
> + local oci_img_dir="${WORKDIR}/oci-image"
> + local rootfs="${IMAGE_ROOTFS}"
> +
> + # prepare OCI container image skeleton
> + bbdebug 1 "prepare OCI container image skeleton"
> + sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
> + sudo umoci init --layout "${oci_img_dir}"
> + sudo umoci new --image "${oci_img_dir}:${empty_tag}"
> + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
> + --config.cmd="${cmd}"
> + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
> + "${oci_img_dir}_unpacked"
> +
> + # add root filesystem as the flesh of the skeleton
> + sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
> + # clean-up temporary files
> + sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
> +
> + # pack container image
> + bbdebug 1 "pack container image"
> + sudo umoci repack --image "${oci_img_dir}:${tag}" \
> + "${oci_img_dir}_unpacked"
> + sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
> + sudo rm -rf "${oci_img_dir}_unpacked"
> +
> + # no root needed anymore
> + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
> }
>
> -addtask container_image before do_image after do_image_tools
> +convert_container() {
> + local tag="${CONTAINER_IMAGE_TAG}"
> + local oci_img_dir="${WORKDIR}/oci-image"
> + local container_type="$1"
> + local image_name="$2"
> + local image_archive="$3"
> +
> + # convert the OCI container image to the desired format
> + bbdebug 1 "Creating container image type: ${container_type}"
> + case "${container_type}" in
> + "docker-archive" | "oci-archive")
> + if [ "${container_type}" = "oci-archive" ] ; then
> + target="${container_type}:${image_archive}:${tag}"
> + else
> + target="${container_type}:${image_archive}:${image_name}:${tag}"
> + fi
> + rm -f "${image_archive}"
> + bbdebug 2 "Converting OCI image to ${container_type}"
> + skopeo --insecure-policy copy \
> + "oci:${oci_img_dir}:${tag}" "${target}"
> + ;;
> + "oci")
> + tar --create --directory "${oci_img_dir}" \
> + --file "${image_archive}" .
> + ;;
> + "docker-daemon" | "containers-storage")
> + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
> + die "Adding the container image to a container runtime (${container_type}) not supported if running from a container (e.g. 'kas-container')"
> + fi
> + skopeo --insecure-policy copy \
> + "oci:${oci_img_dir}:${tag}" \
> + "${container_type}:${image_name}:${tag}"
> + ;;
> + *)
> + die "Unsupported format for convert_container: ${container_type}"
> + ;;
> + esac
> +}
> diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
> index 2a49456b..f4c33bd9 100644
> --- a/meta/classes/cpiogz-img.bbclass
> +++ b/meta/classes/cpiogz-img.bbclass
> @@ -3,20 +3,12 @@
> #
> # SPDX-License-Identifier: MIT
>
> -CPIOGZ_FNAME ?= "${IMAGE_FULLNAME}.cpio.gz"
> -CPIOGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${CPIOGZ_FNAME}"
> -IMAGER_INSTALL += "cpio"
> +IMAGER_INSTALL_cpio += "cpio"
> CPIO_IMAGE_FORMAT ?= "newc"
>
> -do_cpiogz_image() {
> - sudo rm -f ${CPIOGZ_IMAGE_FILE}
> - image_do_mounts
> - sudo chroot ${BUILDCHROOT_DIR} \
> - sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
> - /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o | /usr/bin/gzip -9 > \
> - ${PP_DEPLOY}/${CPIOGZ_FNAME}"
> - sudo chown $(id -u):$(id -g) ${CPIOGZ_IMAGE_FILE}
> +IMAGE_CMD_cpio() {
> + ${SUDO_CHROOT} \
> + sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
> + /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
> + ${IMAGE_FILE_CHROOT}"
> }
> -
> -addtask cpiogz_image before do_image after do_image_tools
> -do_cpiogz_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
> index 5085afcc..73d1bb57 100644
> --- a/meta/classes/ext4-img.bbclass
> +++ b/meta/classes/ext4-img.bbclass
> @@ -1,24 +1,15 @@
> # This software is a part of ISAR.
> # Copyright (C) 2015-2017 ilbers GmbH
>
> -EXT4_IMAGE_FILE = "${IMAGE_FULLNAME}.ext4.img"
> -
> -IMAGER_INSTALL += "e2fsprogs"
> +IMAGER_INSTALL_ext4 += "e2fsprogs"
>
> MKE2FS_ARGS ?= "-t ext4"
>
> # Generate ext4 filesystem image
> -do_ext4_image() {
> - rm -f '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}'
> -
> - truncate -s ${ROOTFS_SIZE}K '${DEPLOY_DIR_IMAGE}/${EXT4_IMAGE_FILE}'
> +IMAGE_CMD_ext4() {
> + truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'
>
> - image_do_mounts
> -
> - sudo chroot ${BUILDCHROOT_DIR} /sbin/mke2fs ${MKE2FS_ARGS} \
> - -F -d '${PP_ROOTFS}' '${PP_DEPLOY}/${EXT4_IMAGE_FILE}'
> + ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
> + -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
> }
> -
> -addtask ext4_image before do_image after do_image_tools
> -do_ext4_image[prefuncs] = 'set_image_size'
> -do_ext4_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> +#IMAGE_CMD_ext4[vardepsexclude] = "ROOTFS_SIZE ROOTFS_EXTRA"
> diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass
> index 1ad0c5b8..ef65af88 100644
> --- a/meta/classes/fit-img.bbclass
> +++ b/meta/classes/fit-img.bbclass
> @@ -6,24 +6,17 @@
> MKIMAGE_ARGS ??= ""
>
> FIT_IMAGE_SOURCE ??= "fitimage.its"
> -FIT_IMAGE_FILE ?= "${IMAGE_FULLNAME}.fit.img"
>
> -IMAGER_INSTALL += "u-boot-tools device-tree-compiler"
> +IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"
>
> # Generate fit image
> -do_fit_image() {
> +IMAGE_CMD_fit() {
> if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
> die "FIT_IMAGE_SOURCE does not contain fitimage source file"
> fi
>
> - rm -f '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}'
> -
> - image_do_mounts
> -
> # Create fit image using buildchroot tools
> - sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \
> - -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${PP_DEPLOY}/${FIT_IMAGE_FILE}'
> - sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${FIT_IMAGE_FILE}'
> + ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
> + -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
> }
> -addtask fit_image before do_image after do_image_tools do_transform_template
> -do_fit_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> +IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
> diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
> deleted file mode 100644
> index cdec4633..00000000
> --- a/meta/classes/image-container-extension.bbclass
> +++ /dev/null
> @@ -1,83 +0,0 @@
> -# This software is a part of ISAR.
> -# Copyright (C) Siemens AG, 2021
> -#
> -# SPDX-License-Identifier: MIT
> -#
> -# This class extends the image.bbclass for containerizing the root filesystem.
> -
> -CONTAINER_IMAGE_FORMATS ?= "docker-archive"
> -CONTAINER_IMAGE_NAME ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
> -CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
> -
> -containerize_rootfs() {
> - local cmd="/bin/dash"
> - local empty_tag="empty"
> - local tag="${CONTAINER_IMAGE_TAG}"
> - local oci_img_dir="${WORKDIR}/oci-image"
> - local rootfs="$1"
> - local container_formats="$2"
> - local container_name_prefix="$3"
> -
> - # prepare OCI container image skeleton
> - bbdebug 1 "prepare OCI container image skeleton"
> - sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
> - sudo umoci init --layout "${oci_img_dir}"
> - sudo umoci new --image "${oci_img_dir}:${empty_tag}"
> - sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
> - --config.cmd="${cmd}"
> - sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
> - "${oci_img_dir}_unpacked"
> -
> - # add root filesystem as the flesh of the skeleton
> - sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
> - # clean-up temporary files
> - sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
> -
> - # pack container image
> - bbdebug 1 "pack container image"
> - sudo umoci repack --image "${oci_img_dir}:${tag}" \
> - "${oci_img_dir}_unpacked"
> - sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
> - sudo rm -rf "${oci_img_dir}_unpacked"
> -
> - # no root needed anymore
> - sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
> -
> - # convert the OCI container image to the desired format
> - image_name="${container_name_prefix}${CONTAINER_IMAGE_NAME}"
> - for image_type in ${CONTAINER_IMAGE_FORMATS} ; do
> - image_archive="${DEPLOY_DIR_IMAGE}/${image_name}-${tag}-${image_type}.tar"
> - bbdebug 1 "Creating container image type: ${image_type}"
> - case "${image_type}" in
> - "docker-archive" | "oci-archive")
> - if [ "${image_type}" = "oci-archive" ] ; then
> - target="${image_type}:${image_archive}:${tag}"
> - else
> - target="${image_type}:${image_archive}:${image_name}:${tag}"
> - fi
> - rm -f "${image_archive}" "${image_archive}.xz"
> - bbdebug 2 "Converting OCI image to ${image_type}"
> - skopeo --insecure-policy copy \
> - "oci:${oci_img_dir}:${tag}" "${target}"
> - bbdebug 2 "Compressing image"
> - xz -T0 "${image_archive}"
> - ;;
> - "oci")
> - tar --create --xz --directory "${oci_img_dir}" \
> - --file "${image_archive}.xz" .
> - ;;
> - "docker-daemon" | "containers-storage")
> - if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
> - die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')"
> - fi
> - skopeo --insecure-policy copy \
> - "oci:${oci_img_dir}:${tag}" \
> - "${image_type}:${image_name}:${tag}"
> - ;;
> - *)
> - die "Unsupported format for containerize_rootfs: ${image_type}"
> - ;;
> - esac
> - done
> -}
> -
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index f87b76e7..4413a7d6 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -10,7 +10,8 @@ STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}-${MACHINE}/*-*"
> SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}"
>
> IMAGE_INSTALL ?= ""
> -IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4-img"}"
> +IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4"}"
> +IMAGE_CONVERSIONS = "gz xz"
> IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
>
> KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
> @@ -83,7 +84,183 @@ inherit image-tools-extension
> inherit image-postproc-extension
> inherit image-locales-extension
> inherit image-account-extension
> -inherit image-container-extension
> +
> +def get_base_type(t, d):
> + bt = t
> + for c in d.getVar('IMAGE_CONVERSIONS').split():
> + if t.endswith('.' + c):
> + bt = t[:-len('.' + c)]
> + break
> + return bt if bt == t else get_base_type(bt, d)
> +
> +# determine image basetypes, just so we can use it in imagetypes* classes
> +python() {
> + basetypes = set()
> + for t in (d.getVar('IMAGE_FSTYPES') or '').split():
> + bt = get_base_type(t, d)
> + if bt.endswith('-img'):
> + # be backwards-compatible
> + bt = bt[:-len('-img')]
> + bb.warn("IMAGE_TYPE '{0}-img' is deprecated. Please use '{0}' instead.".format(bt))
> + basetypes.add(bt)
> + deps = (d.getVar('IMAGE_TYPEDEP_' + bt.replace('-', '_').replace('.', '_')) or '').split()
> + basetypes |= set([get_base_type(t, d) for t in deps])
> + d.setVar('IMAGE_BASETYPES', ' '.join(basetypes))
> +}
> +
> +# image types
> +IMAGE_CLASSES ??= ""
> +IMGCLASSES = "container-img cpiogz-img ext4-img fit-img targz-img ubi-img ubifs-img vm-img wic-img"
> +IMGCLASSES += "${IMAGE_CLASSES}"
> +inherit ${IMGCLASSES}
> +
> +# image conversions
> +CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
> +CONVERSION_DEPS_gz = "gzip"
> +
> +XZ_OPTIONS ?= ""
> +CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'cat ${IMAGE_FILE_CHROOT} | xz ${XZ_OPTIONS} > ${IMAGE_FILE_CHROOT}.xz'"
> +CONVERSION_DEPS_xz = "xz-utils"
> +
> +# hook up IMAGE_CMD_*
> +python() {
> + image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
> + conversions = set(d.getVar('IMAGE_CONVERSIONS').split())
> +
> + basetypes = {}
> + typedeps = {}
> + vardeps = set()
> +
> + def collect_image_type(t):
> + bt = get_base_type(t, d)
> + if bt.endswith('-img'):
> + # be backwards-compatible
> + bt = bt[:-len('-img')]
> + bb.warn("IMAGE_TYPE '{0}-img' is deprecated. Please use '{0}' instead.".format(bt))
> +
> + if bt not in basetypes:
> + basetypes[bt] = []
> + if t not in basetypes[bt]:
> + basetypes[bt].append(t)
> + t_clean = t.replace('-', '_').replace('.', '_')
> + deps = (d.getVar('IMAGE_TYPEDEP_' + t_clean) or '').split()
> + vardeps.add('IMAGE_TYPEDEP_' + t_clean)
> + if bt not in typedeps:
> + typedeps[bt] = set()
> + for dep in deps:
> + if dep not in image_types:
> + image_types.append(dep)
> + collect_image_type(dep)
> + typedeps[bt].add(get_base_type(dep, d))
> + if bt != t:
> + collect_image_type(bt)
> +
> + for t in image_types[:]:
> + collect_image_type(t)
> +
> + # TODO: OE uses do_image, but Isar is different...
> + d.appendVarFlag('do_image_tools', 'vardeps', ' '.join(vardeps))
> +
> + imager_install = set()
> + imager_build_deps = set()
> + conversion_install = set()
> + for bt in basetypes:
> + vardeps = set()
> + cmds = []
> + bt_clean = bt.replace('-', '_').replace('.', '_')
> +
> + # prepare local environment
> + localdata = bb.data.createCopy(d)
> + localdata.setVar('OVERRIDES', bt_clean + ':' + d.getVar('OVERRIDES', False))
> + localdata.setVar('PV', d.getVar('PV'))
> + localdata.delVar('DATETIME')
> + localdata.delVar('DATE')
> + localdata.delVar('TMPDIR')
> + vardepsexclude = (d.getVarFlag('IMAGE_CMD_' + bt_clean, 'vardepsexclude', True) or '').split()
> + for dep in vardepsexclude:
> + localdata.delVar(dep)
> +
> + # check if required args are set
> + required_args = (localdata.getVar('IMAGE_CMD_REQUIRED_ARGS') or '').split()
> + if any([d.getVar(arg) is None for arg in required_args]):
> + bb.fatal("IMAGE_TYPE '%s' requires these arguments: %s" % (image_type, ', '.join(required_args)))
> +
> + # convenience variables to be used by CMDs
> + localdata.setVar('IMAGE_FILE_HOST', '${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.${type}')
> + #bb.warn("FULLNAME is %s -> %s" % (localdata.getVar('IMAGE_FULLNAME', False), localdata.getVar('IMAGE_FULLNAME', True)))
> + localdata.setVar('IMAGE_FILE_CHROOT', '${PP_DEPLOY}/${IMAGE_FULLNAME}.${type}')
> + localdata.setVar('SUDO_CHROOT', localdata.expand('sudo chroot ${BUILDCHROOT_DIR}'))
> +
> + # imager install
> + for dep in (d.getVar('IMAGER_INSTALL_' + bt_clean) or '').split():
> + imager_install.add(dep)
> + for dep in (d.getVar('IMAGER_BUILD_DEPS_' + bt_clean) or '').split():
> + imager_build_deps.add(dep)
> +
> + # construct image command
> + cmds.append('\timage_do_mounts')
> + image_cmd = localdata.getVar('IMAGE_CMD_' + bt_clean)
> + if image_cmd:
> + localdata.setVar('type', bt)
> + cmds.append(localdata.expand(image_cmd))
> + #bb.warn("IMAGE_CMD\n*** %s\n*** %s" % (image_cmd, localdata.expand(image_cmd)))
> + cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
> + else:
> + bb.fatal("No IMAGE_CMD for %s" % bt)
> + vardeps.add('IMAGE_CMD_' + bt_clean)
> + d.delVarFlag('IMAGE_CMD_' + bt_clean, 'func')
> + task_deps = d.getVarFlag('IMAGE_CMD_' + bt_clean, 'depends')
> +
> + # add conversions
> + conversion_depends = set()
> + rm_images = set()
> + def create_conversions(t):
> + for c in sorted(conversions):
> + if t.endswith('.' + c):
> + t = t[:-len(c) - 1]
> + create_conversions(t)
> + localdata.setVar('type', t)
> + cmd = '\t' + localdata.getVar('CONVERSION_CMD_' + c)
> + if cmd not in cmds:
> + cmds.append(cmd)
> + cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
> + vardeps.add('CONVERSION_CMD_' + c)
> + for dep in (localdata.getVar('CONVERSION_DEPS_' + c) or '').split():
> + conversion_install.add(dep)
> + # remove temporary image files
> + if t not in image_types:
> + rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
> +
> + for t in basetypes[bt]:
> + create_conversions(t)
> +
> + if bt not in image_types:
> + localdata.setVar('type', t)
> + rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
> +
> + for image in rm_images:
> + cmds.append('\trm ' + image)
> +
> + # image type dependencies
> + after = 'do_image_tools'
> + for dep in typedeps[bt]:
> + after += ' do_image_%s' % dep.replace('-', '_').replace('.', '_')
> +
> + # create the task
> + task = 'do_image_%s' % bt_clean
> + d.setVar(task, '\n'.join(cmds))
> + d.setVarFlag(task, 'func', '1')
> + d.appendVarFlag(task, 'prefuncs', ' set_image_size')
> + d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps))
> + d.appendVarFlag(task, 'vardepsexclude', ' ' + ' '.join(vardepsexclude))
> + d.appendVarFlag(task, 'dirs', localdata.expand(' ${DEPLOY_DIR_IMAGE}'))
> + if task_deps:
> + d.appendVarFlag(task, 'depends', task_deps)
> + bb.build.addtask(task, 'do_image', after, d)
> +
> + d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
> + d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
> +}
>
> # Extra space for rootfs in MB
> ROOTFS_EXTRA ?= "64"
> @@ -256,6 +433,3 @@ do_rootfs_quality_check() {
> }
>
> addtask rootfs_quality_check after do_rootfs_finalize before do_rootfs
> -
> -# Last so that the image type can overwrite tasks if needed
> -inherit ${IMAGE_FSTYPES}
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index b021e728..7e9fb0f3 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -172,6 +172,7 @@ rootfs_install_pkgs_install() {
>
> do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
> do_rootfs_install[vardeps] += "${ROOTFS_CONFIGURE_COMMAND} ${ROOTFS_INSTALL_COMMAND}"
> +do_rootfs_install[vardepsexclude] += "IMAGE_ROOTFS"
> do_rootfs_install[depends] = "isar-bootstrap-${@'target' if d.getVar('ROOTFS_ARCH') == d.getVar('DISTRO_ARCH') else 'host'}:do_build"
> do_rootfs_install[recrdeptask] = "do_deploy_deb"
> python do_rootfs_install() {
> diff --git a/meta/classes/sdk.bbclass b/meta/classes/sdk.bbclass
> index adf9a1fe..477dff70 100644
> --- a/meta/classes/sdk.bbclass
> +++ b/meta/classes/sdk.bbclass
> @@ -31,7 +31,7 @@ SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${MACHINE}"
>
> # SDK settings
> SDK_INCLUDE_ISAR_APT ?= "0"
> -SDK_FORMATS ?= "targz-img"
> +SDK_FORMATS ?= "tar.xz"
> SDK_INSTALL ?= ""
> SDK_PREINSTALL += " \
> debhelper \
> diff --git a/meta/classes/targz-img.bbclass b/meta/classes/targz-img.bbclass
> index bf94af02..74d34e29 100644
> --- a/meta/classes/targz-img.bbclass
> +++ b/meta/classes/targz-img.bbclass
> @@ -3,13 +3,6 @@
> #
> # SPDX-License-Identifier: MIT
>
> -TARGZ_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz"
> -
> -do_targz_image() {
> - rm -f ${TARGZ_IMAGE_FILE}
> - sudo tar -cvzf ${TARGZ_IMAGE_FILE} --one-file-system -C ${IMAGE_ROOTFS} .
> - sudo chown $(id -u):$(id -g) ${TARGZ_IMAGE_FILE}
> +IMAGE_CMD_tar() {
> + sudo tar -cvzf ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz --one-file-system -C ${IMAGE_ROOTFS} .
> }
> -
> -addtask targz_image before do_image after do_image_tools
> -do_targz_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass
> index efaf058e..92acb6f8 100644
> --- a/meta/classes/ubi-img.bbclass
> +++ b/meta/classes/ubi-img.bbclass
> @@ -3,30 +3,18 @@
> #
> # SPDX-License-Identifier: MIT
>
> -python() {
> - if not d.getVar("UBINIZE_ARGS"):
> - raise bb.parse.SkipRecipe("UBINIZE_ARGS must be set")
> -}
> -
> UBINIZE_CFG ??= "ubinize.cfg"
> -UBI_IMAGE_FILE ?= "${IMAGE_FULLNAME}.ubi.img"
>
> -IMAGER_INSTALL += "mtd-utils"
> +IMAGER_INSTALL_ubi += "mtd-utils"
>
> # Generate ubi filesystem image
> -do_ubi_image() {
> +IMAGE_CMD_ubi() {
> if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
> die "UBINIZE_CFG does not contain ubinize config file."
> fi
>
> - rm -f '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}'
> -
> - image_do_mounts
> -
> - # Create ubi image using buildchroot tools
> - sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \
> - -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}' '${PP_WORK}/${UBINIZE_CFG}'
> - sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBI_IMAGE_FILE}'
> + ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
> + -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
> }
> -addtask ubi_image before do_image after do_image_tools do_transform_template
> -do_ubi_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> +IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
> +IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
> diff --git a/meta/classes/ubifs-img.bbclass b/meta/classes/ubifs-img.bbclass
> index 229eb3ef..60c3bf14 100644
> --- a/meta/classes/ubifs-img.bbclass
> +++ b/meta/classes/ubifs-img.bbclass
> @@ -3,30 +3,19 @@
> #
> # SPDX-License-Identifier: MIT
>
> -python() {
> - if not d.getVar("MKUBIFS_ARGS"):
> - raise bb.parse.SkipRecipe("mkubifs_args must be set")
> -}
> -
> -UBIFS_IMAGE_FILE ?= "${IMAGE_FULLNAME}.ubifs.img"
> -
> -IMAGER_INSTALL += "mtd-utils"
> +IMAGER_INSTALL_ubifs += "mtd-utils"
>
> # glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
> # should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
> -ISAR_CROSS_COMPILE_armhf = "1"
> +python() {
> + if 'ubifs' in (d.getVar('IMAGE_BASETYPES') or '').split():
> + d.setVar('ISAR_CROSS_COMPILE_armhf', '1')
> +}
>
This will also not work and actually fails on imx6-sabrelite-bullseye
target.
Reasons:
1) IMAGE_BASETYPES = "ubi ubi-ubifs" for that target and condition is False
2) ISAR_CROSS_COMPILE should be set before anonymous python code because
buildchroot selection is done also in python() {}
Also it better to document (or at least comment) IMAGE_BASETYPES as a
new variable.
> # Generate ubifs filesystem image
> -do_ubifs_image() {
> - rm -f '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}'
> -
> - image_do_mounts
> -
> +IMAGE_CMD_ubifs() {
> # Create ubifs image using buildchroot tools
> - sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
> - -r '${PP_ROOTFS}' '${PP_DEPLOY}/${UBIFS_IMAGE_FILE}'
> - sudo chown $(id -u):$(id -g) '${DEPLOY_DIR_IMAGE}/${UBIFS_IMAGE_FILE}'
> + ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
> + -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
> }
> -
> -addtask ubifs_image before do_image after do_image_tools
> -do_ubifs_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> +IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
> diff --git a/meta/classes/vm-img.bbclass b/meta/classes/vm-img.bbclass
> index 4bc977b9..8a676aca 100644
> --- a/meta/classes/vm-img.bbclass
> +++ b/meta/classes/vm-img.bbclass
> @@ -5,16 +5,18 @@
> #
>
> inherit buildchroot
> -inherit wic-img
> +
> +USING_OVA = "${@bb.utils.contains('IMAGE_BASETYPES', 'ova', '1', '0', d)}"
>
> FILESEXTRAPATHS_prepend := "${LAYERDIR_core}/classes/vm-img:"
> OVF_TEMPLATE_FILE ?= "vm-img-virtualbox.ovf.tmpl"
> -SRC_URI += "file://${OVF_TEMPLATE_FILE}"
> +SRC_URI += "${@'file://${OVF_TEMPLATE_FILE}' if d.getVar('USING_OVA') == '1' else ''}"
>
> -IMAGER_INSTALL += "qemu-utils gawk uuid-runtime"
> +IMAGE_TYPEDEP_ova = "wic"
> +IMAGER_INSTALL_ova += "qemu-utils gawk uuid-runtime"
>
> # virtual machine disk settings
> -SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic.img"
> +SOURCE_IMAGE_FILE ?= "${IMAGE_FULLNAME}.wic"
>
> # For VirtualBox, this needs to be "monolithicSparse" (default to it).
> # VMware needs this to be "streamOptimized".
> @@ -34,7 +36,7 @@ def set_convert_options(d):
>
> CONVERSION_OPTIONS = "${@set_convert_options(d)}"
>
> -do_convert_wic() {
> +convert_wic() {
> rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
> image_do_mounts
> bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${SOURCE_IMAGE_FILE}"
> @@ -43,8 +45,6 @@ do_convert_wic() {
> '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${VIRTUAL_MACHINE_DISK}'
> }
>
> -addtask convert_wic before do_build after do_wic_image do_copy_boot_files do_install_imager_deps do_transform_template
> -
> # User settings for OVA
> OVA_NAME ?= "${IMAGE_FULLNAME}"
> OVA_MEMORY ?= "8192"
> @@ -67,10 +67,11 @@ OVA_VARS = "OVA_NAME OVA_MEMORY OVA_NUMBER_OF_CPU OVA_VRAM \
> OVA_FIRMWARE OVA_ACPI OVA_3D_ACCEL \
> OVA_SHA_ALG VIRTUAL_MACHINE_IMAGE_FILE"
>
> -TEMPLATE_FILES += "${OVF_TEMPLATE_FILE}"
> +TEMPLATE_FILES += "${@'${OVF_TEMPLATE_FILE}' if d.getVar('USING_OVA') == '1' else ''}"
> TEMPLATE_VARS += "${OVA_VARS}"
>
> -do_create_ova() {
> +do_image_ova[prefuncs] += "convert_wic"
> +IMAGE_CMD_ova() {
> if [ ! ${VIRTUAL_MACHINE_IMAGE_TYPE} = "vmdk" ]; then
> exit 0
> fi
> @@ -81,10 +82,7 @@ do_create_ova() {
> export PRIMARY_MAC=$(macgen)
> export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
> export OVA_FIRMWARE_UPPERCASE=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]')
> -
> export OVF_TEMPLATE_STAGE2=$(echo ${OVF_TEMPLATE_FILE} | sed 's/.tmpl$//' )
> - image_do_mounts
> -
> sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO'
> set -e
> export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \
> @@ -104,5 +102,3 @@ do_create_ova() {
> tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE}
> EOSUDO
> }
> -
> -addtask do_create_ova after do_convert_wic before do_deploy
> diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
> index 7537a27b..cfcc94c7 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -4,7 +4,8 @@
> # this class is heavily inspired by OEs ./meta/classes/image_types_wic.bbclass
> #
>
> -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}')}"
> +USING_WIC = "${@bb.utils.contains('IMAGE_BASETYPES', 'wic', '1', '0', d)}"
> +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if d.getVar('USING_WIC') == '1' else ''}"
>
> WKS_FILE ??= "sdimage-efi"
>
> @@ -14,6 +15,9 @@ do_copy_wks_template () {
> }
>
> python () {
> + if not d.getVar('USING_WIC') == '1':
> + return
> +
> import itertools
> import re
>
> @@ -74,13 +78,13 @@ python () {
> except (IOError, OSError) as exc:
> pass
> else:
> - bb.build.addtask('do_copy_wks_template', 'do_transform_template do_wic_image', None, d)
> - bb.build.addtask('do_transform_template', 'do_wic_image', None, d)
> + bb.build.addtask('do_copy_wks_template', 'do_transform_template do_image_wic', None, d)
> + bb.build.addtask('do_transform_template', 'do_image_wic', None, d)
> }
>
> inherit buildchroot
>
> -IMAGER_INSTALL += "${WIC_IMAGER_INSTALL}"
> +IMAGER_INSTALL_wic += "${WIC_IMAGER_INSTALL}"
> # wic comes with reasonable defaults, and the proper interface is the wks file
> ROOTFS_EXTRA ?= "0"
>
> @@ -125,32 +129,23 @@ python do_rootfs_wicenv () {
>
> }
>
> -addtask do_rootfs_wicenv after do_rootfs before do_wic_image
> +addtask do_rootfs_wicenv after do_rootfs before do_image_wic
> do_rootfs_wicenv[vardeps] += "${WICVARS}"
> do_rootfs_wicenv[prefuncs] = 'set_image_size'
>
> -WIC_IMAGE_FILE ="${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.img"
> -
> -python check_for_wic_warnings() {
> - with open("{}/log.do_wic_image".format(d.getVar("T"))) as f:
> - for line in f.readlines():
> - if line.startswith("WARNING"):
> - bb.warn(line.strip())
> +check_for_wic_warnings() {
> + WARN="$(grep -e '^WARNING' ${T}/log.do_image_wic || true)"
> + if [ -n "$WARN" ]; then
> + bbwarn "$WARN"
> + fi
> }
>
> -do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
> -do_wic_image[dirs] = "${DEPLOY_DIR_IMAGE}"
> -python do_wic_image() {
> - cmds = ['wic_do_mounts', 'generate_wic_image', 'check_for_wic_warnings']
> - weights = [5, 90, 5]
> - progress_reporter = bb.progress.MultiStageProgressReporter(d, weights)
> -
> - for cmd in cmds:
> - progress_reporter.next_stage()
> - bb.build.exec_func(cmd, d)
> - progress_reporter.finish()
> +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> +IMAGE_CMD_wic() {
> + wic_do_mounts
> + generate_wic_image
> + check_for_wic_warnings
> }
> -addtask wic_image before do_image after do_image_tools
>
> wic_do_mounts() {
> buildchroot_do_mounts
> @@ -209,7 +204,7 @@ generate_wic_image() {
> sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
> find ${BUILDCHROOT_DIR}/${WICTMP} -type f -name "*.direct*" | while read f; do
> suffix=$(basename $f | sed 's/\(.*\)\(\.direct\)\(.*\)/\3/')
> - mv -f ${f} ${WIC_IMAGE_FILE}${suffix}
> + mv -f ${f} "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic${suffix}"
> done
> rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
> rm -rf ${IMAGE_ROOTFS}/../pseudo
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2 3/7] meta: introduce IMAGE_CMD_*
2022-04-14 17:46 ` Anton Mikanovich
@ 2022-04-15 6:55 ` Schmidt, Adriaan
0 siblings, 0 replies; 10+ messages in thread
From: Schmidt, Adriaan @ 2022-04-15 6:55 UTC (permalink / raw)
To: Anton Mikanovich, isar-users
Hi Anton,
Anton Mikanovich, Donnerstag, 14. April 2022 19:47:
> >
> > # glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
> > # should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
> > -ISAR_CROSS_COMPILE_armhf = "1"
> > +python() {
> > + if 'ubifs' in (d.getVar('IMAGE_BASETYPES') or '').split():
> > + d.setVar('ISAR_CROSS_COMPILE_armhf', '1')
> > +}
> >
> This will also not work and actually fails on imx6-sabrelite-bullseye
> target.
> Reasons:
> 1) IMAGE_BASETYPES = "ubi ubi-ubifs" for that target and condition is False
Yes, this is actually a mistake I made. When caclucating IMAGE_BASETYPES,
we need to add IMAGE_TYPEDEPS_xxx recursively. Fixed in v3.
> 2) ISAR_CROSS_COMPILE should be set before anonymous python code because
> buildchroot selection is done also in python() {}
Fixed in v3:
THIS_ISAR_CROSS_COMPILE := "${ISAR_CROSS_COMPILE}"
ISAR_CROSS_COMPILE_armhf = "${@bb.utils.contains('IMAGE_BASETYPES', 'ubifs', '1', '${THIS_ISAR_CROSS_COMPILE}', d)}"
It looks a little ugly because I'm trying to preserve any previous value
of ISAR_CROSS_COMPILE without the variable referencing itself...
With these changes I can now build the imx6-sabrelite-bullseye target, and I
confirmed that ISAR_CROSS_COMPILE is not set when using armhf with image
types != ubifs.
> Also it better to document (or at least comment) IMAGE_BASETYPES as a
> new variable.
For the user manual, I think adding this would be much more detail than
we generally have throughout the document, but I added a comment in v3.
Adriaan
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 4/7] imagetypes: restructure files
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
` (2 preceding siblings ...)
2022-04-05 13:24 ` [PATCH v2 3/7] meta: introduce IMAGE_CMD_* Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 5/7] imagetypes: switch meta-isar to the new scheme Adriaan Schmidt
` (2 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
Instead of per-type classes *-img.bbclass, we move to
- imagetypes.bbclass for the "simple" types and the conversions
- dedicated files for the more complex types:
- imagetypes_wic.bbclass
- imagetypes_vm.bbclass
- imagetypes_contaier.bbclass
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta/classes/cpiogz-img.bbclass | 14 ---
meta/classes/ext4-img.bbclass | 15 ---
meta/classes/fit-img.bbclass | 22 -----
meta/classes/image.bbclass | 50 +++++-----
meta/classes/imagetypes.bbclass | 91 +++++++++++++++++++
...g.bbclass => imagetypes_container.bbclass} | 0
.../{vm-img.bbclass => imagetypes_vm.bbclass} | 0
...wic-img.bbclass => imagetypes_wic.bbclass} | 0
meta/classes/targz-img.bbclass | 8 --
meta/classes/ubi-img.bbclass | 20 ----
meta/classes/ubifs-img.bbclass | 21 -----
11 files changed, 112 insertions(+), 129 deletions(-)
delete mode 100644 meta/classes/cpiogz-img.bbclass
delete mode 100644 meta/classes/ext4-img.bbclass
delete mode 100644 meta/classes/fit-img.bbclass
create mode 100644 meta/classes/imagetypes.bbclass
rename meta/classes/{container-img.bbclass => imagetypes_container.bbclass} (100%)
rename meta/classes/{vm-img.bbclass => imagetypes_vm.bbclass} (100%)
rename meta/classes/{wic-img.bbclass => imagetypes_wic.bbclass} (100%)
delete mode 100644 meta/classes/targz-img.bbclass
delete mode 100644 meta/classes/ubi-img.bbclass
delete mode 100644 meta/classes/ubifs-img.bbclass
diff --git a/meta/classes/cpiogz-img.bbclass b/meta/classes/cpiogz-img.bbclass
deleted file mode 100644
index f4c33bd9..00000000
--- a/meta/classes/cpiogz-img.bbclass
+++ /dev/null
@@ -1,14 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2020 Siemens AG
-#
-# SPDX-License-Identifier: MIT
-
-IMAGER_INSTALL_cpio += "cpio"
-CPIO_IMAGE_FORMAT ?= "newc"
-
-IMAGE_CMD_cpio() {
- ${SUDO_CHROOT} \
- sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
- /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
- ${IMAGE_FILE_CHROOT}"
-}
diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
deleted file mode 100644
index 73d1bb57..00000000
--- a/meta/classes/ext4-img.bbclass
+++ /dev/null
@@ -1,15 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
-
-IMAGER_INSTALL_ext4 += "e2fsprogs"
-
-MKE2FS_ARGS ?= "-t ext4"
-
-# Generate ext4 filesystem image
-IMAGE_CMD_ext4() {
- truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'
-
- ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
- -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
-}
-#IMAGE_CMD_ext4[vardepsexclude] = "ROOTFS_SIZE ROOTFS_EXTRA"
diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass
deleted file mode 100644
index ef65af88..00000000
--- a/meta/classes/fit-img.bbclass
+++ /dev/null
@@ -1,22 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-MKIMAGE_ARGS ??= ""
-
-FIT_IMAGE_SOURCE ??= "fitimage.its"
-
-IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"
-
-# Generate fit image
-IMAGE_CMD_fit() {
- if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
- die "FIT_IMAGE_SOURCE does not contain fitimage source file"
- fi
-
- # Create fit image using buildchroot tools
- ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
- -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
-}
-IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 4413a7d6..dfe7bfa9 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -11,7 +11,6 @@ SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${MACHINE}-${DISTRO}-${DISTRO_ARCH}
IMAGE_INSTALL ?= ""
IMAGE_FSTYPES ?= "${@ d.getVar("IMAGE_TYPE", True) if d.getVar("IMAGE_TYPE", True) else "ext4"}"
-IMAGE_CONVERSIONS = "gz xz"
IMAGE_ROOTFS ?= "${WORKDIR}/rootfs"
KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
@@ -85,6 +84,26 @@ inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
+# Extra space for rootfs in MB
+ROOTFS_EXTRA ?= "64"
+
+def get_rootfs_size(d):
+ import subprocess
+ rootfs_extra = int(d.getVar("ROOTFS_EXTRA", True))
+
+ output = subprocess.check_output(
+ ["sudo", "du", "-xs", "--block-size=1k", d.getVar("IMAGE_ROOTFS", True)]
+ )
+ base_size = int(output.split()[0])
+
+ return base_size + rootfs_extra * 1024
+
+python set_image_size () {
+ rootfs_size = get_rootfs_size(d)
+ d.setVar('ROOTFS_SIZE', str(rootfs_size))
+ d.setVarFlag('ROOTFS_SIZE', 'export', '1')
+}
+
def get_base_type(t, d):
bt = t
for c in d.getVar('IMAGE_CONVERSIONS').split():
@@ -110,18 +129,10 @@ python() {
# image types
IMAGE_CLASSES ??= ""
-IMGCLASSES = "container-img cpiogz-img ext4-img fit-img targz-img ubi-img ubifs-img vm-img wic-img"
+IMGCLASSES = "imagetypes imagetypes_wic imagetypes_vm imagetypes_container"
IMGCLASSES += "${IMAGE_CLASSES}"
inherit ${IMGCLASSES}
-# image conversions
-CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
-CONVERSION_DEPS_gz = "gzip"
-
-XZ_OPTIONS ?= ""
-CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'cat ${IMAGE_FILE_CHROOT} | xz ${XZ_OPTIONS} > ${IMAGE_FILE_CHROOT}.xz'"
-CONVERSION_DEPS_xz = "xz-utils"
-
# hook up IMAGE_CMD_*
python() {
image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
@@ -262,19 +273,6 @@ python() {
d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
}
-# Extra space for rootfs in MB
-ROOTFS_EXTRA ?= "64"
-
-def get_rootfs_size(d):
- import subprocess
- rootfs_extra = int(d.getVar("ROOTFS_EXTRA", True))
-
- output = subprocess.check_output(
- ["sudo", "du", "-xs", "--block-size=1k", d.getVar("IMAGE_ROOTFS", True)]
- )
- base_size = int(output.split()[0])
-
- return base_size + rootfs_extra * 1024
# here we call a command that should describe your whole build system,
# this could be "git describe" or something similar.
@@ -293,12 +291,6 @@ get_build_id() {
fi
}
-python set_image_size () {
- rootfs_size = get_rootfs_size(d)
- d.setVar('ROOTFS_SIZE', str(rootfs_size))
- d.setVarFlag('ROOTFS_SIZE', 'export', '1')
-}
-
ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
image_configure_fstab[weight] = "2"
image_configure_fstab() {
diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass
new file mode 100644
index 00000000..a31a1989
--- /dev/null
+++ b/meta/classes/imagetypes.bbclass
@@ -0,0 +1,91 @@
+# This software is a part of ISAR.
+# Copyright (C) 2021 Siemens AG
+#
+# SPDX-License-Identifier: MIT
+
+#image type: tar
+IMAGER_INSTALL_tar = "tar"
+TAR_OPTIIONS ?= ""
+
+IMAGE_CMD_tar() {
+ ${SUDO_CHROOT} tar ${TAR_OPTIONS} -cvzf \
+ ${IMAGE_FILE_CHROOT} --one-file-system -C ${PP_ROOTFS} .
+}
+
+# image type: ext4
+IMAGER_INSTALL_ext4 += "e2fsprogs"
+MKE2FS_ARGS ?= "-t ext4"
+
+IMAGE_CMD_ext4() {
+ truncate -s ${ROOTFS_SIZE}K '${IMAGE_FILE_HOST}'
+
+ ${SUDO_CHROOT} /sbin/mke2fs ${MKE2FS_ARGS} \
+ -F -d '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
+}
+
+# image type: cpio
+IMAGER_INSTALL_cpio += "cpio"
+CPIO_IMAGE_FORMAT ?= "newc"
+
+IMAGE_CMD_cpio() {
+ ${SUDO_CHROOT} \
+ sh -c "cd ${PP_ROOTFS}; /usr/bin/find . | \
+ /usr/bin/cpio -H ${CPIO_IMAGE_FORMAT} -o > \
+ ${IMAGE_FILE_CHROOT}"
+}
+
+# image type: fit
+MKIMAGE_ARGS ??= ""
+FIT_IMAGE_SOURCE ??= "fitimage.its"
+IMAGER_INSTALL_fit += "u-boot-tools device-tree-compiler"
+
+IMAGE_CMD_fit() {
+ if [ ! -e "${WORKDIR}/${FIT_IMAGE_SOURCE}" ]; then
+ die "FIT_IMAGE_SOURCE does not contain fitimage source file"
+ fi
+
+ ${SUDO_CHROOT} /usr/bin/mkimage ${MKIMAGE_ARGS} \
+ -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${IMAGE_FILE_CHROOT}'
+}
+IMAGE_CMD_fit[depends] = "${PN}:do_transform_template"
+
+# image type: ubifs
+IMAGER_INSTALL_ubifs += "mtd-utils"
+IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
+
+# glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
+# should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
+python() {
+ if 'ubifs' in (d.getVar('IMAGE_BASETYPES') or '').split():
+ d.setVar('ISAR_CROSS_COMPILE_armhf', '1')
+}
+
+IMAGE_CMD_ubifs() {
+ ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
+ -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
+}
+
+# image type: ubi
+IMAGER_INSTALL_ubi += "mtd-utils"
+IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
+UBINIZE_CFG ??= "ubinize.cfg"
+
+IMAGE_CMD_ubi() {
+ if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
+ die "UBINIZE_CFG does not contain ubinize config file."
+ fi
+
+ ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
+ -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
+}
+IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
+
+# image conversions
+IMAGE_CONVERSIONS = "gz xz"
+
+CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
+CONVERSION_DEPS_gz = "gzip"
+
+XZ_OPTIONS ?= ""
+CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'cat ${IMAGE_FILE_CHROOT} | xz ${XZ_OPTIONS} > ${IMAGE_FILE_CHROOT}.xz'"
+CONVERSION_DEPS_xz = "xz-utils"
diff --git a/meta/classes/container-img.bbclass b/meta/classes/imagetypes_container.bbclass
similarity index 100%
rename from meta/classes/container-img.bbclass
rename to meta/classes/imagetypes_container.bbclass
diff --git a/meta/classes/vm-img.bbclass b/meta/classes/imagetypes_vm.bbclass
similarity index 100%
rename from meta/classes/vm-img.bbclass
rename to meta/classes/imagetypes_vm.bbclass
diff --git a/meta/classes/wic-img.bbclass b/meta/classes/imagetypes_wic.bbclass
similarity index 100%
rename from meta/classes/wic-img.bbclass
rename to meta/classes/imagetypes_wic.bbclass
diff --git a/meta/classes/targz-img.bbclass b/meta/classes/targz-img.bbclass
deleted file mode 100644
index 74d34e29..00000000
--- a/meta/classes/targz-img.bbclass
+++ /dev/null
@@ -1,8 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-IMAGE_CMD_tar() {
- sudo tar -cvzf ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.tar.gz --one-file-system -C ${IMAGE_ROOTFS} .
-}
diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass
deleted file mode 100644
index 92acb6f8..00000000
--- a/meta/classes/ubi-img.bbclass
+++ /dev/null
@@ -1,20 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-UBINIZE_CFG ??= "ubinize.cfg"
-
-IMAGER_INSTALL_ubi += "mtd-utils"
-
-# Generate ubi filesystem image
-IMAGE_CMD_ubi() {
- if [ ! -e "${WORKDIR}/${UBINIZE_CFG}" ]; then
- die "UBINIZE_CFG does not contain ubinize config file."
- fi
-
- ${SUDO_CHROOT} /usr/sbin/ubinize ${UBINIZE_ARGS} \
- -o '${IMAGE_FILE_CHROOT}' '${PP_WORK}/${UBINIZE_CFG}'
-}
-IMAGE_CMD_ubi[depends] = "${PN}:do_transform_template"
-IMAGE_CMD_REQUIRED_ARGS_ubi = "UBINIZE_ARGS"
diff --git a/meta/classes/ubifs-img.bbclass b/meta/classes/ubifs-img.bbclass
deleted file mode 100644
index 60c3bf14..00000000
--- a/meta/classes/ubifs-img.bbclass
+++ /dev/null
@@ -1,21 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-IMAGER_INSTALL_ubifs += "mtd-utils"
-
-# glibc bug 23960 https://sourceware.org/bugzilla/show_bug.cgi?id=23960
-# should not use QEMU on armhf target with mkfs.ubifs < v2.1.3
-python() {
- if 'ubifs' in (d.getVar('IMAGE_BASETYPES') or '').split():
- d.setVar('ISAR_CROSS_COMPILE_armhf', '1')
-}
-
-# Generate ubifs filesystem image
-IMAGE_CMD_ubifs() {
- # Create ubifs image using buildchroot tools
- ${SUDO_CHROOT} /usr/sbin/mkfs.ubifs ${MKUBIFS_ARGS} \
- -r '${PP_ROOTFS}' '${IMAGE_FILE_CHROOT}'
-}
-IMAGE_CMD_REQUIRED_ARGS_ubifs = "MKUBIFS_ARGS"
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 5/7] imagetypes: switch meta-isar to the new scheme
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
` (3 preceding siblings ...)
2022-04-05 13:24 ` [PATCH v2 4/7] imagetypes: restructure files Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 6/7] meta-isar: remove IMAGE_FSTYPES from multiconfig definitions Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 7/7] docs: add new imagetypes to user manual Adriaan Schmidt
6 siblings, 0 replies; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
- new image type names, basically replacing wic-img with wic, etc.
- adapt custom image type rpi-sdimg to the new scheme
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta-isar/classes/rpi-sdimg.bbclass | 2 +-
meta-isar/classes/ubi-ubifs-img.bbclass | 11 ++++++++---
meta-isar/conf/machine/bananapi.conf | 2 +-
meta-isar/conf/machine/container-amd64.conf | 2 +-
meta-isar/conf/machine/de0-nano-soc.conf | 2 +-
meta-isar/conf/machine/hikey.conf | 2 +-
meta-isar/conf/machine/imx6-sabrelite.conf | 3 ++-
meta-isar/conf/machine/nanopi-neo.conf | 2 +-
meta-isar/conf/machine/phyboard-mira.conf | 3 ++-
meta-isar/conf/machine/qemuamd64.conf | 2 +-
meta-isar/conf/machine/qemuarm.conf | 2 +-
meta-isar/conf/machine/qemuarm64.conf | 2 +-
meta-isar/conf/machine/qemui386.conf | 2 +-
meta-isar/conf/machine/qemumipsel.conf | 2 +-
meta-isar/conf/machine/qemuriscv64.conf | 2 +-
meta-isar/conf/machine/rpi-common.conf | 2 +-
meta-isar/conf/machine/rpi.conf | 2 +-
meta-isar/conf/machine/sifive-fu540.conf | 2 +-
meta-isar/conf/machine/stm32mp15x.conf | 2 +-
meta-isar/conf/machine/virtualbox.conf | 2 +-
meta-isar/conf/machine/vmware.conf | 2 +-
.../conf/multiconfig/qemuamd64-buster-cpiogz.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf | 2 +-
meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 +-
meta-isar/recipes-core/images/isar-image-ubi.bb | 4 ++--
testsuite/cibase.py | 4 ++--
26 files changed, 37 insertions(+), 30 deletions(-)
diff --git a/meta-isar/classes/rpi-sdimg.bbclass b/meta-isar/classes/rpi-sdimg.bbclass
index 010c573c..508c4bc4 100644
--- a/meta-isar/classes/rpi-sdimg.bbclass
+++ b/meta-isar/classes/rpi-sdimg.bbclass
@@ -3,7 +3,7 @@
#
# Based on SD class from meta-raspberrypi
-inherit wic-img
+IMAGE_TYPEDEP_rpi_sdimg = "wic"
WKS_FILE ?= "rpi-sdimg"
diff --git a/meta-isar/classes/ubi-ubifs-img.bbclass b/meta-isar/classes/ubi-ubifs-img.bbclass
index 095719aa..7a917b47 100644
--- a/meta-isar/classes/ubi-ubifs-img.bbclass
+++ b/meta-isar/classes/ubi-ubifs-img.bbclass
@@ -5,6 +5,11 @@
#
# SPDX-License-Identifier: MIT
-inherit ubi-img ubifs-img fit-img
-addtask do_ubi_image after do_ubifs_image
-addtask do_ubi_image after do_fit_image
+IMAGE_TYPEDEP_ubi_ubifs = "ubi"
+IMAGE_TYPEDEP_ubi += "ubifs fit"
+
+IMAGE_CMD_ubi_ubifs() {
+ # we need to produce output (with extension .ubi-ubifs),
+ # so just create a symlink
+ ln -sf ${IMAGE_FULLNAME}.ubi ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.ubi-ubifs
+}
diff --git a/meta-isar/conf/machine/bananapi.conf b/meta-isar/conf/machine/bananapi.conf
index dd3a0667..36cda28a 100644
--- a/meta-isar/conf/machine/bananapi.conf
+++ b/meta-isar/conf/machine/bananapi.conf
@@ -9,6 +9,6 @@ KERNEL_NAME ?= "armmp"
IMAGE_INSTALL += "u-boot-script"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "bananapi"
IMAGER_INSTALL += "u-boot-sunxi"
diff --git a/meta-isar/conf/machine/container-amd64.conf b/meta-isar/conf/machine/container-amd64.conf
index 1f7669ad..e24729ff 100644
--- a/meta-isar/conf/machine/container-amd64.conf
+++ b/meta-isar/conf/machine/container-amd64.conf
@@ -4,4 +4,4 @@
# SPDX-License-Identifier: MIT
DISTRO_ARCH ?= "amd64"
-IMAGE_FSTYPES ?= "container-img"
+IMAGE_FSTYPES ?= "docker-archive.gz"
diff --git a/meta-isar/conf/machine/de0-nano-soc.conf b/meta-isar/conf/machine/de0-nano-soc.conf
index f822e24b..55343e9f 100644
--- a/meta-isar/conf/machine/de0-nano-soc.conf
+++ b/meta-isar/conf/machine/de0-nano-soc.conf
@@ -11,7 +11,7 @@ PREFERRED_PROVIDER_u-boot-de0-nano-soc = "u-boot-de0-nano-soc"
U_BOOT_CONFIG_de0-nano-soc = "socfpga_de0_nano_soc_defconfig"
U_BOOT_BIN_de0-nano-soc = "u-boot-with-spl.sfp"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "de0-nano-soc.wks.in"
IMAGER_INSTALL += "u-boot-de0-nano-soc"
IMAGER_BUILD_DEPS += "u-boot-de0-nano-soc"
diff --git a/meta-isar/conf/machine/hikey.conf b/meta-isar/conf/machine/hikey.conf
index 615d6749..7f9cccb2 100644
--- a/meta-isar/conf/machine/hikey.conf
+++ b/meta-isar/conf/machine/hikey.conf
@@ -7,7 +7,7 @@ DISTRO_ARCH = "arm64"
KERNEL_NAME ?= "mainline"
-IMAGE_FSTYPES = "wic-img"
+IMAGE_FSTYPES = "wic"
WKS_FILE ?= "hikey"
IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
diff --git a/meta-isar/conf/machine/imx6-sabrelite.conf b/meta-isar/conf/machine/imx6-sabrelite.conf
index 5be337f9..bbc2b505 100644
--- a/meta-isar/conf/machine/imx6-sabrelite.conf
+++ b/meta-isar/conf/machine/imx6-sabrelite.conf
@@ -6,8 +6,9 @@
MACHINE_SERIAL ?= "ttymxc0"
BAUDRATE_TTY ?= "115200"
+INHERIT += "ubi-ubifs-img"
MKUBIFS_ARGS := "-m 0x1000 -e 0x3e000 -c 1500"
UBINIZE_ARGS = "-vv -m 0x1000 -p 0x40000"
-IMAGE_FSTYPES ?= "ubi-ubifs-img"
+IMAGE_FSTYPES ?= "ubi-ubifs"
DTB_FILES = "imx6q-sabrelite.dtb"
diff --git a/meta-isar/conf/machine/nanopi-neo.conf b/meta-isar/conf/machine/nanopi-neo.conf
index 2475ce9d..5aa717e5 100644
--- a/meta-isar/conf/machine/nanopi-neo.conf
+++ b/meta-isar/conf/machine/nanopi-neo.conf
@@ -7,7 +7,7 @@ DISTRO_ARCH ?= "armhf"
KERNEL_NAME ?= "armmp"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "nanopi-neo.wks.in"
IMAGE_INSTALL += "u-boot-script"
diff --git a/meta-isar/conf/machine/phyboard-mira.conf b/meta-isar/conf/machine/phyboard-mira.conf
index 2d052d0c..65382630 100644
--- a/meta-isar/conf/machine/phyboard-mira.conf
+++ b/meta-isar/conf/machine/phyboard-mira.conf
@@ -6,8 +6,9 @@
MACHINE_SERIAL ?= "ttymxc0"
BAUDRATE_TTY ?= "115200"
+INHERIT += "ubi-ubifs-img"
MKUBIFS_ARGS := "-m 0x1000 -e 0x7e000 -c 8083"
UBINIZE_ARGS = "-vv -m 0x1000 -p 0x80000"
-IMAGE_FSTYPES ?= "ubi-ubifs-img"
+IMAGE_FSTYPES ?= "ubi-ubifs"
DTB_FILES = "imx6q-phytec-mira-rdk-nand.dtb"
diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf
index ccfde391..eca2628c 100644
--- a/meta-isar/conf/machine/qemuamd64.conf
+++ b/meta-isar/conf/machine/qemuamd64.conf
@@ -6,7 +6,7 @@ DISTRO_ARCH ?= "amd64"
KERNEL_NAME ?= "amd64"
KERNEL_NAME_ubuntu-focal ?= "generic"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "sdimage-efi"
IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
diff --git a/meta-isar/conf/machine/qemuarm.conf b/meta-isar/conf/machine/qemuarm.conf
index 20a71ea4..06d4b34d 100644
--- a/meta-isar/conf/machine/qemuarm.conf
+++ b/meta-isar/conf/machine/qemuarm.conf
@@ -5,7 +5,7 @@ DISTRO_ARCH ?= "armhf"
KERNEL_NAME ?= "armmp"
-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"
QEMU_ROOTFS_DEV ?= "vda"
diff --git a/meta-isar/conf/machine/qemuarm64.conf b/meta-isar/conf/machine/qemuarm64.conf
index ca23cef5..b30fad47 100644
--- a/meta-isar/conf/machine/qemuarm64.conf
+++ b/meta-isar/conf/machine/qemuarm64.conf
@@ -6,7 +6,7 @@ DISTRO_ARCH ?= "arm64"
KERNEL_NAME ?= "arm64"
KERNEL_NAME_ubuntu-focal ?= "generic"
-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"
QEMU_ROOTFS_DEV ?= "vda"
diff --git a/meta-isar/conf/machine/qemui386.conf b/meta-isar/conf/machine/qemui386.conf
index 17cff1eb..ddfc077b 100644
--- a/meta-isar/conf/machine/qemui386.conf
+++ b/meta-isar/conf/machine/qemui386.conf
@@ -4,7 +4,7 @@ DISTRO_ARCH ?= "i386"
KERNEL_NAME ?= "686-pae"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "directdisk-isar"
IMAGER_INSTALL += "${SYSLINUX_BOOTLOADER_INSTALL}"
diff --git a/meta-isar/conf/machine/qemumipsel.conf b/meta-isar/conf/machine/qemumipsel.conf
index b34e46de..40f5e744 100644
--- a/meta-isar/conf/machine/qemumipsel.conf
+++ b/meta-isar/conf/machine/qemumipsel.conf
@@ -5,7 +5,7 @@ DISTRO_ARCH ?= "mipsel"
KERNEL_NAME ?= "4kc-malta"
-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"
QEMU_ROOTFS_DEV ?= "sda"
diff --git a/meta-isar/conf/machine/qemuriscv64.conf b/meta-isar/conf/machine/qemuriscv64.conf
index 5c687a03..cd04d873 100644
--- a/meta-isar/conf/machine/qemuriscv64.conf
+++ b/meta-isar/conf/machine/qemuriscv64.conf
@@ -8,7 +8,7 @@ DISTRO_ARCH ?= "riscv64"
KERNEL_NAME ?= "riscv64"
-IMAGE_FSTYPES ?= "ext4-img"
+IMAGE_FSTYPES ?= "ext4"
QEMU_ROOTFS_DEV ?= "vda"
diff --git a/meta-isar/conf/machine/rpi-common.conf b/meta-isar/conf/machine/rpi-common.conf
index 8d6dff9e..327ac91b 100644
--- a/meta-isar/conf/machine/rpi-common.conf
+++ b/meta-isar/conf/machine/rpi-common.conf
@@ -5,7 +5,7 @@ BOOT_SPACE ?= "81920"
MACHINE_SERIAL ?= "serial0"
BAUDRATE_TTY ?= "115200"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "rpi-sdimg"
KERNEL_TYPE = "raspios"
diff --git a/meta-isar/conf/machine/rpi.conf b/meta-isar/conf/machine/rpi.conf
index 0ab701c9..917527f0 100644
--- a/meta-isar/conf/machine/rpi.conf
+++ b/meta-isar/conf/machine/rpi.conf
@@ -4,5 +4,5 @@
MACHINE_SERIAL ?= "ttyAMA0"
BAUDRATE_TTY ?= "9600"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "rpi-sdimg"
diff --git a/meta-isar/conf/machine/sifive-fu540.conf b/meta-isar/conf/machine/sifive-fu540.conf
index d73f0475..1eb58db1 100644
--- a/meta-isar/conf/machine/sifive-fu540.conf
+++ b/meta-isar/conf/machine/sifive-fu540.conf
@@ -7,7 +7,7 @@ DISTRO_ARCH = "riscv64"
KERNEL_NAME ?= "mainline"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "sifive-fu540"
IMAGER_INSTALL += "opensbi-sifive-fu540"
IMAGER_BUILD_DEPS += "opensbi-sifive-fu540"
diff --git a/meta-isar/conf/machine/stm32mp15x.conf b/meta-isar/conf/machine/stm32mp15x.conf
index 22e88ea9..62de7dd1 100644
--- a/meta-isar/conf/machine/stm32mp15x.conf
+++ b/meta-isar/conf/machine/stm32mp15x.conf
@@ -11,7 +11,7 @@ PREFERRED_PROVIDER_u-boot-stm32mp15x = "u-boot-stm32mp15x"
U_BOOT_CONFIG_stm32mp15x = "stm32mp15_trusted_defconfig"
U_BOOT_BIN_stm32mp15x = "u-boot.stm32"
-IMAGE_FSTYPES ?= "wic-img"
+IMAGE_FSTYPES ?= "wic"
WKS_FILE ?= "stm32mp15x.wks.in"
IMAGER_INSTALL += "trusted-firmware-a-stm32mp15x optee-os-stm32mp15x u-boot-stm32mp15x"
IMAGER_BUILD_DEPS += "trusted-firmware-a-stm32mp15x optee-os-stm32mp15x u-boot-stm32mp15x"
diff --git a/meta-isar/conf/machine/virtualbox.conf b/meta-isar/conf/machine/virtualbox.conf
index c0f65842..de3d04c3 100644
--- a/meta-isar/conf/machine/virtualbox.conf
+++ b/meta-isar/conf/machine/virtualbox.conf
@@ -12,4 +12,4 @@ WKS_FILE ?= "sdimage-efi"
IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
VMDK_SUBFORMAT = "monolithicSparse"
-IMAGE_FSTYPES ?= "vm-img"
+IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/machine/vmware.conf b/meta-isar/conf/machine/vmware.conf
index 9bf1fd96..fba639b1 100644
--- a/meta-isar/conf/machine/vmware.conf
+++ b/meta-isar/conf/machine/vmware.conf
@@ -14,4 +14,4 @@ IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
OVF_TEMPLATE_FILE ?= "vm-img-vmware.ovf.tmpl"
VMDK_SUBFORMAT = "streamOptimized"
-IMAGE_FSTYPES ?= "vm-img"
+IMAGE_FSTYPES ?= "ova"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
index 37f89581..675d934a 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
@@ -6,4 +6,4 @@
MACHINE ?= "qemuamd64"
DISTRO ?= "debian-buster"
-IMAGE_FSTYPES = "cpiogz-img"
+IMAGE_FSTYPES = "cpio.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
index e3992e60..b48b0a6c 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
@@ -6,4 +6,4 @@
MACHINE ?= "qemuamd64"
DISTRO ?= "debian-buster"
-IMAGE_FSTYPES ?= "targz-img"
+IMAGE_FSTYPES ?= "tar.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
index 5615198d..1765d602 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
@@ -3,4 +3,4 @@
MACHINE ?= "qemuamd64"
DISTRO ?= "debian-buster"
-IMAGE_FSTYPES ?= "wic-img ext4-img"
+IMAGE_FSTYPES ?= "wic ext4"
diff --git a/meta-isar/recipes-core/images/isar-image-ubi.bb b/meta-isar/recipes-core/images/isar-image-ubi.bb
index ef2cf1c3..8c70c1f7 100644
--- a/meta-isar/recipes-core/images/isar-image-ubi.bb
+++ b/meta-isar/recipes-core/images/isar-image-ubi.bb
@@ -23,5 +23,5 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_IMAGE}"
# only one dtb file supported, pick the first
DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"
-UBIFS_IMG = "${PP_DEPLOY}/${UBIFS_IMAGE_FILE}"
-FIT_IMG = "${PP_DEPLOY}/${FIT_IMAGE_FILE}"
+UBIFS_IMG = "${PP_DEPLOY}/${IMAGE_FULLNAME}.ubifs"
+FIT_IMG = "${PP_DEPLOY}/${IMAGE_FULLNAME}.fit"
diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index 0281be41..1dd7c7a0 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -120,7 +120,7 @@ class CIBaseTest(CIBuilder):
['do_bootstrap_setscene', '!do_bootstrap']),
check_executed_tasks('buildchroot-target',
['do_rootfs_install_setscene', '!do_rootfs_install']),
- check_executed_tasks('isar-image-base-*-wic-img',
+ check_executed_tasks('isar-image-base-*',
['do_rootfs_install_setscene', '!do_rootfs_install'])
]):
self.fail("Failed rebuild image")
@@ -163,7 +163,7 @@ class CIBaseTest(CIBuilder):
# TODO: if we actually make a change to hello, then we could test
# that do_rootfs is executed. currently, hello is rebuilt,
# but its sstate sig/hash does not change.
- check_executed_tasks('isar-image-base-*-wic-img',
+ check_executed_tasks('isar-image-base-*',
['do_rootfs_install_setscene', '!do_rootfs_install'])
]):
self.fail("Failed rebuild package and image")
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 6/7] meta-isar: remove IMAGE_FSTYPES from multiconfig definitions
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
` (4 preceding siblings ...)
2022-04-05 13:24 ` [PATCH v2 5/7] imagetypes: switch meta-isar to the new scheme Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
2022-04-05 13:24 ` [PATCH v2 7/7] docs: add new imagetypes to user manual Adriaan Schmidt
6 siblings, 0 replies; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
Now that we have support for building multiple image types using
IMAGE_FSTYPES, doing it via multiconfig is the wrong approach.
This removes the configs that are still using that pattern (buster-cpiogz
and buster-targz), and instead adds the building of cpio and tar
images to the qemuamd64-bullseye config.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
meta-isar/conf/local.conf.sample | 4 ----
.../conf/multiconfig/qemuamd64-bullseye-cpiogz.conf | 7 -------
meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf | 7 -------
meta-isar/conf/multiconfig/qemuamd64-bullseye.conf | 2 ++
meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf | 9 ---------
meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf | 9 ---------
testsuite/citest.py | 2 --
7 files changed, 2 insertions(+), 38 deletions(-)
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
delete mode 100644 meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 71ee8579..58f3e1a2 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -52,11 +52,7 @@ BBMULTICONFIG = " \
qemui386-bookworm \
qemuamd64-stretch \
qemuamd64-buster \
- qemuamd64-buster-tgz \
- qemuamd64-buster-cpiogz \
qemuamd64-bullseye \
- qemuamd64-bullseye-tgz \
- qemuamd64-bullseye-cpiogz \
qemuamd64-bookworm \
container-amd64-stretch \
container-amd64-buster \
diff --git a/meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
deleted file mode 100644
index ff491b2c..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-bullseye-cpiogz.conf
+++ /dev/null
@@ -1,7 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2022 ilbers GmbH
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES = "cpiogz-img"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf b/meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
deleted file mode 100644
index 8eea112e..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-bullseye-tgz.conf
+++ /dev/null
@@ -1,7 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2022 ilbers GmbH
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES ?= "targz-img"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf b/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf
index 1badc889..5c8cfd33 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-bullseye.conf
@@ -2,3 +2,5 @@
MACHINE ?= "qemuamd64"
DISTRO ?= "debian-bullseye"
+
+IMAGE_FSTYPES += "cpio.gz tar.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
deleted file mode 100644
index 675d934a..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-cpiogz.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2020
-#
-# SPDX-License-Identifier: MIT
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES = "cpio.gz"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf b/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
deleted file mode 100644
index b48b0a6c..00000000
--- a/meta-isar/conf/multiconfig/qemuamd64-buster-tgz.conf
+++ /dev/null
@@ -1,9 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
-#
-# SPDX-License-Identifier: MIT
-
-MACHINE ?= "qemuamd64"
-DISTRO ?= "debian-buster"
-
-IMAGE_FSTYPES ?= "tar.gz"
diff --git a/testsuite/citest.py b/testsuite/citest.py
index 8f47338c..f8377f79 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -136,8 +136,6 @@ class NoCrossTest(CIBaseTest):
'mc:qemui386-bullseye:isar-image-base',
'mc:qemuamd64-buster:isar-image-base',
'mc:qemuamd64-bullseye:isar-image-base',
- 'mc:qemuamd64-bullseye-tgz:isar-image-base',
- 'mc:qemuamd64-bullseye-cpiogz:isar-image-base',
'mc:qemuamd64-bullseye:isar-initramfs',
'mc:qemumipsel-buster:isar-image-base',
'mc:qemumipsel-bullseye:isar-image-base',
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 7/7] docs: add new imagetypes to user manual
2022-04-05 13:24 [PATCH v2 0/7] imagetypes Adriaan Schmidt
` (5 preceding siblings ...)
2022-04-05 13:24 ` [PATCH v2 6/7] meta-isar: remove IMAGE_FSTYPES from multiconfig definitions Adriaan Schmidt
@ 2022-04-05 13:24 ` Adriaan Schmidt
6 siblings, 0 replies; 10+ messages in thread
From: Adriaan Schmidt @ 2022-04-05 13:24 UTC (permalink / raw)
To: isar-users; +Cc: Adriaan Schmidt
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
---
doc/user_manual.md | 86 ++++++++++++++++++++++++++++------------------
1 file changed, 53 insertions(+), 33 deletions(-)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index b950ff7f..334e91a1 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -213,7 +213,7 @@ tmp/deploy/images/rpi/isar-image-base-raspbian-stretch-rpi.wic.img
### Generate full disk image
-A bootable disk image is generated if `wic-img` is listed in IMAGE_FSTYPES.
+A bootable disk image is generated if `wic` is listed in IMAGE_FSTYPES.
Behind the scenes a tool called `wic` is used to assemble the images.
It is controlled by a `.wks` file which you can choose with changing WKS_FILE.
Some examples in the tree use that feature already.
@@ -253,7 +253,8 @@ https://github.com/intel/bmap-tools
### Generate container image with root filesystem
A runnable container image is generated if IMAGE_FSTYPES variable includes
-'container-img'.
+one of the supported container formats `oci`, `oci-archive`, `docker-archive`,
+`docker-daemon`, or `containers-storage`.
Getting a container image can be the main purpose of an Isar configuration,
but not only.
A container image created from an Isar configuration meant for bare-metal or
@@ -261,10 +262,9 @@ virtual machines can be helpfull to test certain applications which
requirements (e.g. libraries) can be easily resolved in a containerized
environment.
-Container images can be generated in different formats, selected with the
-variable `CONTAINER_IMAGE_FORMATS`. One or more (whitespace separated) of following
-options can be given:
- - `docker-archive`: (default) an archive containing a Docker image that can
+Container images can be generated in different formats. One or more (whitespace
+separated) of following options can be given:
+ - `docker-archive`: an archive containing a Docker image that can
be imported with [`docker load`](https://docs.docker.com/engine/reference/commandline/load)
- `docker-daemon`: resulting container image is made available on the local
Docker Daemon
@@ -280,12 +280,6 @@ It's technically possible, but requires making host resources (e.g. the
Docker Daemon socket) accessible in the container, which can endanger the
stability and security of the host.
-The resulting container image archives (only for `docker-archive` and
-`oci-archive`) are made available as
-`tmp/deploy/images/${MACHINE}/${DISTRO}-${DISTRO_ARCH}-${container_format}.tar.xz`
-(being `container_format` each one of the formats specified in
-`CONTAINER_IMAGE_FORMATS`).
-
### Example
- Make the relevant environment variables available to the task
@@ -293,9 +287,8 @@ The resulting container image archives (only for `docker-archive` and
For one-shot builds (use `local.conf` otherwise):
```
-export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_IMAGE_FORMATS"
-export IMAGE_FSTYPES="container-img"
-export CONTAINER_IMAGE_FORMATS="docker-archive"
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_FSTYPES"
+export IMAGE_FSTYPES="docker-archive.xz"
```
- Trigger creation of container image from root filesystem
@@ -307,7 +300,7 @@ bitbake mc:qemuarm-buster:isar-image-base
- Load the container image into the Docker Daemon
```
-docker load -i build/tmp/deploy/images/qemuarm/isar-image-base-debian-buster-armhf-1.0-r0-docker-archive.tar.xz
+docker load -i build/tmp/deploy/images/qemuarm/isar-image-base-debian-buster-armhf-1.0-r0.docker-archive.xz
```
- Run a container using the container image (following commands starting with
@@ -460,10 +453,15 @@ Isar can generate various images types for specific machine. The type of the
image to be generated may be specified through the `IMAGE_FSTYPES` variable.
Currently, the following image types are provided:
- - `ext4` - Raw ext4 filesystem image (default option for `qemuarm` machine).
- - `wic-img` - A full disk image with user-specified partitions created and populated using the wic tool.
- - `ubi-img` - A image for use on mtd nand partitions employing UBI
- - `vm-img` - A image for use on VirtualBox or VMware
+ - `tar` - tarball of the root file system
+ - `cpio` - cpio archive
+ - `ext4` - raw ext4 filesystem image (default option for `qemuarm` machine)
+ - `wic` - full disk image with user-specified partitions created and populated using the wic tool
+ - `ubi` - image for use on mtd nand partitions employing UBI
+ - `ova` - Open Virtual Appliance: image for use on VirtualBox or VMware
+
+In addition, image types can be converted using suffixes, e.g. `tar.gz`.
+Available conversions are `gz` and `xz`, which both provide image compression.
There are several image types can be listed in `IMAGE_FSTYPES` divided by space.
@@ -539,7 +537,7 @@ IMAGE_PREINSTALL = "linux-image-rpi-rpfv \
KERNEL_IMAGE = "vmlinuz-4.4.0-1-rpi"
INITRD_IMAGE = "initrd.img-4.4.0-1-rpi"
MACHINE_SERIAL = "ttyAMA0"
-IMAGE_FSTYPES = "wic-img"
+IMAGE_FSTYPES = "wic"
WKS_FILE = "rpi-sdimg"
```
@@ -579,22 +577,44 @@ Every image type in Isar is implemented as a `bitbake` class. The goal of these
### Create Custom Image Type
-As already mentioned, Isar uses `bitbake`to accomplish the work. The whole build process is a sequence of tasks. This sequence is generated using task dependencies, so the next task in chain requires completion of previous ones.
-The last task of image recipe is `do_populate`, so the class that implement new image type should continue execution from this point. According to the BitBake syntax, this can be implemented as follows:
+The following steps are required to implement a custom image type:
Create a new class:
```
$ vim meta-user/classes/my-image.bbclass
```
-Add these lines:
+
+Specify the command to generate the new image, and optionally image type
+dependencies or required arguments:
```
-do_my_image() {
+IMAGE_TYPEDEPS_my_image = "ext4"
+IMAGE_CMD_REQUIRED_ARGS_my_image = "MY_ARG"
+IMAGE_CMD_my_image() {
+ INPUT="${PP_DEPLOY}/${IMAGE_FULLNAME}.ext4"
+ ${SUDO_CHROOT} my_command ${MY_ARG }-i ${INPUT} -o ${IMAGE_FILE_CHROOT}
}
-addtask my_image before do_build after do_populate
```
-The content of `do_my_image` function can be implemented either in shell or in Python.
+The IMAGE_CMD is a shell function, and the environment has some pre-set
+variables:
+
+ - `IMAGE_FILE_HOST` and `IMAGE_FILE_CHROOT` are the paths of the output image
+ (including extension) in the host or buildchroot.
+ - `SUDO_CHROOT` is a prefix you can use to have a command run inside the
+ buildchroot.
+
+If the code you provide in `IMAGE_CMD` requires the building and/or installation
+of additional packages in the buildchroot, you can specify this:
+```
+IMAGER_BULID_DEPS_my_image = "my_command"
+IMAGER_INSTALL_my_image = "my_command"
+```
+
+To use your custom image class, add it to `IMAGE_CLASSES` in your machine config:
+```
+IMAGE_CLASSES += "my-image"
+```
-In the machine configuration file, set the following:
+And finally select the new image type:
```
IMAGE_FSTYPES = "my-image"
```
@@ -603,11 +623,11 @@ IMAGE_FSTYPES = "my-image"
Isar contains additional image type classes that can be used as reference:
- - `ext4-img`
- - `targz-img`
- - `ubifs-img`
- - `ubi-img`
- - `wic-img`
+ - `ext4`
+ - `tar.gz`
+ - `ubifs`
+ - `ubi`
+ - `wic`
---
--
2.30.2
^ permalink raw reply [flat|nested] 10+ messages in thread