From: Zhihang Wei <wzh@ilbers.de>
To: Felix Moessbauer <felix.moessbauer@siemens.com>,
isar-users@googlegroups.com
Cc: clara.kowalsky@siemens.com
Subject: Re: [PATCH 1/1] Revert "image: remove unrequested intermediate image types"
Date: Thu, 19 Nov 2026 18:06:24 +0100 [thread overview]
Message-ID: <d511d684-34b4-4acd-bad7-ae74d3d9ce54@ilbers.de> (raw)
In-Reply-To: <20261118130243.2042081-1-felix.moessbauer@siemens.com>
Applied to next, thanks.
Zhihang
On 11/18/26 14:02, Felix Moessbauer wrote:
> This reverts commit 6393523290ee4bf108dc7b0a4337720dbf4602c8.
>
> While the intention of the commit is fine, the current implementation
> has some serious drawbacks:
>
> - it does not remove additional image artifacts (like wic.bmap,
> wic.spdx.json, ...) and there is no way to register these
> - it breaks partial rebuilds of a derived image type (e.g. a wic ->
> squashfs dependency cannot rebuild just the wic type as the squashfs
> was removed in the meantime).
>
> Both aspects cannot be easily fixes implementation wise, as we don't
> track who deploys what. The proper way to do it would be to deploy via
> the sstate cache and let the cache handle the deletion of the
> artifacts, but this requires a major re-design. Given that, we just
> revert this optimization for now.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> meta/classes-recipe/image.bbclass | 24 ++---
> .../unittests/test_image_type_cleanup.py | 92 -------------------
> 2 files changed, 12 insertions(+), 104 deletions(-)
> delete mode 100644 testsuite/unittests/test_image_type_cleanup.py
>
> diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
> index 229a37fe..ce19ae1c 100644
> --- a/meta/classes-recipe/image.bbclass
> +++ b/meta/classes-recipe/image.bbclass
> @@ -187,8 +187,6 @@ SUDO_CHROOT = "imager_run -d ${PP_ROOTFS} -u root --"
> # hook up IMAGE_CMD_*
> python() {
> image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
> - requested_types = set(image_types)
> - intermediate_files = set()
> conversions = set(d.getVar('IMAGE_CONVERSIONS').split())
> chroot_mode = d.getVar('ISAR_CHROOT_MODE')
>
> @@ -287,10 +285,8 @@ python() {
>
> # add conversions
> conversion_depends = set()
> + rm_images = set()
> def create_conversions(t):
> - localdata.setVar('type', t)
> - if t not in requested_types:
> - intermediate_files.add(localdata.getVar('IMAGE_FILE_HOST'))
> for c in sorted(conversions):
> if t.endswith('.' + c):
> t = t[:-len(c) - 1]
> @@ -305,9 +301,20 @@ python() {
> for dep in (localdata.getVar('CONVERSION_DEPS:' + c) or '').split():
> conversion_install.add(dep)
> local_conversion_install.add(dep)
> + # remove temporary image files
> + if t not in image_types:
> + rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
> +
> for t in basetypes[bt]:
> create_conversions(t)
>
> + if bt not in image_types:
> + localdata.setVar('type', t)
> + rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
> +
> + for image in rm_images:
> + cmds.append('\trm ' + image)
> +
> # image type dependencies
> after = 'do_image_tools'
> for dep in typedeps[bt]:
> @@ -333,7 +340,6 @@ python() {
> d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install)))
> d.appendVarFlag(task, 'vardeps', ' INSTALL_image_%s' % bt_clean)
>
> - d.setVar('IMAGE_INTERMEDIATE_FILES', ' '.join(sorted(intermediate_files)))
> d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
> d.appendVar('IMAGER_BOM', ' ' + ' '.join(sorted(imager_bom)))
> d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
> @@ -423,12 +429,6 @@ python do_image() {
> """Virtual task"""
> pass
> }
> -python cleanup_image_intermediates() {
> - for image in (d.getVar('IMAGE_INTERMEDIATE_FILES') or '').split():
> - bb.utils.remove(image)
> -}
> -do_image[postfuncs] += "cleanup_image_intermediates"
> -do_image[vardeps] += "IMAGE_INTERMEDIATE_FILES"
> addtask image before do_build after do_image_tools
>
> python do_deploy() {
> diff --git a/testsuite/unittests/test_image_type_cleanup.py b/testsuite/unittests/test_image_type_cleanup.py
> deleted file mode 100644
> index 1e2ad5b9..00000000
> --- a/testsuite/unittests/test_image_type_cleanup.py
> +++ /dev/null
> @@ -1,92 +0,0 @@
> -# This software is a part of Isar.
> -# Copyright (C) Siemens AG, 2026
> -#
> -# SPDX-License-Identifier: MIT
> -
> -import sys
> -import tempfile
> -import unittest
> -from pathlib import Path
> -
> -root = Path(__file__).resolve().parents[2]
> -sys.path.extend([
> - str(root / 'bitbake/lib'),
> - str(root / 'meta/lib'),
> -])
> -
> -import bb.data
> -from bb.build import exec_func
> -from bb.parse import handle
> -
> -
> -data = bb.data.init()
> -data.setVar('BBPATH', f'{root}/meta:{root}/bitbake')
> -data.setVar('__bbclasstype', 'recipe')
> -data.setVar('IMGCLASSES', '')
> -parsed = handle(str(root / 'meta/classes-recipe/image.bbclass'), data)
> -data.setVar('IMAGE_TYPEDEP:ova', 'vmdk')
> -data.setVar('IMAGE_TYPEDEP:vmdk', 'wic')
> -data.setVar('IMAGE_CMD:vmdk', ':')
> -namespace = {'bb': bb}
> -exec(parsed['get_base_type'], namespace)
> -image_tasks = next(
> - function for function in parsed.getVar('__BBANONFUNCS', False)
> - if 'def collect_image_type(' in parsed.getVar(function, False)
> -)
> -exec(parsed[image_tasks], namespace)
> -
> -
> -class TestImageTypeCleanup(unittest.TestCase):
> - def generate_image_tasks(self, image_types, deploy_dir='/deploy'):
> - d = bb.data.createCopy(data)
> - d.setVar('IMAGE_FSTYPES', image_types)
> - d.setVar('DEPLOY_DIR_IMAGE', str(deploy_dir))
> - d.setVar('IMAGE_FULLNAME', 'image')
> - namespace[image_tasks](d)
> - return d
> -
> - def assert_intermediates(self, image_types, expected):
> - d = self.generate_image_tasks(image_types)
> - self.assertEqual(
> - set(d.getVar('IMAGE_INTERMEDIATE_FILES').split()),
> - {'/deploy/image.' + image_type for image_type in expected},
> - )
> -
> - def test_only_unrequested_types_are_intermediates(self):
> - self.assert_intermediates('ova.zst', {'ova', 'vmdk', 'wic'})
> - self.assert_intermediates('vmdk ova.zst', {'ova', 'wic'})
> - self.assert_intermediates('ova ova.zst vmdk wic', set())
> -
> - def test_unrequested_conversion_stages_are_cleaned(self):
> - self.assert_intermediates('tar.gz.xz', {'tar', 'tar.gz'})
> - self.assert_intermediates('tar.gz tar.gz.xz', {'tar'})
> -
> - def test_cleanup_uses_image_type_filename_overrides(self):
> - d = bb.data.createCopy(data)
> - d.setVar('IMAGE_FSTYPES', 'tar.gz')
> - d.setVar('IMAGE_FULLNAME:tar', 'custom-image')
> - d.setVar('DEPLOY_DIR_IMAGE:tar', '/custom-deploy')
> - namespace[image_tasks](d)
> - self.assertEqual(
> - d.getVar('IMAGE_INTERMEDIATE_FILES'),
> - '/custom-deploy/custom-image.tar',
> - )
> -
> - def test_cleanup_removes_only_intermediate_files(self):
> - with tempfile.TemporaryDirectory() as deploy_dir:
> - deploy_dir = Path(deploy_dir)
> - for image_type in ('ova', 'wic', 'vmdk', 'ova.zst'):
> - (deploy_dir / ('image.' + image_type)).touch()
> -
> - d = self.generate_image_tasks('vmdk ova.zst', deploy_dir)
> - d.setVar('T', str(deploy_dir / 'temp'))
> - exec_func('cleanup_image_intermediates', d)
> -
> - self.assertEqual(
> - {path.name for path in deploy_dir.glob('image.*')},
> - {'image.vmdk', 'image.ova.zst'},
> - )
> -
> -
> -if __name__ == '__main__':
> - unittest.main()
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/d511d684-34b4-4acd-bad7-ae74d3d9ce54%40ilbers.de.
prev parent reply other threads:[~2026-11-19 17:08 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 13:02 'Felix Moessbauer' via isar-users
2026-11-19 17:06 ` Zhihang Wei [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d511d684-34b4-4acd-bad7-ae74d3d9ce54@ilbers.de \
--to=wzh@ilbers.de \
--cc=clara.kowalsky@siemens.com \
--cc=felix.moessbauer@siemens.com \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox