From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 11 Nov 2026 17:09:19 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f188.google.com (mail-lj1-f188.google.com [209.85.208.188]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 6ABG9I9L007078 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 11 Nov 2026 17:09:18 +0100 Received: by mail-lj1-f188.google.com with SMTP id 38308e7fff4ca-3adf57a6f70sf16870451fa.3 for ; Wed, 11 Nov 2026 08:09:18 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1794413353; cv=pass; d=google.com; s=arc-20260327; b=p/vVLscO9I+WvnLVRpWWaTf3OqU9Gu9Ti7I2TsXa8eHv5YRPI5r8S9kP+jvu46Ypyc 7R24QwjG3LTmWzhgaid3iF5mn1XqTjmp0mx7plLiEJ+ylUKAKMZ+ztfcpo2M5J6O5pDX XKR4liQ312qrkuw7z+HCdD7i0ynAO22Q2B+ztM6YCY8TZ36UtDX+zASwLVMhizNrynsr psqyepcU8e3A94V2nxPcJE0JwpzWpW0AtkcAqnK5tKSMQyZCBoBIUk3toLoZFPSj0tlk t4TFlM89XINZtG6vlwtsw+M24nQKu4CaLIGS3GQIsP96MrFQ7gC+DA9l29jgN8YzN9/T XMDg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :sender:dkim-signature; bh=cTqyJopNe+km9kc/gOIViURnVAnpIqCqlH6bnF5jWKE=; fh=kFA+IeUl3KJCYaIuvVxfpyHIv67+LZ7NBaswYuimsnw=; b=lc2/cT3qFHIzlQn7KViXRsFTPijGMAkQh98WZQwCL1DduSEj0UaDwnr9MHyqpv9IL2 hLe+bJuwHJ6rrARjUN4U6fbwfEN/y5AeZBahxB3J2GHN5J9bYf5zj5hFhCb2qcSNpr5I W5ZfnkVxv5ej8ATc3WtmBJAk+E0f35UfYnrDTAxGwaqstNCyv5MM7CpHzCAojPSNpKyt z5k2cxNa+UdOtw/L4tj8q+uhfuYmViGOIyXNr5dVX/LU7pqn7NsN6whWOIn9XWOiGUY7 CqQj6WiWJMlB6J6mO9zK7IAmM7ELdmlo90vSwBcUiSSW6u7mfY2h2Nl0NEJzdPtYGU9f B1Jg==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20260707; t=1794413353; x=1795018153; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:content-type:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to:content-type; bh=cTqyJopNe+km9kc/gOIViURnVAnpIqCqlH6bnF5jWKE=; b=pTx5asyYF/A9NYMueBhIExwO0q1k9g17JvRxeOu2rcE5z5nDOs1ZM7jQ3Z2HI6C70u 89Yb5yha317M2hyrmB14o2TM3qOy0pbzC/VgYchAUay670+Dfq0Vq9TlFceb2m5CggLI xq+tE9smK1H1W8yp1smsVhl65sjOGiaeCG40lcDidQNqJC1G57KsShBDbYoedU24A00W n0c2koA9vlpGNlSnO/zL3EqPYhqVjsMoLW4YcmNNJ0j8/N5tYQsjUvS9XZO5YXzZZ9ZF XgyHpiYi5AugxBew0FLpedCeSh3HLP8KYptZHZCP0H9eixYLc2C3rRmskG4B6JkDPrXv wrFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794413353; x=1795018153; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:content-type :in-reply-to:from:content-language:references:to:subject:user-agent :mime-version:date:message-id:x-beenthere:x-gm-message-state:sender :from:to:cc:subject:date:message-id:reply-to:content-type; bh=cTqyJopNe+km9kc/gOIViURnVAnpIqCqlH6bnF5jWKE=; b=BvDWe1nYuHGGFvyDgZ8kOB3H/u0S9AMcJiTbKKgEdE9/5ou5ZnYuu36t/qYnv12VU8 Puj/1e6JLOPl7E0HAimzyc38KLIZWRCEknu0SoAKr1R6DJsIWSDJftFa05tM2TsqIFOp 4sA4wQy1agoNyajmK61g8Qv2/DOq01h68LkBrFzWX8uzBWS2H48y5JTOq6mfNvMzsTPk 6SGsjtXbImMNMo5a4xXfwknhquF/VZHnx5uHMf/xfCbWu/yMyHdojo7o45a1i090HkyH qia4gyAsE5iRLqiwK3rUw+6ip8J9m/oq2Mc/lUqgUvacFpr+9DSemn5bb5X/d0+Ud26v Rcpw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AL+hErFX4V5akkxUCB3DqHlvjnCNXuLXolbOzxE2w5WnrB7LDyGHsVYBknWgAWbmTK6T/gyj3sbc@ilbers.de X-Gm-Message-State: APuU4Z+ip8w9O53JlBjbbugmPdpjixS+SLhgcuEE6JGsVqJr8C81Ingk FxpKvbdfR8svszzayZt+sK7UWwPXH6HSU8lnxvOVJs62lhXaveE+uup5 X-Received: by 2002:a05:6512:124a:b0:5c1:4c4b:6b2d with SMTP id 2adb3069b0e04-5c15d653806mr912791e87.58.1794413352568; Wed, 11 Nov 2026 08:09:12 -0800 (PST) X-BeenThere: isar-users@googlegroups.com; h="ATskLdesJgUsf+EoDaE3VlsfusHKs37079C8152jXL7uK6oSIQ==" Received: by 2002:a05:6512:1c4:b0:5c1:4dbf:6251 with SMTP id 2adb3069b0e04-5c15cf0bdefls376218e87.1.-pod-prod-07-eu; Wed, 11 Nov 2026 08:09:10 -0800 (PST) X-Forwarded-Encrypted: i=2; AL+hErGJjcDpaQp85bUkh6zV1+54MYoAk/EaNg6VYJorEpq8Mtu7Eldb9Go2MGzVoTou9n1c0E51I0B1leru@googlegroups.com X-Received: by 2002:a05:6512:3ca6:b0:5c1:5a83:2fd7 with SMTP id 2adb3069b0e04-5c15d669496mr879097e87.66.1794413350026; Wed, 11 Nov 2026 08:09:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1794413350; cv=none; d=google.com; s=arc-20260327; b=nTR21JqdpSIuOomXcpJGJi3Ckvf+26XTLqUiPFaFLEeMzVg3pF/jG7rMGP2URbKHgP 5JS3md1gGOmMKiP25qV+nMAb+sHWs78YtD+6dsZHwGwh3ncthJax3zH76u4Ps24d5DWJ ndo16YVTA6yj88B105dFgNFlU7hGj+CX7jEN4EKX32LBvEwdyBRz9vTPYNadE3g1qYa3 ozQZZJ2NUiq/LSBQTrHtMb0tLyY6lvdnzt1f0u7TsUx8IFGHyEYnJnFuAg/4VxrOaZug DUdbVeRtwLh421pf6uKcoiLv4jYXXhXU20YhbBrZWTTuy5AXDBfaAzBKWnr37Th4gwbl Q7Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id; bh=r+cG72dyk7tGAIfw0RM2tN7EzT5OLm5bKVK+Yve9JBk=; fh=lOecfPjbysiGzlRBef36l/WlHWoMC0vHaB+pO6uJ+To=; b=P5yB+VrdGTR9mB6HTbD/U6DQrVYNTUHZH/tXU9zu3Nno0/4DudiB9rhNauG4GiovTC 1YiQ7ZQITUohQR7pKkeGJZEJ7WOUYMv1MiHEKoVERm6BddOuKrWmys/IGcapKF+0h/Dn NUqZuKucoAKvY8foEgnvoJV2INz6svfk46MgPq2DUn8uOXbWDNwDV2oT8vhgKB1809fQ zh6Cc/GFzCfQuywutQNUkrkk7BHPql5pMkqBcpHGj0LpJq/aWTOJ75/RlIlO1vCGfF6e gR5/ehIiIIFwA5myWfmdaY5MzVjPmbrV5e54Qm9F0W+V5PLFzyUuvaKWyPQ4PGox2XPB Dzsw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-5c157bb7b8esi77025e87.8.2026.11.11.08.09.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 11 Nov 2026 08:09:09 -0800 (PST) Received-SPF: pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from [192.168.178.24] (dslb-188-101-247-215.188.101.pools.vodafone-ip.de [188.101.247.215]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 6ABG78Cp006712 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 11 Nov 2026 17:07:08 +0100 Message-ID: <2f98ffb2-506b-4cd2-a020-75f3d5ee0bef@ilbers.de> Date: Wed, 11 Nov 2026 17:07:08 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] image: remove unrequested intermediate image types To: Clara Kowalsky , isar-users@googlegroups.com References: <20261104072600.3229817-1-clara.kowalsky@siemens.com> Content-Language: en-US From: Zhihang Wei In-Reply-To: <20261104072600.3229817-1-clara.kowalsky@siemens.com> Content-Type: text/plain; charset="UTF-8"; format=flowed X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: wzh@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: IhL+YSOYpJ4z Applied to next, thanks. Zhihang On 11/4/26 08:25, 'Clara Kowalsky' via isar-users wrote: > Images built only as dependencies currently remain in the deploy > directory even when they are not listed in IMAGE_FSTYPES. > Fix this by keeping only explicitly requested image outputs and removing > dependency-only images and conversion intermediates. Users who need a > dependency image retained should include it in IMAGE_FSTYPES. > > Since dependencies are now also removed, run cleanup after all image > tasks finish rather than at the end of each producing task. This keeps > dependency images available until their consumers have completed. > Expanded file paths are recorded, so cleanup also handles type-specific > filenames and deploy directories. > Unittests are added to test dependency cleanup and retention of > requested outputs. > > Signed-off-by: Clara Kowalsky > --- > meta/classes-recipe/image.bbclass | 24 ++--- > .../unittests/test_image_type_cleanup.py | 92 +++++++++++++++++++ > 2 files changed, 104 insertions(+), 12 deletions(-) > create mode 100644 testsuite/unittests/test_image_type_cleanup.py > > diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass > index ce19ae1c..229a37fe 100644 > --- a/meta/classes-recipe/image.bbclass > +++ b/meta/classes-recipe/image.bbclass > @@ -187,6 +187,8 @@ SUDO_CHROOT = "imager_run -d ${PP_ROOTFS} -u root --" > # hook up IMAGE_CMD_* > python() { > image_types = (d.getVar('IMAGE_FSTYPES') or '').split() > + requested_types = set(image_types) > + intermediate_files = set() > conversions = set(d.getVar('IMAGE_CONVERSIONS').split()) > chroot_mode = d.getVar('ISAR_CHROOT_MODE') > > @@ -285,8 +287,10 @@ python() { > > # add conversions > conversion_depends = set() > - rm_images = set() > def create_conversions(t): > + localdata.setVar('type', t) > + if t not in requested_types: > + intermediate_files.add(localdata.getVar('IMAGE_FILE_HOST')) > for c in sorted(conversions): > if t.endswith('.' + c): > t = t[:-len(c) - 1] > @@ -301,20 +305,9 @@ python() { > for dep in (localdata.getVar('CONVERSION_DEPS:' + c) or '').split(): > conversion_install.add(dep) > local_conversion_install.add(dep) > - # remove temporary image files > - if t not in image_types: > - rm_images.add(localdata.expand('${IMAGE_FILE_HOST}')) > - > for t in basetypes[bt]: > create_conversions(t) > > - if bt not in image_types: > - localdata.setVar('type', t) > - rm_images.add(localdata.expand('${IMAGE_FILE_HOST}')) > - > - for image in rm_images: > - cmds.append('\trm ' + image) > - > # image type dependencies > after = 'do_image_tools' > for dep in typedeps[bt]: > @@ -340,6 +333,7 @@ python() { > d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install))) > d.appendVarFlag(task, 'vardeps', ' INSTALL_image_%s' % bt_clean) > > + d.setVar('IMAGE_INTERMEDIATE_FILES', ' '.join(sorted(intermediate_files))) > d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install))) > d.appendVar('IMAGER_BOM', ' ' + ' '.join(sorted(imager_bom))) > d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps))) > @@ -429,6 +423,12 @@ python do_image() { > """Virtual task""" > pass > } > +python cleanup_image_intermediates() { > + for image in (d.getVar('IMAGE_INTERMEDIATE_FILES') or '').split(): > + bb.utils.remove(image) > +} > +do_image[postfuncs] += "cleanup_image_intermediates" > +do_image[vardeps] += "IMAGE_INTERMEDIATE_FILES" > addtask image before do_build after do_image_tools > > python do_deploy() { > diff --git a/testsuite/unittests/test_image_type_cleanup.py b/testsuite/unittests/test_image_type_cleanup.py > new file mode 100644 > index 00000000..1e2ad5b9 > --- /dev/null > +++ b/testsuite/unittests/test_image_type_cleanup.py > @@ -0,0 +1,92 @@ > +# This software is a part of Isar. > +# Copyright (C) Siemens AG, 2026 > +# > +# SPDX-License-Identifier: MIT > + > +import sys > +import tempfile > +import unittest > +from pathlib import Path > + > +root = Path(__file__).resolve().parents[2] > +sys.path.extend([ > + str(root / 'bitbake/lib'), > + str(root / 'meta/lib'), > +]) > + > +import bb.data > +from bb.build import exec_func > +from bb.parse import handle > + > + > +data = bb.data.init() > +data.setVar('BBPATH', f'{root}/meta:{root}/bitbake') > +data.setVar('__bbclasstype', 'recipe') > +data.setVar('IMGCLASSES', '') > +parsed = handle(str(root / 'meta/classes-recipe/image.bbclass'), data) > +data.setVar('IMAGE_TYPEDEP:ova', 'vmdk') > +data.setVar('IMAGE_TYPEDEP:vmdk', 'wic') > +data.setVar('IMAGE_CMD:vmdk', ':') > +namespace = {'bb': bb} > +exec(parsed['get_base_type'], namespace) > +image_tasks = next( > + function for function in parsed.getVar('__BBANONFUNCS', False) > + if 'def collect_image_type(' in parsed.getVar(function, False) > +) > +exec(parsed[image_tasks], namespace) > + > + > +class TestImageTypeCleanup(unittest.TestCase): > + def generate_image_tasks(self, image_types, deploy_dir='/deploy'): > + d = bb.data.createCopy(data) > + d.setVar('IMAGE_FSTYPES', image_types) > + d.setVar('DEPLOY_DIR_IMAGE', str(deploy_dir)) > + d.setVar('IMAGE_FULLNAME', 'image') > + namespace[image_tasks](d) > + return d > + > + def assert_intermediates(self, image_types, expected): > + d = self.generate_image_tasks(image_types) > + self.assertEqual( > + set(d.getVar('IMAGE_INTERMEDIATE_FILES').split()), > + {'/deploy/image.' + image_type for image_type in expected}, > + ) > + > + def test_only_unrequested_types_are_intermediates(self): > + self.assert_intermediates('ova.zst', {'ova', 'vmdk', 'wic'}) > + self.assert_intermediates('vmdk ova.zst', {'ova', 'wic'}) > + self.assert_intermediates('ova ova.zst vmdk wic', set()) > + > + def test_unrequested_conversion_stages_are_cleaned(self): > + self.assert_intermediates('tar.gz.xz', {'tar', 'tar.gz'}) > + self.assert_intermediates('tar.gz tar.gz.xz', {'tar'}) > + > + def test_cleanup_uses_image_type_filename_overrides(self): > + d = bb.data.createCopy(data) > + d.setVar('IMAGE_FSTYPES', 'tar.gz') > + d.setVar('IMAGE_FULLNAME:tar', 'custom-image') > + d.setVar('DEPLOY_DIR_IMAGE:tar', '/custom-deploy') > + namespace[image_tasks](d) > + self.assertEqual( > + d.getVar('IMAGE_INTERMEDIATE_FILES'), > + '/custom-deploy/custom-image.tar', > + ) > + > + def test_cleanup_removes_only_intermediate_files(self): > + with tempfile.TemporaryDirectory() as deploy_dir: > + deploy_dir = Path(deploy_dir) > + for image_type in ('ova', 'wic', 'vmdk', 'ova.zst'): > + (deploy_dir / ('image.' + image_type)).touch() > + > + d = self.generate_image_tasks('vmdk ova.zst', deploy_dir) > + d.setVar('T', str(deploy_dir / 'temp')) > + exec_func('cleanup_image_intermediates', d) > + > + self.assertEqual( > + {path.name for path in deploy_dir.glob('image.*')}, > + {'image.vmdk', 'image.ova.zst'}, > + ) > + > + > +if __name__ == '__main__': > + unittest.main() -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/2f98ffb2-506b-4cd2-a020-75f3d5ee0bef%40ilbers.de.