From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7071655505424285696 X-Received: by 2002:a05:6512:dc:b0:448:2624:aa81 with SMTP id c28-20020a05651200dc00b004482624aa81mr6624422lfp.137.1646662188648; Mon, 07 Mar 2022 06:09:48 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:a95:b0:448:1eeb:4c26 with SMTP id m21-20020a0565120a9500b004481eeb4c26ls606455lfu.0.gmail; Mon, 07 Mar 2022 06:09:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJzYUy7Hb6yktfa+BxOH01u3fBSZl8QjhuXbaax728ign6vKNXRkbFTAdpQlgetk/No2/vm0 X-Received: by 2002:ac2:434a:0:b0:443:e48d:50b7 with SMTP id o10-20020ac2434a000000b00443e48d50b7mr7982063lfl.45.1646662187638; Mon, 07 Mar 2022 06:09:47 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1646662187; cv=pass; d=google.com; s=arc-20160816; b=eH68E0iQwH246UivAx2Z0uDrHiplSY+Dz0aogY/F1TSFNjdw04De3eM9YcIYUWQiiU 1O0/lN1ig1KXDZQxM9V1o5/pi4BTl2paFUFmFRjQ0+cnXOBRSgtiArKBsMhxnSOCp9jq 93qD6zLKjbjefQxZ/xRrjRXJH5u+f9zkcdwuhvDzRoDHF4BYHkWK6/5zpk1Ke0g71gAy yyi6Hr3IgMHajf61r1qiIYIYJGezZ42qttuprb/lTPGbV4QweAjYoce3Hu/VGD4UencO ApyN1RwnSqiHuDuBo2bxUAD+o6lYtyOW8UFECD/322Zdpa+gRKiKRYofEA/uRyWGrTXa +NsQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=AhiBjjFBBiKIZ5PIikITqGu86do37VoqzbHSj8o5WwU=; b=Y8mmmr0JkND86NXA4HSOw5mSCKteGvvSLsVgwK6sc4Mtz8D2trvwsJLG84crHuGq46 GD031BuyNbD2al08FjrO93fimwvKvY5aX8K16MPelZx4nZT4syZEqIYVDll4WKS8cevk ecxD4L9Zk1vq+bjz0HKSj62N7vhIPyc9QBmVVZ4GAODk2mwMpYDsKFoLHO8VVZBBFNAE eMjcBF3TDSXWKsq2YwAJ/2FoGiA2tvrw5i0itlVb2+yWJNkR4IaWMcJwHnJwFmWBq/C4 8fMQq93FHboCjE4fK2eyS9yP+z8kZGWk8mnWVnD4hDM3MrfIOfVUkrDcN1A5zNDLVAYW ePmQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=Z5AMggMh; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::610 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on20610.outbound.protection.outlook.com. [2a01:111:f400:7e1b::610]) by gmr-mx.google.com with ESMTPS id j17-20020a056512345100b0044567a67e24si583990lfr.3.2022.03.07.06.09.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Mar 2022 06:09:47 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::610 as permitted sender) client-ip=2a01:111:f400:7e1b::610; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=Z5AMggMh; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::610 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z8ipalomBAL6Ae4KvWBcuf8bgu750bTKF6yUs91lcCoe89vpzO1JiPG4osFQ9unQ0pSP7B8v7tmFCVBk2w1ZL90tALsGcVKJ74q3IJaOX5uYuOYi/BCz92BguSyESx7G7YKfrZyU8t3si0dqc/f4flEDjhk3qM3PcIdZfX3svNzjOXOI3y1zfxOZchmaVI5x3/9eV6Y0FyHUrfncBmG3juMQTH8a2AfXcUQaKbX5m1KEujqxT4z4EN3MaILtS6q1hOkdnEbkKDceUMOISh0HfDHoR5hdG+X7JQlfLvQ93DckUNSlePuT2Ssq0wLnxLNVDPQ771PxIpxIgjFlFz2NNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AhiBjjFBBiKIZ5PIikITqGu86do37VoqzbHSj8o5WwU=; b=RN3qiacGR6zvL2HrMTok+iBsJwnLAfPiT7QKLFTmHpzOOudiBs6O/aS3L573a57oz+E5AM3d7iSJjsxCE6WfyTbW92+8VvMSDdb5O0tzAbq5Oe1xCWUb7y/f2OIp30C0URuXEQx+MhrcahK+W1EOKyy4+gz8n8jKAogE27KAOsYS1FJvTgNcNzgq6D2tTuRc04cfVAEa8agZw4RGSYz8LiMG75Sbns6ZKjh3H26qaInAju1w4fTKHygLTOfEQELYVdSXIV6r8ukiZkmVvlUh8gN0lcNgTZndjQrD0kYM+Hb4cbXc7pt0houB2rEAF6WxY0PEZmtlHXYroLVmZFNfMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AhiBjjFBBiKIZ5PIikITqGu86do37VoqzbHSj8o5WwU=; b=Z5AMggMhM58VhM2tKOzH1/k6u4Y0zAFTaXJJbU0CHpYxuzxQPBmbUVXGNpykfQuytFiLbZvbPPdZBrqokVfPRJ6rBCwuI1+UDOFQFf45ryx7H0FSPI8g8OSnuyQzDIRjfUJWKXruFw3aWmhSYWg2jUDMYmRb7gDA0vALUQdH42YWzoGXdLKbEj8XI9VQt+V2JxmARfkXUxnP8K+HoYyKvsvhqILfQnCd4ysI4gV6I/LykFinF8MLEB+mNvQFishej9sksjpF4BfU+mw+7TEybP5P7uuufQ+ZTvLkoSfiYMM7Fax1BzX9FsuSyWI5NmfpoFbU8pS4ZVN1VEp6Xtd4og== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM7PR10MB3462.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:13d::17) by VI1PR10MB3984.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:800:13d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.13; Mon, 7 Mar 2022 14:09:45 +0000 Received: from AM7PR10MB3462.EURPRD10.PROD.OUTLOOK.COM ([fe80::4855:c766:bd0e:ab46]) by AM7PR10MB3462.EURPRD10.PROD.OUTLOOK.COM ([fe80::4855:c766:bd0e:ab46%7]) with mapi id 15.20.5038.026; Mon, 7 Mar 2022 14:09:45 +0000 Date: Mon, 7 Mar 2022 15:09:41 +0100 From: Henning Schild To: Jan Kiszka Cc: isar-users Subject: Re: [PATCH 3/3] vm-img: Handle sparse files efficiently Message-ID: <20220307150941.046b852f@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20220307100403.3ac04025@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FRYP281CA0003.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::13) To AM7PR10MB3462.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:13d::17) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 25274e28-3bad-489b-ed21-08da004421ea X-MS-TrafficTypeDiagnostic: VI1PR10MB3984:EE_ X-LD-Processed: 38ae3bcd-9579-4fd4-adda-b42e1495d55a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7HDJFHC0W0CSiGTW9agweLSg9mKUiJD51gWruc96CNBPKylkSfQcSB7CU3N+NU66VmUv5jE6x4ybvMBJJEt1aRL+SeP8Eokv1/maBc/K+MSF9fPR/1Nssf1oKz2qPOlntJrsYGUiiqRBRwAas67VmJO5zKot4fOvXkZs5wfdUEHYN9vTca8UeENXGkhkx4I+mrrQWGEuWGP15SRws4W2YWclLu+6Fi7K0oChuk6vpRQ8afGX9qsMjtmXU+ycZy8xFcG5HkcSJu9oK42feJCuQeryNPXmzWcy+vSX759Hol135vHrAWAlZEXqjjUCPU1iSu2UbrrJmwb9HunA0I9eKXzDlB/2OP8aP0xPvVt0o6SLs/Cys6v/U5vCTizY8KKBCLS3XZLjoa7IQWqQXOC9sd9YBREbpeR4Vs+Xp+6oDoDZMNq56Fh7A1DILmBnde3lB9kXtKiiowkgjI4Jt/NKo5Pvs9y4dJ7LbslNBITwKKJ/GrkNHB2DrPhCGYNR9qp/GWwy8GlW7hK07IAjUMqYXk7A+aAWf1ejYi95S/e+Nao1btQksrCg7hQPnrZ8mP4XkkZpuGIFnxl2tHC7wAk/PyJ/NPbjsrsJhHnxdhNUXEMN4JPCO7DLzsJtmfT8pXPuKw33dZFmkO7ot6YoX2ycuw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM7PR10MB3462.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(66556008)(316002)(66946007)(6636002)(66476007)(6862004)(4326008)(8676002)(38100700002)(86362001)(44832011)(8936002)(82960400001)(2906002)(5660300002)(9686003)(6512007)(186003)(26005)(1076003)(6486002)(508600001)(53546011)(6506007)(6666004)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?hbQ8SVLcRmD1KKRc8WWrdh8hnAwEru8cQWZpUri6zQ4/+EoEq+mNHcvrGa1g?= =?us-ascii?Q?TbmIi/yg+8ztn1J/SKDeY0r3v8Hb+YeaHouY4dGMVqhYKnIv2s3fAfdpHihD?= =?us-ascii?Q?OhnsDAvoBtZbCMWv9OzlzWoGv0G9jG1wulHicdZc55KCkh7AKGUqqv9b7HxV?= =?us-ascii?Q?SRcQiTKectj+wehZO0LYsSqVpDHQQwnhdQyINl29gJA1hH99P5C+6zkRNX2u?= =?us-ascii?Q?NkuoV5TrhAf9AnCYst5WcqPSmZhPAVi2HkY+UA+0qBeURTcVaTYP0LJBSG1D?= =?us-ascii?Q?srJAE6naS2ghJd7f/U/JjG0QzqAqu/kfMQBdwXHcM3ji+d2fm+VRsSnPPUMK?= =?us-ascii?Q?F88uHti4m2SZxHmHyLqtNHo/GGSQNvKf5xBEr2qlCA/0UFy1Eu+KkM3H8zsR?= =?us-ascii?Q?dTJSX2fh4eXptdx/cR9U5DMAZtN7ZRyA3iW0WNDuLFNu52eG7HF6ymJs+I4g?= =?us-ascii?Q?RJtGUhDVIGcTJ66aymsz1h0sdPgWySf+60HaJzPvikgnSRf8OvOBQiTc/W6n?= =?us-ascii?Q?TGFK3KANesiT3mlMa55i3YlWa+ymVT23e7Ky6eehHmJfGx/q/XpH/abjebpb?= =?us-ascii?Q?/x1rcLTAD+LYc8reGq35XR6c/Fgqi+0fAsPcDjFTgIC10+UdLfStsdW0rOBf?= =?us-ascii?Q?pANSszlKBlA/K7pkCpsNt8uz+7iaJJLFY7OBJXWDJ6i+FRh4MR3HCx3DcaBl?= =?us-ascii?Q?as6HHLBbw7hyF+lpsZUhysCKSHvYi/hJOvmpRTZGIRxBC9iDKnJIYwAPytaA?= =?us-ascii?Q?00dJ40ko7mgXOVxATnspuITlSZJZSnKSJ1hxK8IRJBbibNj5n+/CcmWDaC0f?= =?us-ascii?Q?Nz6ryUsaldm+bnHb55Ie5QvbzVZZSQm8oUImWDvii39hTP00phoWrjb3KuaU?= =?us-ascii?Q?cI6O2G2uy2vUZjWGQw9c2vtn4NiavxIUoi+8p8r3KOUFjfu1G5tGKrkrr7Lj?= =?us-ascii?Q?Bdwm2vYpWR4BZyF6nr1FjHwPRTqwqqwn5Ykwg3NBZ8cf2sYbj0Ua4ml7WoVa?= =?us-ascii?Q?vT3srv7JKFJCZKQYpvPWnlxlKx1y3hfgvREHT4W2anUb1WqKd5nMNi78emu5?= =?us-ascii?Q?6qLL9HZNMWhLTqQ+tY/DauyxCYp8UH2mBAKpFIjqdTlZLAaZc2C56ne0hFo5?= =?us-ascii?Q?2rFQoc3XO74TB12DKtha5mjev/bOFrgBVQXOcS9QJxktRz0MnArki0DzKGGw?= =?us-ascii?Q?gr7JM8QdwY4u5392jWNvurEgBEtJLAuhCXnXdl8u5jEqdQ4zBKQjGiXuZZNm?= =?us-ascii?Q?KTCAhMG0pol7N244hc0Uow/eU9EkPoFjOxkVnIf+a0LLe0hG2UYWub8JkNq4?= =?us-ascii?Q?kNaNogYyGCjOBNT/EbsSloFUdhxdB7mQfv7tWCERmEI/8MeUf81IesY2G7We?= =?us-ascii?Q?4OZ0djeRNE+PX39uKV1a71Kx1sMNGSXLneXpC5QVl6q8kBLDMBb6LfgNvc9c?= =?us-ascii?Q?oykw1tj2wQM9HJjWdBZ9Fr86RNu4M51OWzKDkfknhhUupbsN0Yi7elUZIGTD?= =?us-ascii?Q?onxB1MHgtOxGxft6kq6oZUNX+yadXwNLb5ClNy2NO7PRPZC0U/cgwy5HLI6P?= =?us-ascii?Q?z7l0Ku1/yuuC8TIPQR5t6fIzunZ4ct8y4SSUe/CEtT10D7OqWaFMrXW5NdPa?= =?us-ascii?Q?dBl5UZ3EkdjCDwQqFZdZSH/iqysmKNEAe3zZMBQ0C/EjPk0muQtr2Z0KYH8z?= =?us-ascii?Q?cuLlhkh0oekn0fvHDYlYM4XKuUY=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 25274e28-3bad-489b-ed21-08da004421ea X-MS-Exchange-CrossTenant-AuthSource: AM7PR10MB3462.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2022 14:09:45.4416 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kpVyIWSVTrjDkLWwAxP2ryMF7vhzVBPRB065ASWoJFxKf99W9/uKJBNVAAehoEvsRmTidc6t2EZMyPfJ23TpsiXRFAjJOrbY6XKhGmUEKtk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR10MB3984 X-TUID: XfMXNz+NT8rD Am Mon, 7 Mar 2022 10:46:34 +0100 schrieb Jan Kiszka : > On 07.03.22 10:04, Henning Schild wrote: > > Am Sat, 5 Mar 2022 17:34:50 +0100 > > schrieb Jan Kiszka : > > > >> From: Jan Kiszka > >> > >> Saves space in case the image contains any sparse files. > >> > >> Signed-off-by: Jan Kiszka > >> --- > >> meta/classes/vm-img.bbclass | 6 +++--- > >> 1 file changed, 3 insertions(+), 3 deletions(-) > >> > >> diff --git a/meta/classes/vm-img.bbclass > >> b/meta/classes/vm-img.bbclass index 4bc977b9..27aedf12 100644 > >> --- a/meta/classes/vm-img.bbclass > >> +++ b/meta/classes/vm-img.bbclass > >> @@ -93,15 +93,15 @@ do_create_ova() { > >> export VM_UUID=$(uuidgen) > >> # create ovf > >> cat ${PP_WORK}/${OVF_TEMPLATE_STAGE2} | envsubst > > >> ${PP_DEPLOY}/${OVA_NAME}.ovf > >> - tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > >> ${OVA_NAME}.ovf > >> + tar -cvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > >> ${OVA_NAME}.ovf > >> # VirtualBox needs here a manifest file. VMware does > >> accept that format. if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" > >> ]; then > > > > Not sure that already suggested it but we did not do it. > > > >> echo > >> "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum > >> ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> > >> ${PP_DEPLOY}/${OVA_NAME}.mf echo > >> "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum > >> ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> > >> ${PP_DEPLOY}/${OVA_NAME}.mf > >> - tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > >> ${OVA_NAME}.mf > >> + tar -uvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > >> ${OVA_NAME}.mf fi > >> - tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > >> ${VIRTUAL_MACHINE_IMAGE_FILE} > >> + tar -uvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} > >> ${VIRTUAL_MACHINE_IMAGE_FILE} EOSUDO > > > > Did you get around to testing this on vmware and vbox? Do you think > > it would need to be tested? I could to that. > > > > That would be valuable. Just add a sparse file to the image (dd > if=/dev/zero of=sparse.file bs=1G seek=1 count=0) and check if that > bloats the files. And, of course, check if things still boot. NACK, breaks ova import into both vmware and virtualbox. I compared and it is really this patch which breaks both. Did not check the vmware logs but vbox says VERR_TAR_UNSUPPORTED_GNU_HDR_TYPE So this tarball does not support it because its consumers both do not. Henning > Thanks, > Jan >