From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7071655505424285696 X-Received: by 2002:a2e:1544:0:b0:247:dce4:681 with SMTP id 4-20020a2e1544000000b00247dce40681mr6579706ljv.430.1646664068701; Mon, 07 Mar 2022 06:41:08 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:6a08:0:b0:247:e311:8788 with SMTP id f8-20020a2e6a08000000b00247e3118788ls605282ljc.6.gmail; Mon, 07 Mar 2022 06:41:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJyRwpjAWjiDTn96WmRZLRBlPw4oQUb+NAtiyxzLb0qZ/sB7xN78lAn3iBTKij4zCoA7c1lQ X-Received: by 2002:a05:651c:1403:b0:246:34e0:bc16 with SMTP id u3-20020a05651c140300b0024634e0bc16mr7753722lje.98.1646664067505; Mon, 07 Mar 2022 06:41:07 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1646664067; cv=pass; d=google.com; s=arc-20160816; b=bgUez+/py51cmwUCmpkqdEX/NDULGimi7ml1kbv9w/c2/iDps+7N3zILVqBaeJ2VG7 R+5JDLPyWmmwANMNT1CHQlUL7291IhcfgF9fXxIjgV8UBbb9GE5kCB9UQzC0XlxC7UBd /ikbX41qWF0DmPrQ84li5MKejAq6nAOP0TQNgM+UFIo5O+04Ac6tVCCAcZzwyeHyEaxK We7Hv5EsbVEV/V2PPgLNgj/b6VvKggi2dQsQ/It9CMLnrHaxZm4b9AqNoZ/tMWPL8jWe e80DlbrmPw7l1X07ZNae6BoZkyeQvyYLnXFuGD+FbEObI88CMqY6m3halpnGlZifFo3B 9r7Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :dkim-signature; bh=19wkLj0xdrhW1c8gA3jQM4Z54CCz7K++pyYUTaUVMFo=; b=pZfUH9AxlOA+hCJqjE4dGvyap6MtUqEYxxMUejDNG7Wo56QVu4ysm/MSkxohYetYLp +pozLnFT7DW9YkDzuoujrXja6CaZxhdvUg+M9ZkoXPsm1bbCyNJ/3loo3we5VDIXrH4a KuQ4/wQF6VOg6J4GLzz4XS0yU6MSBpKHQINpymFKuJwzBIn18SOfcZ+vT2TwBQZ9cpg8 bjBI9TYFimGZAZ8/z/x4tU98UPkGebXR1AGhzy2RdABk8XTuXgBS8K+Pb/EdloT4GNju 2JhIlf+/2qncs3wrJ2Nnt2/j8ZGpnzSOey/ehV8vzH52jR0elRmPRppCrx0AZ/oiCZJI npEA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=SnNrbBW5; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 2a01:111:f400:fe08::60f as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-am5eur03on060f.outbound.protection.outlook.com. [2a01:111:f400:fe08::60f]) by gmr-mx.google.com with ESMTPS id j17-20020a056512345100b0044567a67e24si588286lfr.3.2022.03.07.06.41.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Mar 2022 06:41:07 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 2a01:111:f400:fe08::60f as permitted sender) client-ip=2a01:111:f400:fe08::60f; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=SnNrbBW5; arc=pass (i=1 spf=pass spfdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of jan.kiszka@siemens.com designates 2a01:111:f400:fe08::60f as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DTi/abPrticY1ovzmbQMy7u98yotGIdmGZds9LR6dQDpW24N40V5fQemBx5XJiCIYnWMCKItwyfdRw5cwfdMlq/opR7hK44lW2yHD/zt2XIQiEE4XNcih59KgeSocwH9cEGUI6phyM7Uz/LAHlQw4+FojTRsSJ/0CNASwWb9wWE/5bMzSypG/2eZiiAcxHpKQuglUI14vzA3kdzm2OSZXgEkgZKG6bFoJYN3bPafi9x6EmaJ7z84TMj87t+l7O/gRLvIMdFE0WZQsOy6T7kIF5vbl8eifaUtcxpD8t3Lx0/FdI3e+WvU6FP4TecW8vaoZ4EkN905sSgkK60pCqrzig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=19wkLj0xdrhW1c8gA3jQM4Z54CCz7K++pyYUTaUVMFo=; b=hFqugg2c9h6GaHdegPfM4Squ9i0Y9jtezTm0hfOnj1sBdpNpfvL55jDaxXKlgPSyrtEGdFRDJDGVm8YFhw9+dObV+RVkjqdjCxnoUBPWzjtWjX7NC8ZOw65Mw00XWcpkkDqdU1Dldm8Mcl/O0Vo2We1NFGyou2RyDvktojBQeda/r/fp5rYzhzq9gXSZUsJVCIQT1qJa4ofRYegUNC158UBqmB7+i+riTehQu7rLiS28h6RC4PGlLGar6ShBPOu1b23ljkzWED4vHY6+vOb/nAFrqTvljjvGYWXFkXOCvFiTVpxnMup8hXOu+fHh0ZgXid05GsFGJMdDtxjaXMSkew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 194.138.21.72) smtp.rcpttodomain=googlegroups.com smtp.mailfrom=siemens.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=siemens.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=19wkLj0xdrhW1c8gA3jQM4Z54CCz7K++pyYUTaUVMFo=; b=SnNrbBW5QDqRDQgqOcmgEo8uwukESGENmTDY1EsLa+rxeFn1KEs27rhXZgUwxgp+oAQjynwcwrhPWZLZoMsEvF9CCiACYHV/Vqm440Vnl+wf0l7VA2BU6hD/vFt3M7jNKkXmud4uyZ1Djv6fljm8W0aMOalbsB3a/nDb3yOpwJ2kuakcBQm1RF/1hB4uw1CZ++F3/PIpc/9jku+BivwJcIJsKbxJyfd/LmBUbwMmF1p8bvsdpGjdDFZoir/yROnRNPrnVv1EXjyWmZ1lRDyWPrUCkUKbwTFx6CqnNvb7qbUhnbx+XdfOC6R0O0LwXDTsXm4RImjZ2j9CO6VLs0+gzw== Received: from AM6P192CA0029.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::42) by AM0PR10MB2916.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:165::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.14; Mon, 7 Mar 2022 14:41:06 +0000 Received: from VE1EUR01FT063.eop-EUR01.prod.protection.outlook.com (2603:10a6:209:83:cafe::36) by AM6P192CA0029.outlook.office365.com (2603:10a6:209:83::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.14 via Frontend Transport; Mon, 7 Mar 2022 14:41:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 194.138.21.72) smtp.mailfrom=siemens.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=siemens.com; Received-SPF: Pass (protection.outlook.com: domain of siemens.com designates 194.138.21.72 as permitted sender) receiver=protection.outlook.com; client-ip=194.138.21.72; helo=hybrid.siemens.com; Received: from hybrid.siemens.com (194.138.21.72) by VE1EUR01FT063.mail.protection.outlook.com (10.152.2.179) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5038.14 via Frontend Transport; Mon, 7 Mar 2022 14:41:05 +0000 Received: from DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) by DEMCHDC9SMA.ad011.siemens.net (194.138.21.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Mon, 7 Mar 2022 15:41:05 +0100 Received: from [167.87.0.7] (167.87.0.7) by DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Mon, 7 Mar 2022 15:41:04 +0100 Message-ID: Date: Mon, 7 Mar 2022 15:41:04 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 Subject: Re: [PATCH 3/3] vm-img: Handle sparse files efficiently Content-Language: en-US To: Henning Schild CC: isar-users References: <20220307100403.3ac04025@md1za8fc.ad001.siemens.net> <20220307150941.046b852f@md1za8fc.ad001.siemens.net> From: Jan Kiszka In-Reply-To: <20220307150941.046b852f@md1za8fc.ad001.siemens.net> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-Path: jan.kiszka@siemens.com X-Originating-IP: [167.87.0.7] X-ClientProxiedBy: DEMCHDC8A1A.ad011.siemens.net (139.25.226.107) To DEMCHDC8A0A.ad011.siemens.net (139.25.226.106) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9f5c584a-046a-49e5-a5df-08da004882bc X-MS-TrafficTypeDiagnostic: AM0PR10MB2916:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: n/gtYhV2K5l6eeHN/moViKR61k91OE+y3KU5dnuJSZJ88y2drm06mGxXxMh9LX/AKdIY3YGR/n0BY1Zy5Pyr1xgWjBhN4LXmUq2rs4FVQO38wMHXw2WOsGfiGVrTzCa4+VPdtc0Ex6SA93jX0iY+Lo8mJOdDRrqs4HNxsV1cg6u6AQFogHhvnM7+eePBUTQwfzsfCAQagWVK0bekkvVozR2sSK/oO+zFyiFl0XYa9mWq9l5A4eqhY4I0aB8vosg3CUvSaATpsnkIp6nG6nZWpdXAubUgvogieH8MI3z53vNqEY46K9Byxf6ouvvsazmd0XSICG6etWjwAfRRZO0nhpMqBpOtB2XejuvV0bTEvwgz37qwHFn5fDwniV4TJWjQl1MA/To1KJ00PSiHa/KRGyJOniDeWFth5y7zFirY3IbFEXMjgd3v3Csd/mUj+TtbLjkxuSGJiS9KrTutTBmk3wl13t/LLVtLSRbWCPdUj+HDmYZbHz7P+f1P9RPDB+7acazF/qralOD8pPc+mb0WfLFAOWnS5nocpfLL8oDxNN0pjexdR0jnByz9ZcF6k2F3GytH1TaouO+oQh0qqPw0Q7cdRNJP76fQ7lypsXeRTP92oP/WvRn2Z4Rgp3RT7c58eLWwoxz7SFklyO/T1zf8uQou1XCscsPua73HKlLEnL/187fjpt6K5EdhbvgFwoILGV/9fykpkiEWIyl0cIqyJldzQuCh73V2HbTK+uqLQtaHVj9lZFcGBlrhqwW3V4Qr5ve8Pm36X3RBKZmmrqXlgw== X-Forefront-Antispam-Report: CIP:194.138.21.72;CTRY:DE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:hybrid.siemens.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(82960400001)(31686004)(508600001)(356005)(86362001)(36860700001)(81166007)(6706004)(6636002)(37006003)(36756003)(44832011)(82310400004)(70206006)(70586007)(4326008)(316002)(47076005)(16576012)(6862004)(8676002)(26005)(186003)(16526019)(8936002)(83380400001)(53546011)(31696002)(40460700003)(956004)(336012)(5660300002)(2616005)(2906002)(3940600001)(43740500002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2022 14:41:05.6924 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9f5c584a-046a-49e5-a5df-08da004882bc X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=38ae3bcd-9579-4fd4-adda-b42e1495d55a;Ip=[194.138.21.72];Helo=[hybrid.siemens.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR01FT063.eop-EUR01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR10MB2916 X-TUID: r7jEa77NtMcV On 07.03.22 15:09, Henning Schild wrote: > Am Mon, 7 Mar 2022 10:46:34 +0100 > schrieb Jan Kiszka : > >> On 07.03.22 10:04, Henning Schild wrote: >>> Am Sat, 5 Mar 2022 17:34:50 +0100 >>> schrieb Jan Kiszka : >>> >>>> From: Jan Kiszka >>>> >>>> Saves space in case the image contains any sparse files. >>>> >>>> Signed-off-by: Jan Kiszka >>>> --- >>>> meta/classes/vm-img.bbclass | 6 +++--- >>>> 1 file changed, 3 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/meta/classes/vm-img.bbclass >>>> b/meta/classes/vm-img.bbclass index 4bc977b9..27aedf12 100644 >>>> --- a/meta/classes/vm-img.bbclass >>>> +++ b/meta/classes/vm-img.bbclass >>>> @@ -93,15 +93,15 @@ do_create_ova() { >>>> export VM_UUID=$(uuidgen) >>>> # create ovf >>>> cat ${PP_WORK}/${OVF_TEMPLATE_STAGE2} | envsubst > >>>> ${PP_DEPLOY}/${OVA_NAME}.ovf >>>> - tar -cvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} >>>> ${OVA_NAME}.ovf >>>> + tar -cvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} >>>> ${OVA_NAME}.ovf >>>> # VirtualBox needs here a manifest file. VMware does >>>> accept that format. if [ "${VMDK_SUBFORMAT}" = "monolithicSparse" >>>> ]; then >>> >>> Not sure that already suggested it but we did not do it. >>> >>>> echo >>>> "SHA${OVA_SHA_ALG}(${VIRTUAL_MACHINE_IMAGE_FILE})=$(sha${OVA_SHA_ALG}sum >>>> ${PP_DEPLOY}/${VIRTUAL_MACHINE_IMAGE_FILE} | cut -d' ' -f1)" >> >>>> ${PP_DEPLOY}/${OVA_NAME}.mf echo >>>> "SHA${OVA_SHA_ALG}(${OVA_NAME}.ovf)=$(sha${OVA_SHA_ALG}sum >>>> ${PP_DEPLOY}/${OVA_NAME}.ovf | cut -d' ' -f1)" >> >>>> ${PP_DEPLOY}/${OVA_NAME}.mf >>>> - tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} >>>> ${OVA_NAME}.mf >>>> + tar -uvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} >>>> ${OVA_NAME}.mf fi >>>> - tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} >>>> ${VIRTUAL_MACHINE_IMAGE_FILE} >>>> + tar -uvSf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} >>>> ${VIRTUAL_MACHINE_IMAGE_FILE} EOSUDO >>> >>> Did you get around to testing this on vmware and vbox? Do you think >>> it would need to be tested? I could to that. >>> >> >> That would be valuable. Just add a sparse file to the image (dd >> if=/dev/zero of=sparse.file bs=1G seek=1 count=0) and check if that >> bloats the files. And, of course, check if things still boot. > > NACK, breaks ova import into both vmware and virtualbox. I compared and > it is really this patch which breaks both. > > Did not check the vmware logs but vbox says > VERR_TAR_UNSUPPORTED_GNU_HDR_TYPE Amazingly primitive format, this OVO... > > So this tarball does not support it because its consumers both do not. > OK, thanks for testing. We may then consider whether to take patch 2 as well as it may surprise constrained consumers. Patch 1 is fine as it only targets Isar itself. Jan -- Siemens AG, Technology Competence Center Embedded Linux