From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7127570847942836224 X-Received: by 2002:a17:907:a0c6:b0:73d:bb3c:328b with SMTP id hw6-20020a170907a0c600b0073dbb3c328bmr16828031ejc.354.1661873909963; Tue, 30 Aug 2022 08:38:29 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:bc51:b0:73d:afe3:ffd9 with SMTP id s17-20020a170906bc5100b0073dafe3ffd9ls1198499ejv.10.-pod-prod-gmail; Tue, 30 Aug 2022 08:38:28 -0700 (PDT) X-Google-Smtp-Source: AA6agR4zs09V8PA3Srv6RyxC96EkXKiWwAnUWsx85BZmL9krEbVq69hETN8VBoSC4ER8iFmHLpTX X-Received: by 2002:a17:906:5a5a:b0:741:559f:f539 with SMTP id my26-20020a1709065a5a00b00741559ff539mr10251754ejc.3.1661873908652; Tue, 30 Aug 2022 08:38:28 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1661873908; cv=pass; d=google.com; s=arc-20160816; b=jeT/FghFJNIItJSjrKHudRbmGqIKZ6GHAwpqvAelKg4RSLeN/4A/KiERZebFf4mK0g 2vS2fqrhJHRpy3lfcZeYR8qlsqj8dfYIwD6wMQdjTHrT2XYSwo2LacyXB4kewKwomsJU Td1MV8Uq7gHvFoLgSSBQzKDRIatYdbdD+R4zpJYQb8XKVJbqC+pts4SztF6mIZfe67Fi sfuv0DUh3RFduI4Gkkt1RDXTbiay4DFhDpvHJoHj5cu94fKR9uC2MgQYROxv5RWf7PV0 bVT+RsnB3Rt6BjFfuBNKCj/CzBNXZxpamcR6xs2eH3sULqkVVsAaaE+dBrVgNvxskjJk 4JMw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:document_confidentiality :msip_labels:content-language:accept-language:in-reply-to:references :message-id:date:thread-index:thread-topic:subject:cc:to:from :dkim-signature; bh=MpZAT+sGCsiILLkI8HfBbOcX4GkHFWMfd2iWwXLUdnY=; b=AdwtErgc0QQ85w3vYGkZVy3rdbgSn6TQoC89u8ZcoaRpDch48YFa3j/ojoQOSGoTLK PYAh18fkU/Bz0mbKNRxIPiYIg6PG131UPxmdi869nrm3jFGokdhlsh3CfwqF4JWb8IJm abznGYv9h7oNJDg3UQWd8sZItZb2N7ZiBikYtcjrLMSszJXksCDCq+l/DiS5UbMUxynD ZeG0jlbBD+0zkfABb20GB7jJwUy0P0PUaDmxvTm/CRjLB2hKDtgxV+40UrYWCkZvC3qD I6FMv6b3o+dKWYGRluuh7XU0yHkI+3kHV53g7EWka/vOBVRv1YIo5YmDDIaXfeoWI5Rh ja0Q== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=N5Z1bBqM; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of felix.moessbauer@siemens.com designates 40.107.20.67 as permitted sender) smtp.mailfrom=felix.moessbauer@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2067.outbound.protection.outlook.com. [40.107.20.67]) by gmr-mx.google.com with ESMTPS id gv36-20020a1709072be400b007420a3f34c3si165269ejc.0.2022.08.30.08.38.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Aug 2022 08:38:28 -0700 (PDT) Received-SPF: pass (google.com: domain of felix.moessbauer@siemens.com designates 40.107.20.67 as permitted sender) client-ip=40.107.20.67; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=N5Z1bBqM; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of felix.moessbauer@siemens.com designates 40.107.20.67 as permitted sender) smtp.mailfrom=felix.moessbauer@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hvWTrYiRZDvcf+VaThqhQNddBB2bmDR+Y5hpRXfN75B+ngjMOMfa81pAYiWmSMnsV8QDGDwYN7V/XXhZC+mldG7WZ4UMsjMiUP7ah9oguC1/Yz0Dq8d0VYj2vtw+bdtzoCo2ct4YkV5Yv4Smjc805qN5hg1XAQ7kKpTWy5BiOzVB4N5dp6QRjcpZvYJeKBmAUxfR0KuVoLEvKGLkQldZWVzpChIKpwFdzCWwAFyAyeBk5fRUPZApEl/KDYTVgVd+sfxdlTY9HzvAiX/ivYb4h7IMKKwmBGcdDGzO77Rm9FvdYuS7K7cwHuDoSFT/DZwelN3pG7cB8j9HfnWopo/9GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MpZAT+sGCsiILLkI8HfBbOcX4GkHFWMfd2iWwXLUdnY=; b=HzWIk6aUhQezeunzq56XALHgwcYOt+gmJ+nR/LNBmRKGYtajnPCjeJ4tANj1dz2fNwj2M6DmHTnzwE3u9Knzu8osSN3p834UyCL49siMiXcQIztdc/PiVfztnwjtjZzLgosBGgqqH2tvFJDIS1KHIzdnW0y4/1I1gujh0/+Wz55e8AlUwx9kE+ZQPme67TC81+7OSMOssgJI+qyVTQu4XCoBcnJGD6KoO0rDl48Rea1LGFxKCieMzjw3JemrqZgL8LhaD4OtGhaLOqpVPJRKg7gUeLkCs49kR3d09m76mC9GCtYxSLL+V66zyHXTUK9zC5Sl/+MmR+XU/IkzQpOfvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MpZAT+sGCsiILLkI8HfBbOcX4GkHFWMfd2iWwXLUdnY=; b=N5Z1bBqMB0Ty7FXQ/zG8qAm18AvWlVXYP6RinSznF/BTCIAaxZUh3SxVbatIPM/AWfzznRItHQQA9JbzpIBzSa9SpyenC68MBsH/dAHE45o/Dy6pT0P4+nVwr13Ndp9N/5lZn7yNFVs2UJ7Oh6gOfoYfZGShgCiB/XQnpAvPDx5lF8qWsTCJP5aB75Yhy7qooFG3i/PGABinW0oGBJa6P8ld1beOLWrtHtafBqG+jRZ33ayHpG7LlYdBzGGcSlzZDg2aS0wxcLu6C2DTJNkFdZ7Ma/L39U0JaliFofkUH8c3F1Fcx3IHcYj2VyA01jUlqBolp9/5Pz/vWlC+uDISwA== Received: from AS8PR10MB4865.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:34a::22) by AM5PR10MB1778.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:1e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.14; Tue, 30 Aug 2022 15:38:27 +0000 Received: from AS8PR10MB4865.EURPRD10.PROD.OUTLOOK.COM ([fe80::f130:6a98:bbad:8ffc]) by AS8PR10MB4865.EURPRD10.PROD.OUTLOOK.COM ([fe80::f130:6a98:bbad:8ffc%4]) with mapi id 15.20.5566.021; Tue, 30 Aug 2022 15:38:27 +0000 From: "Moessbauer, Felix" To: Anton Mikanovich , "isar-users@googlegroups.com" , "Heine, Claudius" CC: "Schild, Henning" , "Schmidt, Adriaan" , Baurzhan Ismagulov Subject: RE: [PATCH v2] use xz and gzip on host (outside chroot) Thread-Topic: [PATCH v2] use xz and gzip on host (outside chroot) Thread-Index: AQHYqvmWT1pqEVxCbUWOj1w4Ksjh4q2xrusAgBYF7bA= Date: Tue, 30 Aug 2022 15:38:27 +0000 Message-ID: References: <20220808073613.133842-1-felix.moessbauer@siemens.com> In-Reply-To: Accept-Language: en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_Enabled=true; MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_SetDate=2022-08-30T15:38:25Z; MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_Method=Standard; MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_Name=restricted; MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_SiteId=38ae3bcd-9579-4fd4-adda-b42e1495d55a; MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_ActionId=89c6cc6c-a1eb-4b63-b2a6-e16ee621c993; MSIP_Label_9d258917-277f-42cd-a3cd-14c4e9ee58bc_ContentBits=0 document_confidentiality: Restricted authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 02d6ccdd-3b67-46d8-1c78-08da8a9daedf x-ms-traffictypediagnostic: AM5PR10MB1778:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SbWfV0zU1Bk2b4GroMBuV47fkwLKznNujag6vTjjKB7ZufEXz/nSlU7qPJaBsYREyIbulWTLAMuwPEzB7bbANs4DnpIdDzucUb6GfYqpRPI5gVa9QHQxLJdjSeovldLS7Kf1B9LV2IcliQ5czUewEcQZDUM7HAU61QV3ku/LmkSvIaGYTRKx/uhL5A52LUZFghPjOQlX7HGrgO4AEWPCnWs7VyP+Fg6m/p2a/+ttrFJUE78svW9OH+4We1xfdsjh95pDe/Sdgk7EgTGDcV9B4pPk1cs0mR21LSVvQ41nssQ3Pc7/nGAHDCyZmVUjO6no2/9xBjVCMUBFPj6IG8+MXqWMgo5xoANhp4frmEdQGeuSbQ4V0bc53RCs58Jj5UWjrwOK3uy0DZCurTMH2qOGFj9hHmxbLShf6ZyFOvaM4/apc8vaAmtbfj/Nes54cXT+HXQmjYmEeU6FQ3E502PSSl1fY+LbBFNEDn1eSvz+lKgNeldeys3an6eKH/jq9e0xnOnat6x83CVFEHccY6gI0teP0McWam8j8uIQC3urnMB9i9k21KMoVwl2X1La94wXdjDgrnDIxlG1RSemp3TgcgrBoiIWxch6E0/BVI9hXGmO0j6Yk+bSgyzXq092TyC/fscgon33oXNxMt/PsRAC7en3heC8cLJ67uBO61Gbw16i2F9NMSD6kADzeJT9BW1twi+R03bERu7v0Gt3I4T8KsAtTA0nzYoEjDoK9ZEUKUlLMrE0iW9hx0aFABwIk6CzBZFo7t+cIZWQhYxjqXGm0g== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR10MB4865.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(366004)(136003)(396003)(39860400002)(346002)(76116006)(41300700001)(66556008)(33656002)(66946007)(478600001)(66446008)(8676002)(4326008)(2906002)(53546011)(7696005)(6506007)(64756008)(5660300002)(26005)(52536014)(55016003)(8936002)(9686003)(71200400001)(110136005)(316002)(38070700005)(66476007)(54906003)(6636002)(86362001)(82960400001)(122000001)(186003)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?wGFMUCLeCQ4dRvSuHrk00KA2qsY3Nfl5vO98zxBn4oK4iBcpB9YsYmujwo?= =?iso-8859-1?Q?Ezjy0+Ve+3EWZqVv55D0TPTrzILRO+JfhM+bAKKNc8JDb2vpj4xUN5v4j+?= =?iso-8859-1?Q?/UfzqxKFQl7iVt2NOiyGxAOxCaoN8NFHCRzx2B3WQAIipwVtLbUiJhpcMg?= =?iso-8859-1?Q?gqJ5aUhJY7f+kuuLKIX8GvdMPSV7kKkVW/PQ/8yjcIzNGHv5h4i3chFMfk?= =?iso-8859-1?Q?5T/STEiK3CUtzzh95E4316vKr+aoPUgOV15i2j5vkGjvi1si+vSOTPC3fh?= =?iso-8859-1?Q?4eljhYJIcNirz+Y6hwo53VFWWdApWDFotbkEw/W7NS2+7l5jRiUPeex4FX?= =?iso-8859-1?Q?Tpvw8i8wUbHga50DITg7mqQ79P9XuNlkK9jQfyxne5T6x7LqZQDOynW4tU?= =?iso-8859-1?Q?jDIumckkvlmElUu9qdJi6njoWFauaVf4ITEpxBCu03dtK9ax2LrtPG8D0g?= =?iso-8859-1?Q?S3uvHjSSa8SQ4uJNN9m1giOKTucnt2u6xf+vtTVrRXeXUBawqmQ3HhvXHD?= =?iso-8859-1?Q?kQKA2R6l56QWAAuHGa5HuWtO1kS78vWxgdjUQ9KQf4x5MRq6Y49+9WmQ6i?= =?iso-8859-1?Q?L7CQUPsWDKHr5oWGp4YVQIbMOJ9naX7SOqJAzv6/QVrR7EsOYLvSIe+6rc?= =?iso-8859-1?Q?iOBWX6fTJ1ktkQ/qcn0MSPBWD/BPGgyTYjR5pjhnG3E/6WHf/d8EZ4HIwX?= =?iso-8859-1?Q?c4MG5DsDMUdepoMcvGB+t1AS5TJqctntqmziIBAzRbmErfxDpfpqZP2+D+?= =?iso-8859-1?Q?h8kkk5oZ+zUFKaN8PO0f1naBN8EQ0UWYccFFnC+UNx7mzDNVNBQkzBifQb?= =?iso-8859-1?Q?LeMKcx/sf4ii92EQxNUCY2Cm1m+adJtWPZIj8DNfW4J81p5L7QK7X/Xh9/?= =?iso-8859-1?Q?9p7nRv/tvVx9cl4hyl1gUPMZUn6SvoPHOL9UhYv5SNEJsJVZaLMjDb6jaL?= =?iso-8859-1?Q?Gsj1AiWX2boatOX69wbRrQbldpt/Jy4rB6a4DfIqJxpq6dssw19SQIhzrS?= =?iso-8859-1?Q?xXESzYkzjp0dxJtZSeQsNXLCqx+Fs4BT2H95ZWAiKJeWkEfTbGE+nNRiBo?= =?iso-8859-1?Q?FHC/0WumpZUdDgC8dzCrNWpbk6NyDp0AgIdfW7w3xq1+EvxtXNvuAezAKq?= =?iso-8859-1?Q?J56K+6gTkPqI06tkiuLgjm7oAL4hq7R2xm+e/HKQkThdg1+HNyxIyTDkyl?= =?iso-8859-1?Q?zeAiX7sMkK34Fg3psVwHgwsKu7+PVFaUWLUu7zREoUfmnPRriwmTizKexA?= =?iso-8859-1?Q?zrcR8nuoM8X3CbYAVPKVDdgOke8yh/l+pwg6y8M3UBWRL1W6CNlAFiraQc?= =?iso-8859-1?Q?3CtqLw8+4v+C5rhck18jPkjpVS+LeqjJKSfue8BQ+VO2ebpKqS03a83bxZ?= =?iso-8859-1?Q?gDjbjkdFHj3HbSLvpBCMIhNCBZq2ijsVAVXhMVSZ6KbW+H78weJENisDV5?= =?iso-8859-1?Q?9/h7Rga+Y0bKsyOWdHr+4X8Jf5aELTrCs6xJF1aHDZXTrtoYxBN7Kkf5SM?= =?iso-8859-1?Q?7ecfcOOZDXXKfy2AQbUzlN2TeFBhVEzZgIyBHLA0ZlbLdf7feriLR4IPDy?= =?iso-8859-1?Q?K4O4Hmhe2d5c9JgLoSr+JjEN+ktqSEcTQ25T6jC5GBDmCeDceUM8UVwh8W?= =?iso-8859-1?Q?gR2TN80JsSLfp8S9zc7yUe+mlrK0LybJ3ny+3yxP+dGV92F7gScxpGZg?= =?iso-8859-1?Q?=3D=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AS8PR10MB4865.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 02d6ccdd-3b67-46d8-1c78-08da8a9daedf X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Aug 2022 15:38:27.4077 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: gpaivt9VZhe8SufNZBxJHiBO+E//Mw6Dd6Lln6Qqhfe/9QX1v5+BiVkiguHuHcJ6VuVNHWKy5Nbhl5WtoRWUevHAzFdg6F9FtN0wY6WualM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR10MB1778 X-TUID: PcwMDzoawNud > From: isar-users@googlegroups.com On > Behalf Of Anton Mikanovich > Sent: Tuesday, August 16, 2022 5:11 PM > To: Moessbauer, Felix (T CED SES-DE) ; > isar-users@googlegroups.com > Cc: Schild, Henning (T CED SES-DE) ; Schmidt, > Adriaan (T CED SES-DE) ; Baurzhan > Ismagulov > Subject: Re: [PATCH v2] use xz and gzip on host (outside chroot) >=20 > 08.08.2022 10:36, Felix Moessbauer wrote: > > From: Adriaan Schmidt > > > > This patch significantly speeds up the final image compression as the > > compression itself is no longer emulated. > > > > For gzip, no additional dep has to be installed on the system, for xz > > the xz-utils package has to be installed but this will be required > > from bitbake 2.0 on anyways. > > > > Signed-off-by: Adriaan Schmidt > > Acked-by: Felix Moessbauer >=20 > Hello, >=20 > I've made some investigations to collect compression speed statistics on > qemuarm64-bullseye Isar target with 490Mb rootfs. Here are the results: >=20 > XZ: > Buildchroot xz (QEMU):=A0=A0 45.305s > Host xz:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 18.783s >=20 > GZIP: > Buildchroot gzip (QEMU):=A0=A0 4m30.155s > Host gzip:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1m30.135s > Buildchroot gzip (native): 1m27.162s >=20 > Buildchroot pigz (QEMU):=A0=A0=A0=A0 32.265s > Host pigz:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 13.131s While this already shows the significant speedup, I saw much larger ones wh= en using xz on multi GB images. >=20 > So it looks reasonable to use native compression and we really need to th= ink > about removing ISAR_CROSS_COMPILE=3D"0" for imager. I just gave that a try and it somehow magically worked out of the box (on m= y arm64 image). Don't know why this was added in the first place. Let's add Claudius to the discussion as he introduced that. Just running the compression in the buildchroot-host is not as easy as many= things are tightly coupled with the imager (like installing the compressio= n utilities). But just switching ISAR_CROSS_COMPILE is even simpler and further reduces t= he amount of work that has to be emulated. Felix > We also can try to use parallelized pigz for gzip compression. >=20