From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7062963440528130048 X-Received: by 2002:a19:5e07:0:b0:44a:48f8:b495 with SMTP id s7-20020a195e07000000b0044a48f8b495mr9598825lfb.512.1648706169952; Wed, 30 Mar 2022 22:56:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3f16:b0:449:f5bf:6f6a with SMTP id y22-20020a0565123f1600b00449f5bf6f6als3761163lfa.2.gmail; Wed, 30 Mar 2022 22:56:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwje6z6Qpslw+tnCJw4uTjfqg6JN4nblNebqCSGpBUL0zZfWOjT8EtE61fHTDGTjKx4a+bj X-Received: by 2002:a05:6512:1085:b0:44a:70a9:4951 with SMTP id j5-20020a056512108500b0044a70a94951mr10148758lfg.17.1648706168928; Wed, 30 Mar 2022 22:56:08 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1648706168; cv=pass; d=google.com; s=arc-20160816; b=JFfDrMpXvsZ/vCWrwgh+YbPV97ZK9sgMFYw7kXbOlMnzGfDlVZVdxR7Sxm6or2lDOp eEwcm5tY6YmmUsPtaBSIYDg70DJffeqLjoYaWre716ylUzeUsYDxf2kKPUYTLJqbrMrz VpagFLhK4dsaKqcKFo5JpYkuL4DGBdBs6XRCOxCh6KDLhzhj92HRz9c35v6KtzCIPXIP 6MEBQMS3bkyWnEmeJSZyfJ5HXYHlpvOt4TopBMtFKTK2veGlG1PbzifSldZPGzW2ew4M HDCFiojSrJHJgEdxkA3xFNeCP/9VtLyDHqd6r5AJCLUGIcA5uOvjSiTvxQYQrikos10k nJjA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:document_confidentiality :msip_labels:content-language:accept-language:in-reply-to:references :message-id:date:thread-index:thread-topic:subject:cc:to:from :dkim-signature; bh=n5qzv65oRDRoOuBBrmxFUqAHnvYi4bdBzExOxqO1ou4=; b=Gak3/pFMAFv05dZoMFCKqwW9AfnaFCzX6q/gyzi50yY+X7IASohuw787E2yLPfXgxo hZyrfvMK7ekr5iAVT6GdUDlvm9b6dfoCJCLZugxhq3GKnzynnVn4XsnJkk5vrvqDM6vQ mPB9TPRbajejG/INoPPpnmBrU372wM0bisUjzWaJb/cvVHI4ov7dbVbhWrJF//PYwvmP VeIOl3U6CEcQbwAthKZT4/tgMxbYbmm+6go+A6uX1u+YIX6gpU0IOY5WaQhu/03wZmyW S6qIcRRHJMqOxDtat80TvMyyEUDxFJpwIQwV58tYu+bQFeSqAAHOYL21h4SLd0UepRXg dzsg== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=tpX1CMed; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:fe1e::62e as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on062e.outbound.protection.outlook.com. [2a01:111:f400:fe1e::62e]) by gmr-mx.google.com with ESMTPS id d24-20020a0565123d1800b0044a28635947si1185560lfv.6.2022.03.30.22.56.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Mar 2022 22:56:08 -0700 (PDT) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:fe1e::62e as permitted sender) client-ip=2a01:111:f400:fe1e::62e; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=tpX1CMed; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 2a01:111:f400:fe1e::62e as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k1fbzM0XWgsrVzM2zrdZkvhJ9y/EzYWCdod8WF67T1aGgmIA6SMyBEkN2rm7VPKKtmf5t1gPn62axAduRbGzRTaPHYeqCno/oNMjJT6JIiZTwyEMxdlUXLymAkEawkKlC+hRE/JI+lQ5K4ybfL1TYxO2Q7+A32thNX6JmcrDlWbJE/o7g8XwYVjqIpYazQu2VljStr5SFreWGi49WNBp2x+axTUztCEd16x8ngB8lWdNkP0SBdefu/7V92w1+9day+8MACt+WQgDWsS2/9ku2d3HtKRkHhhSJlfkAj4z2qTNNNEIumwLTYcX4SxXGryW3nhzcoGYCLrB8bLei0fkeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=n5qzv65oRDRoOuBBrmxFUqAHnvYi4bdBzExOxqO1ou4=; b=Zu6MH/867of4altQUXECtisO31ZOluVeNpTXH5HXYwMVeNtpSVgxBvom4EHE7g+CrUIaJEFcrol4YCBcDQOo8cBiwntL8dm9ymGhsvtfbPl//cePbeTOLu0nJ01THdFxB1VfjFKy9gvGD1H2AWtu4Hm8dTvdYfNvfyT7y1La0EiHiNqWto/9DTXY12jQ5qrieSjXpAVTDwJgMJ4EO2HWNeulkOBuL+mkf41f5jbFB0ciuldRNxTP+/cHfMbz7gJQkCpH8gAZswcpV14A/ONOXlhT/Os83QhcyP618/L1xPpNLDMBIeVSFXthLpi2cK8eu7fYLGmFJXIcsGXHMhX6Lw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n5qzv65oRDRoOuBBrmxFUqAHnvYi4bdBzExOxqO1ou4=; b=tpX1CMedEw+rDkd1YAkZnfx2MYigbKtmdYKyp5KoFCm1JVkNJglqWUv/vPuEIxAlHqz8w2Y8eAn85HorA9j79OEKutPy9RLeHgCb7jt6ros6fZENcvUF5PerCcN+6XegJhW52b3uIhsIEhZguRNV1+/zHJVBbn6/u1r9kyVi8LSN7TImDvG0FVPzUIkrLJZEzDpc7E/hW7u+ssV4b9UPp91DaiLQoldd7KxBTzkE8pvE+miT90fcCu1Ec7Hsnf0DGS+Fp3WSB6nJgL0/QCPTZPZ2QWpvlWg6PNvwdvmtlWjlrY41aZLANpQN308bnVYlghrN0UeDyFqG2c4f62Mduw== Received: from AS4PR10MB5318.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:4b8::15) by AM6PR10MB2934.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:ef::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.18; Thu, 31 Mar 2022 05:56:07 +0000 Received: from AS4PR10MB5318.EURPRD10.PROD.OUTLOOK.COM ([fe80::4582:f10:f69d:ded1]) by AS4PR10MB5318.EURPRD10.PROD.OUTLOOK.COM ([fe80::4582:f10:f69d:ded1%6]) with mapi id 15.20.5102.019; Thu, 31 Mar 2022 05:56:07 +0000 From: "Schmidt, Adriaan" To: "Moessbauer, Felix" CC: "isar-users@googlegroups.com" Subject: RE: [PATCH 0/6] imagetypes Thread-Topic: [PATCH 0/6] imagetypes Thread-Index: AQHYHkb7P9dKQz8Ca0+SCGQzLU++eqy1qnUAgCOevTA= Date: Thu, 31 Mar 2022 05:56:07 +0000 Message-ID: References: <20220210062446.4145565-1-adriaan.schmidt@siemens.com> In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_Enabled=true; MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_SetDate=2022-03-31T05:56:05Z; MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_Method=Standard; MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_Name=restricted-default; MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_SiteId=38ae3bcd-9579-4fd4-adda-b42e1495d55a; MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_ActionId=0c9806d5-21db-4b61-8396-68c6f48a66c8; MSIP_Label_a59b6cd5-d141-4a33-8bf1-0ca04484304f_ContentBits=0 document_confidentiality: Restricted authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6aa8f576-0894-4c8a-bb6f-08da12db2634 x-ms-traffictypediagnostic: AM6PR10MB2934:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: nYmv1/SrC4bATlZIud6CU0/dM3ljk7NZ+oIVlJ9HlAs39v3h0B7IPdNQ6EWKne9/O3vPEyIC3TKJannw67/kDxndP9gHQLeWZOvPTpFCGZ4twcoTjgcw/dwjwQNQRThDXrWBW8wDvObl+bHdUEuNg5HpePexRwffgfqs7NFSKUSXycOWfb4fLKD3cWVmEsRvqaOS/ACdSxcGJAqk/f9hrMNP8J+TAyTGyqhMe1Hyz0LoxGhGMSMwkTae0lpECczMFKXPGZrVAoKmjcdCfaiF8tEXUnYnnjamvD13g1Rj8zj00JhV01RHSPkc7U7eU3Y3E5pgmU/+bvLdz7AU0Btd2rrE4Dlv84NhmZNTwDfOeLKu67cWY4u7vu/BBSJIfB1WzPkhUYSSCd94qsPEwLBGMP3BnpRZPQD3VkK3A65da8nfi3Edu5f7DRvE0gfbC6vts+OlMCA9pVecFuv+SBpyCu1etQQbwD358PZa2h0N57hyTWFC5N+CmIcYTf5TEhS5P+pF8XeNFfTDM68d3UNJV4v6UxizdPIBtal3TRTIy9fImfCyotM/CYGB1jODne3+/cA4oFBVATnseb58Uef4qPKQ6dhBwJNSM3meNMSHpAJunlcIswTJGt3+WWGi8Fx8cu53iu+mn/b8+i6qd7U8gXyU2YUdpPxdFj6DOyd8EIv6pJYgXyrp51Ypb1Nwp6BZbBvYPqDLcQlYKa7zP6qljQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS4PR10MB5318.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(38070700005)(316002)(508600001)(6506007)(6636002)(33656002)(53546011)(7696005)(66946007)(8676002)(6862004)(66446008)(64756008)(71200400001)(9686003)(55016003)(66556008)(76116006)(66476007)(86362001)(4326008)(38100700002)(5660300002)(8936002)(52536014)(26005)(2906002)(83380400001)(82960400001)(186003)(122000001)(66574015);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?9RaygY26Hb7S75OmjUgY7gGaqz53NLJl/nX+/p3Tku/DSuSgBuurKS8Z+2?= =?iso-8859-1?Q?AjiPA02wkmyGEtetBXsACbR6AXqqYrP2tvbcqoul1uo164rz83KOG5wnC9?= =?iso-8859-1?Q?63X8cqBtJP+XiIBWlFj/R/bm1lvUIAlztQKsTelr952u3Q8T/NEw9C12vj?= =?iso-8859-1?Q?oNrnuqK9ryuvI0Rcy4KJkU1KNrwz1hKpZqM6vQ72Xqw89OewTQGJh6E+z+?= =?iso-8859-1?Q?GUNhJg94Wq+BZi2K+UXimLKIr8riBGafyEpXJWttk91y8I0WdNeH3Q06Lp?= =?iso-8859-1?Q?xs9NdwWqq2mzjTnIDjtuw0DrEtZxXCFzaiqr3P1M0kcExyalwPugZbtHgE?= =?iso-8859-1?Q?gmNeJ4fgX2hqNfyhAp10RnnoEtdaOYRz2vnYauQfauKrK2ZKCxM6un+ZR4?= =?iso-8859-1?Q?3Oa7NFCBIlsACbBZRMU9aBC2K12AhO/X3K9NGJXdlH3gBu23+8eSkyCY9N?= =?iso-8859-1?Q?eDwaYnAJk4uUVi9eoljY6NkEEUcb33xinbRU0rjehOd9cdxOCxBY9WgCZA?= =?iso-8859-1?Q?hCYntJrWtHqfDvboaXoqC2s1T/4M87RN6Ujp2Rkk6q4S5oU05arLMpQ6Nm?= =?iso-8859-1?Q?bjwl5vJRXWy+5yJrwhccxy9xY01DyqflzsBdzW01vW6xxNTEnkFNlMWPew?= =?iso-8859-1?Q?uT0eCh9Dsfrg+b3w/waGIu14sOCsZE9W0pvh2MNBA9mNwrDJtzVNowyJ5y?= =?iso-8859-1?Q?roFlfJXoltrqLz2DaIA9kknu16YUuyhxfs9QX+oWJdBo766IwJoZll9IuS?= =?iso-8859-1?Q?SNUfFdmhHcndpkz8zFxKDqStCjZ4Y+SoT+eOs9K0779wBbQV6WAmc7oJ1D?= =?iso-8859-1?Q?GR5jrZIc9xxCjqQXxho6ek4Cbaobvg4R7lA90OVONwtEJt/I4a/hFoCm6d?= =?iso-8859-1?Q?Zamhdg0q3dve/OVooFJ6k5yzKfhPCE+TfVa40NO9PzaOW+K5BM7lYkELbT?= =?iso-8859-1?Q?QvIpdpisXTj4jEQE7LxhBxH+HWeS8Ak+4/XZDV6x66RRVVeMwcfGnsWwy5?= =?iso-8859-1?Q?TUtaFoHVAd/PwYvrgvCn2LsJTS4K3GU9u0HLWI16lj4dawqGvhjCYk8UBi?= =?iso-8859-1?Q?1qgvN/Gydn+1g9IZadEmGzVFGkNu78ltkQkUAXk/TCZabgi6whscHZdiiV?= =?iso-8859-1?Q?jCYNvMEyebJoVsODfRFDOBWiOWmNSCUhIDLKKnRQgXtsLiIQpLBq5BSYut?= =?iso-8859-1?Q?spttE3KBkzFgiT7idHECrh4ftcs0gkgYIegr6hbBejzpbUX10Ww4V1n4/e?= =?iso-8859-1?Q?cDB8DYuKCmRZT7NBoh7xGD7q0NST08im/eEphBlTvD4e8vmT9tFVftZyKn?= =?iso-8859-1?Q?YyTUPqdT8KO/1D0goxg7V/Zr33f6aIUgVtbJ1spEsLjjhOx+y2Wb48SkuT?= =?iso-8859-1?Q?zbem5PQcYH6sVHaR+K60cQZd12399CFCOjOI7Yi1KyV0SdXFo4oGaXTZsR?= =?iso-8859-1?Q?zG96JMCqCWMyJWrZEsudDOjVaQvDeyyJFgJaMCYY2aHiEJEDWcJK7GRyC9?= =?iso-8859-1?Q?MDjQFKGZrBn9isGZYl1dkMv1k03O9eBs0r3XHVN/TgAVlal2WBDTatWjnN?= =?iso-8859-1?Q?fK8WdXR8NxLshCeoFT3c2FpWoxnl+8dbkHVQxteSJFjFMmQ/D7DNOnenxf?= =?iso-8859-1?Q?8Bb7ZHiL65L235P34Jv9u+vboOeHVh6EjawHFmtSy1Cpd/6vugKiV+oTeJ?= =?iso-8859-1?Q?cLA6qFCMRorhOeqBJHRaA/6Nrri43CmmbySfKx/EIHTWTz9xmptg1i2Ygd?= =?iso-8859-1?Q?1n3ng6vDcs59birq4jhBFyK8QoogV6eqTvZtRlBi1XPto6zf+6sFLHFwUi?= =?iso-8859-1?Q?Gj8ppGVAqBsuI561GonGVqUhTSlSpQ4=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AS4PR10MB5318.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 6aa8f576-0894-4c8a-bb6f-08da12db2634 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Mar 2022 05:56:07.3546 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: iGovdjjAoRyoIpvbVEqz0WFDuNByKzGtH9CNdQti933x6mDHOLiCtYGusqwjyTE+Q9E8QlQLvZsG0aP7FVI2mce1l1bI1o4s1kO3fy4l/Lk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR10MB2934 X-TUID: wIMjIKeQXI68 Moessbauer, Felix (T CED SES-DE), 8. M=E4rz 2022 14:51: > Hi Adriaan, >=20 > > -----Original Message----- > > From: isar-users@googlegroups.com On > > Behalf Of Adriaan Schmidt > > Sent: Thursday, February 10, 2022 7:25 AM > > To: isar-users@googlegroups.com > > Cc: Schmidt, Adriaan (T CED SES-DE) > > Subject: [PATCH 0/6] imagetypes > > > > This series introduces image types and conversions as known from OE. > > The main motivation is to provide a more flexible way of generating and > > processing images, without the need to insert custom tasks (e.g., curre= ntly > > we have custom classes adding tasks for things like bz2 or xz compressi= on > of > > images in several downstream layers). > > > > These patches have been tested on some of our layers, and also work wit= h > > some more "challenging" use cases like image-in-image recipes and custo= m > > image types brought by layers. > > > > It's not perfect (e.g., documentation is still missing). Also I added j= ust > the > > bare minimum of conversions, so there should probably be follow-ups for > > additional compressions, or things like checksums, which can also be ni= cely > > done with this. >=20 > While testing this on larger images, I observed that the compression time= s > for "wic.xz" are much higher than with the manual approach in our interna= l > recipes. > This is due to the XZ_OPTIONS =3D "-T0 -M200" and default compression lev= el. > Compared to XZ_OPTIONS =3D "-T0 -9" (less compression, no memory limit), = the > patch version is almost a magnitude slower (20 min vs 3min). > Maybe we should just provide better default options here. With "-T0 -9" I ran into issues in our CI: xz: /home/builder/isar-image-base-sdk-qemuarm/deploy/isar-image-base-sdk-de= bian-bullseye-qemuarm.tar: Cannot allocate memory Actually, "-9" is maximum (and slowest) compression, with up to 800MB of me= mory used, and if runners don't have swap those peaks can kill jobs. I guess my = quick fix of a memory limit made things even slower while trying to keep the "-9 = compression". I vote for setting XZ_OPTIONS ?=3D "" Which makes the compression default to "-6", and the number of threads to number of CPUs (seems that "-T0" is already the default behavior). Downstream/local config can override as needed. Adriaan > Best regards, > Felix