From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7127570847942836224 X-Received: by 2002:a05:6e02:1aa7:b0:2de:b9f1:593f with SMTP id l7-20020a056e021aa700b002deb9f1593fmr2892703ilv.243.1659703513534; Fri, 05 Aug 2022 05:45:13 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:9544:0:b0:678:623b:7360 with SMTP id a4-20020a5d9544000000b00678623b7360ls472642ios.11.-pod-prod-gmail; Fri, 05 Aug 2022 05:45:12 -0700 (PDT) X-Google-Smtp-Source: AA6agR6lZWnpQbjge0iq6/mKX9PPovqxjaKMf51QreSvbwhA62TQs3JOWHMy1HebhywahG1S5Ior X-Received: by 2002:a05:6602:2b09:b0:67f:c159:91b9 with SMTP id p9-20020a0566022b0900b0067fc15991b9mr2764895iov.182.1659703512837; Fri, 05 Aug 2022 05:45:12 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1659703512; cv=pass; d=google.com; s=arc-20160816; b=k4a/iCMkFExsIAaFlvJFrddbu81uAnkIcIm1seaNPmxLKUPX3w/KOhtMYd/kwdJNuW dMtQzQAEhxS4CrFZX5vGea5qJ21ljR0mQlT35EaThiTNX9iYUYJSoIG6SX13uo8Z1eVY E5VbW5lXBnKs648B8xdKTuuWkcPLnp4O0ugdQXOlbqQ4jhdp6GXDpKTa/E8KjLMHMWBv fMCZzK9i3HJgS2zwUKbNPa4Fk/SX3ZgLHRttcJB8JmPtaWdtvVqPlVG8rxEYx8JCBiBM I0dkpxt4qU517WfExSpvSqf21j9IZ0nxHEATUJv1RjA4t9KTpTjZbEr88oUJ9BeMSp9l gdrg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=8AeLHnMoc4llPO1BVp23Kfh3YCzRVdYymzQcZ6aQ6DE=; b=G7zcHaNQbrfuFgkBmUvXR7YFstdkXRbYsuM0seraTszfuhq4RjRNzqQyA/kNIxqLHE 4K1qHfkUAlE6wzWvD+7CUfbCsm0LkbcLjQulHybyn+hIsmrMWo3jXnaqq674v6M0I0Ya QCzKB84+4R0Owhde2RXHBXSMUfj1xwSnVdVwe1gmhERzPdsync34Xy8vJ7V+5iNJzAU6 qWN5O8NCOzugy6WuSTkgGCWgCo4vM+lArycTuzphHIHfbwt2337otNR+TZwMSa9St5iu kRvuXYy0EdiZhDRxC+M3AkFGCkiixsdUSLxCY32N3IPe/j4oh8NDTYcAo+uez5kZH4aX 46IQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=AeRJYFUa; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.13.59 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130059.outbound.protection.outlook.com. [40.107.13.59]) by gmr-mx.google.com with ESMTPS id k1-20020a92c241000000b002dad0373761si194056ilo.0.2022.08.05.05.45.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Aug 2022 05:45:12 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.13.59 as permitted sender) client-ip=40.107.13.59; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=AeRJYFUa; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.13.59 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=effgpvajF+hG7qi3YcS0sb6mWuhjqr0i2NEdK1faUnIZbQwLaKBm1st9mW3ZCjRSisedltIKzP04I8iuq+aBrI+7xtUeMYFN9RTr+oQpMithNEvvyqizJzIK2MqVEbX582QDorAT2sw2pDceTnBrljtxTytxDIu80Zn8nvLXtb8W82VuYEQMaJZ1rIF6Qn7/msc40vZ/PC/F6RQ6wgup8uYFZbuOfjptI5I881G6r5V2PprWrc92ZFpBct+3rguriWix35ZwJ+M/cC/3TDvkrZ/rWTo/z5xAALeNDbe34j+sXjHElRPRv3PHU0YywJLbJP6RfTdaaXi1L6jMg5hzog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8AeLHnMoc4llPO1BVp23Kfh3YCzRVdYymzQcZ6aQ6DE=; b=mtzy69EnuD/WuBw0N3+W8KjmOWnpNZTjsgy5I75rq2UkAEU8NEM01A/nCpPq8Y65THYIvfNi3WzOv4ob3ax8KjrXXu/dYnlGawjf3sOxxiBsyg8LIa9yzf0RnSATQvHCwpO/IW0nkTTuSEp+TXHhR8zn5MTEE/p4nEKn8VKT44LkDcEHHuv7eCc9jbqp3Q1FOY2PvjULpmLloCL5kgLn0USzL9lCYPWP7rJGWXIRC0GG9p093oh/VfiwfyJycmgFktSclx9Pvzm6nhJz6E5qDgK8qewaamZW1j9Ebw9taLY648NXd9aBWm/wDU+wvqsmOc1/nICtVaGcleKfiLVYWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8AeLHnMoc4llPO1BVp23Kfh3YCzRVdYymzQcZ6aQ6DE=; b=AeRJYFUaaR29NudtaRe7lczh/g+hXe+E1IW3Wlwv5iA89m89hvysKPcjFNp/Fy2G1XmLRvk+134lV6yLaqKKw8IkvLCM8Bap6fgUIeOLPYPY6mVHLeZ4FJxRg/Xnn5VanXSMoHD8rL/C1gXdAEf/kr0J8NGl0PwsI33PthkyOWkEFTDpNs5GvLJejZxz+cNKiOHqaHLJnPM3++qL+1iU/KgnbUnysD5QwbPTbVOxV1trgQkd8ZRZ1tkBT+/EumZv6Yf4Re4+cIf+xAEgTkNp2sUISJ5I3jy5xEus1kBJiK0O4xMkydW/z6rQSl2aAs8x12YQUibdIu8rtrbg6eEgQw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by GV2PR10MB6357.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:150:c2::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.14; Fri, 5 Aug 2022 12:45:09 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::75ee:d5d2:6b1d:150b]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::75ee:d5d2:6b1d:150b%3]) with mapi id 15.20.5504.014; Fri, 5 Aug 2022 12:45:09 +0000 Date: Fri, 5 Aug 2022 14:45:01 +0200 From: Henning Schild To: Felix Moessbauer Cc: isar-users@googlegroups.com, Adriaan Schmidt Subject: Re: [PATCH] use xz and gzip on host (outside chroot) Message-ID: <20220805144501.22d9541c@md1za8fc.ad001.siemens.net> In-Reply-To: <20220803085448.2765072-1-felix.moessbauer@siemens.com> References: <20220803085448.2765072-1-felix.moessbauer@siemens.com> X-Mailer: Claws Mail 3.19.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH0PR03CA0001.namprd03.prod.outlook.com (2603:10b6:610:b0::6) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2faec055-fc00-431c-a580-08da76e054ce X-MS-TrafficTypeDiagnostic: GV2PR10MB6357:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DUDHcrPBblausKJhFvGU/FCxcBj7A+DW0OOJ6oYCXMYlAOCymaQrB5HW21yxJB2J+l9gsT/6g6L0gq2e46bHmClJ8Tjow/sugaDsJVhQKCu8dFIHR1Dl5P+tE0nKvGNQHeEmf5AZMNcMtMFZFkcgXRJ4k6Seam3M0voa25vvF/Trnto2H4QsaYhXQW5UHSR8GT5N3o/OzHAWPR043ulXv+exyKovsAbhltM6CZnWmXYAJfLrFVB5Dc8owR0wEJ7TdtsaBd4cR5OeYARQyCZ7yWS2FSmweLrck1rXV26G4N6s73+8fF3L+xdRySktymTCPXTAVNsJ3122JwisghdjpctJ2Q5fh8KZ84Q8iGAEXao0VzH1pbBX4TvGmZag4TV93eNHrnBV1w400W6aFO9Jq9JQImB7fk8d5AZkf2YL/bxrGyVV9Y3mRyu6urHSBs+VkuzV6nGNS3TRixoSKjpNTivaNojFiVPD79qqHLApNsY3pxs+wnTWrdmoxoykMMBPY5XugvWWQbXZJ8KoOvec8BM8mSyUU2LlM3jsbD9L1gQa4BjtulrFIfUrM9N5T7E17BaYii/X3srHoHWK/LP/Y+1vpJPAGCUcz6xd8Bpn4iaAgTdW0q/HEzyHvrnQZJM8v/91PJgjJDfZhOczp3Ot9F9DpRBp8unv4rA++UFGv2FsOa+j8MtQeelkq74+JQcI3lFhzVEjb51wg4145hsIRLvvxfLLalP19WjP+NC1DmwdqxSgr2DLQUKCKnMVn0Bx X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(366004)(376002)(136003)(396003)(346002)(9686003)(107886003)(186003)(1076003)(8676002)(316002)(6512007)(4326008)(66476007)(66946007)(86362001)(26005)(6636002)(44832011)(6862004)(41300700001)(6666004)(478600001)(6506007)(8936002)(66556008)(5660300002)(38100700002)(82960400001)(2906002)(83380400001)(6486002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?TAwIxqtyiUuVp2ekm2MSWxkG7KKfMmPe5gg+lHBQoyiaZWdwLjsWHvtYwFca?= =?us-ascii?Q?pgwgQPHGchr2kD5yYL8MHuJGBkTXFLZXCdfSRF4sjYfQNGqWP/69/E+yTzJ2?= =?us-ascii?Q?NDsXuxB6RLG63jadQRGijbAZYgSbzc4ybrZGtI6IHgqfCpOucvUvzzwRBPVw?= =?us-ascii?Q?i/OvbZxE4S7Yai6qAP2aJDMEbla/82pNAT2mX3+iRf2V+GdyczLQuBgjwlX0?= =?us-ascii?Q?OyHTUCVgf96RyYRgQVtBdV+KQe5kJnObnX2VfCaJ4atZlJ0y4/l24XNby9RV?= =?us-ascii?Q?Js0NV45HvOfbUh3Tgy3YGl8dkHrkFoa3q+e9CpNaLRIk+FPMWvbIqDX4I7gu?= =?us-ascii?Q?afcrKbbbqV+G1L1OTLNkJzslLehwCYVygwhWU1bIASLDbImIXffaWXFF+xPW?= =?us-ascii?Q?dRjSpZ3ehiAxDBHrf+HkaRSCB8HQVe2cGb3w3JG86GkzFLwGcCO28tcvxnmZ?= =?us-ascii?Q?hHwDCSa7BdImzU0S8ZjHeCcQiWRNwspU+eQh3jFsM9seQ2ggkxowHfhF5AdS?= =?us-ascii?Q?hESpOAhyMK7xuZlaJ1kPnqvLjHhExBrJtRQNLp9YlMkiQbm6Qd/aTDPwzUCm?= =?us-ascii?Q?nhyFALPlL5LL7u2Q305nCdRFJj8PdkCUWAlMtP2P2zeoDSGsBLlIcKO7HBdH?= =?us-ascii?Q?4ETq+82zVcD7M0fi78CGJ3q4IeBSrMbvTvd/AtHmdku8oH4KR4eLTADXZTC2?= =?us-ascii?Q?Vr4CG1BxFXuty77SWK4a7XOoedZYPKdZloWcyqTfHU99NClC9W82caTSPrVA?= =?us-ascii?Q?suLh9FwHWHlo7tAb233ruvD+PmhwmQwzVUwJZ7DRUYMW9Rox5/em/LSkYqRv?= =?us-ascii?Q?Sx0QeBd/d95WmLaOM6apHRHF5e0gXfpxXnLIyWoDVWiY7gUqjSJ+aM1YMneK?= =?us-ascii?Q?hPDRFuFFlwLwNDwdO0wkhylq/xmtAywLL/5RyxiFcQNUN6ReE3E7MS0ARdjl?= =?us-ascii?Q?p+UR3h7+98mo2wBZSke1ocTacYCaMKxWYAKUWXg2ta32TK687Sy4XwJWHFhc?= =?us-ascii?Q?bTUohY11QJCIfCvupy3YnHa69ZiNg5KNwsrIQcvaNkM8vZVuNpJsnRNgDRiJ?= =?us-ascii?Q?iKE/IfZZBEmvPi4CRQICtlsMhdVIhwrW+hEzfQez6EjJw+RzeBn8pJfOkeYl?= =?us-ascii?Q?qEymNTa3kGQUdixkTX57rWlk2tBOAunO4EK8ki45u61yMOvG5sujJLSnjXus?= =?us-ascii?Q?LBZCFDja/en/0e073BDWT2AHN8ZWo9oHs9MVE7Hz0GOxOvfmd6o4mi0IW0bu?= =?us-ascii?Q?2Hajgr6zo6/upZmRUQZ53e27+PUQg1cXa+DRyVL3c1fz9AnkC8YtSTAYhk/h?= =?us-ascii?Q?umNi4/Sz4N6pmPvFJkvWM7JJqy4Cg2DeXZP4ZYkOJgKx4is2m0vz+DMgJzNz?= =?us-ascii?Q?KRy1ezwh/bGGp1V4WPC3ELAAV8l7NJUYOX/H9+BLE413UpHO931gFBG/J7mP?= =?us-ascii?Q?heSAHELF4NncmRPvDfnDYjfmdDwwMu71gYDpLro1dPSq4xX0MEaSvOo/ireH?= =?us-ascii?Q?0xZK2yLDeOlxcCmfdpuwOIcpbG0Zxqi/oGKknfKXXTCtjUpyWmhSXxynBnXf?= =?us-ascii?Q?ajmL1kXyqCjVsJii+sKcwq2yEiksUKnXmVA/r9Sm78OJVRckmbgYanME7Izr?= =?us-ascii?Q?KA=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2faec055-fc00-431c-a580-08da76e054ce X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2022 12:45:09.4720 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m5+d4Zmr6sUE2+x0fffBkkY9EnHcpf5eJUSc6rtMEhBfAnD55SyPvg8WKy2riPE2WYo48CVkgG9aO7giWAhblis+el6uu2RrgLB/WwemhJg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR10MB6357 X-TUID: d4zLyhtzndqt I am not sure i like that. The version of the tools one gets will be out of control when doing that. While that might seem harmless ... it is not! I have seen bzip2 problems when the compressor was too new and the extractor too old. Windows machines could not unpack or "etcher". We have also seen several settle differences on other "super basic tools that just act on storage", like filesystem and partition utils. However i get the speed argument. So i would suggest to come up with a "? = " pattern for the CMDs. Or when you really just want to run a native compressor in the "target version" we just need a cross buildchroot instead of a native buildchroot. If at least one package builds cross we have that anyhow, but we could switch all "imagers" to using that ... code for creating a cross buildchroot is in place. Henning Am Wed, 3 Aug 2022 10:54:48 +0200 schrieb Felix Moessbauer : > From: Adriaan Schmidt > > This patch significantly speeds up the final image compression > as the compression itself is no longer emulated. > > For gzip, no additional dep has to be installed on the system, > for xz the xz-utils package has to be installed but this will > be required from bitbake 2.0 on anyways. > > Signed-off-by: Adriaan Schmidt > Acked-by: Felix Moessbauer > --- > doc/user_manual.md | 3 ++- > meta/classes/imagetypes.bbclass | 6 ++---- > 2 files changed, 4 insertions(+), 5 deletions(-) > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index 546ae75c..932bfbe5 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -83,7 +83,8 @@ apt install \ > reprepro \ > sudo \ > git-buildpackage \ > - pristine-tar > + pristine-tar \ > + xz-utils > ``` > > Additional setup is required since `sbuild` is now used for package > build. diff --git a/meta/classes/imagetypes.bbclass > b/meta/classes/imagetypes.bbclass index 1e8e223d..c4b06ca6 100644 > --- a/meta/classes/imagetypes.bbclass > +++ b/meta/classes/imagetypes.bbclass > @@ -95,13 +95,11 @@ IMAGE_CMD_ubi[depends] = > "${PN}:do_transform_template" # image conversions > IMAGE_CONVERSIONS = "gz xz" > > -CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c > --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'" > -CONVERSION_DEPS_gz = "gzip" +CONVERSION_CMD_gz = "gzip -f -9 -n -c > --rsyncable ${IMAGE_FILE_HOST} > ${IMAGE_FILE_HOST}.gz" > XZ_MEMLIMIT ?= "50%" > XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}" > XZ_THREADS[vardepvalue] = "1" > XZ_OPTIONS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}" > XZ_OPTIONS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS" > -CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS} > ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'" -CONVERSION_DEPS_xz > = "xz-utils" +CONVERSION_CMD_xz = "xz -c ${XZ_OPTIONS} > ${IMAGE_FILE_HOST} > ${IMAGE_FILE_HOST}.xz"