From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7067831991103651840 X-Received: by 2002:a05:6902:571:b0:619:5915:e477 with SMTP id a17-20020a056902057100b006195915e477mr1414851ybt.580.1645651995942; Wed, 23 Feb 2022 13:33:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a0d:d7c6:0:b0:2d6:b54c:9b25 with SMTP id z189-20020a0dd7c6000000b002d6b54c9b25ls587971ywd.0.gmail; Wed, 23 Feb 2022 13:33:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJyobLtuE24/NJQRUdGeqaBrcCwV3glxYQEboslg85xcVuHgUABaZd6e6oxiZ5oJfLqxt8Ls X-Received: by 2002:a81:84d5:0:b0:2d1:e85:bf04 with SMTP id u204-20020a8184d5000000b002d10e85bf04mr1504539ywf.465.1645651995475; Wed, 23 Feb 2022 13:33:15 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1645651995; cv=pass; d=google.com; s=arc-20160816; b=oCJGh+UOuyIJyF2MnoftalwErMAlKibos+dYxIj9nvv3eVqK3GKEyfieC3Dk8/fckc 5AOC13GvKjuHuUrkIrEoXQoFHiAtf2QplxPEa/zdRloERfjSho2/Vf5AFNmMdofz9yDs vcKEE3BCXtWIDBPKYlI+d8/yYe3WZBeilXpjnhVbRBAfoj+b2BCBgWoEHf4UcwoWDud6 /wE/9pCQjoGiBjXdKFrwGySnE0Cztgd8pXI9DCUf+1Q9sZSn7yZSAjUMzCoJ9B3+koeL UpQioxK2tVXN3VHlRh6EQerE/HkjgP61yTEiIEH8ZQkkloKLy6C/YHrlI8BvGDfdiKrk dG2A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=Tufno1RJZy/x7kg8U5W0S81Y72wh50wOgntbvFKyvic=; b=faTZkNoQ74vCgROfm5FIxkTNsdVf0Sc2EAOr234SfWc1BeYm0mwYhIUKfMtGi7TBQT o+l5hW9KP0g2JQVvOR2IbKaVtaxUZzYAkZ4V8cp//3QwkgZOvhrZimyhf/dUGQnhr/0H 5djpFJ6lGAvhY8HfMyJNAHyOyE6WoOWvR6hqzaXS9p2XbvvKkMgKYDAq4KaVtbqk3p6x bxLFbvzQAK4Z0J36W1zpQvVxw+KdyMKOI2k36K/ZyZlIGz7ZdOV14bkcB6hvT7PY18rC Ci/ofPTvTTjMX6FLuNaqgIQGv2FfhbRo5CSW5QfZL3vADrZbsMHUc/3V5p6AtFnHMc66 ZUbg== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=fTod9EUc; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::604 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on20604.outbound.protection.outlook.com. [2a01:111:f400:7e1b::604]) by gmr-mx.google.com with ESMTPS id e187si70344ywd.2.2022.02.23.13.33.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Feb 2022 13:33:15 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::604 as permitted sender) client-ip=2a01:111:f400:7e1b::604; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=fTod9EUc; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:7e1b::604 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c3nHRssz+9ZCSzNwROY/Zh9P8iB9kDod8jJGWh2G6khtW/k1NDPyWN509MW2yRSqPEHrwCc8GBeQFRKxBVs/iFaZFXIIZrCUbKfg+AqW/+ICuea9l/NY3bXJfehntidDQfwnr+ZRI0qoGSVFhUXB0p8eweUd5LG8okMlyYvGBX9/26yArfMGlnlJHJAYhT0YH4ex1FQyI4KxGkYRMBu55jD4KutKV1OWdxACEXYzlc70As4vqCQS8PFHdXd8nWwbMqDY8OSTaghhJmjQQzPA5qF8dWgvMH/meISrMLfArhiuQmhWevHCyqrRBcIG+JtJn8PWn94iDwVmafieUfLrSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Tufno1RJZy/x7kg8U5W0S81Y72wh50wOgntbvFKyvic=; b=JPZo5eo3cbwazihNL+1K3auI+Wwv8Ln7gxZLNVc6eXu4xsdUMXLSLyOpIABZZy4TM9xDRZYpyPFw3Rrz2nGjPHPkkRmvAUQ27vcHKagoBQ8VyZy2BtsSVlyTzie+lx1fgFcy71RUlTursQKfGuX5wrUR3aVPLRG/tCr0V9d80Gslti8s9nAMcjJNLxf9/mD1/GSKFAyK6tfyR3WRHzOtzu7WtWOSMUP/Vo3hyo180sR9PXG/aEWrKRtShtVmXiiqiCV/DQLMYmZ+i2ZuxN4bxrXt2UFc4CEBnQ+ZpImv1Scu3XsrT+mWC0uid7na2v9EgDniLkHbJ3BUsrytty7Hzw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tufno1RJZy/x7kg8U5W0S81Y72wh50wOgntbvFKyvic=; b=fTod9EUcV2JkRlpOgtlHylGoehbUOigKM95Iav7Q4O0Q8wyEvC4OlPbOYPaZn43BxPSdtwmnns/0Xavx75ovmPQbAr36ttZU1fvGNjcSl+DplEOB9SFwkHpLdRp70t1liWZBKI3KPc/PGDEVscSiD6tys4L5NaidK8Tqbu1fD+2iaEnfImdthjvkQZPP3ojM797spQPChFMpsjA9s5mdH0DaF/Omo45+HqEIhPrIljJXB29pGnAI4rbwk2BMBBo4kfxbL84CPE738SjAcXINRQ7MuLLAH+LEF+owVxENghOPi+91SGnZq4vImFip89ud5VnLblHai5MbXfXU3UeeGQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:155::20) by AS8PR10MB4952.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:403::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Wed, 23 Feb 2022 21:33:13 +0000 Received: from AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM ([fe80::d433:79c4:958b:1551]) by AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM ([fe80::d433:79c4:958b:1551%7]) with mapi id 15.20.5017.022; Wed, 23 Feb 2022 21:33:13 +0000 Date: Wed, 23 Feb 2022 22:33:06 +0100 From: Henning Schild To: Uladzimir Bely Cc: isar-users@googlegroups.com, "Schmidt, Adriaan (T CED SES-DE)" Subject: Re: [PATCH 1/1] sstate: compress rootfs sstate files Message-ID: <20220223223306.7b74e270@md1za8fc.ad001.siemens.net> In-Reply-To: <1793829.LH7GnMWURc@home> References: <20220223091749.17260-1-ubely@ilbers.de> <20220223091749.17260-2-ubely@ilbers.de> <20220223164321.0e55c5a4@md1za8fc.ad001.siemens.net> <1793829.LH7GnMWURc@home> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: AM6P192CA0093.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:8d::34) To AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:155::20) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0c758ca9-761e-4700-6629-08d9f714185a X-MS-TrafficTypeDiagnostic: AS8PR10MB4952:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Uvo7P2PyGr/m3GTOgAhTy7uEv58e4h5bMjZR/qfBmy1/p1BcYgdiTFTqZyfIKOWzGz6qWITlikwxBaKHIUdWtoijIv1WD0MYILw3OgROsm/pfQyE8t41ugUaBB8mH6nGAZpNPtMjwefa4kC/gmfjWfRImymsjKv1D5aLNgl4G1jDGKL65IB4VvjN1Ah5zWqn51CVwHnayKlRUdSu0HiWwGZowvkyySmluZiE2gr9QWmdWFGPxj+Tm5SgubvdGs0o9LwSsDn0y6GnZj2+qd9mHFQExhUT5KyAGMYrLFxr1N4HUOQxc5ufcIgkeC+UwysUCjsS8MMMUtjVHuGBsS0dpXsZIpq+J/I9CiSec4cmmfWmyqujSJxVIDw0VS4DsmHlW5kpohesApZ0xRDGXDH3tI7EWcmM49S7O6GtPwRYhFK+IOyrbKEdsEHZrBP53xsqPYce5C10Kxx00NRrnH3PRAdlMg/F7kosJOovcZDchpKFpPbwLZMOYUlhNjPPHSdurTFdv1Am/aroycblyCoOm/QeWq9b8TYMWbw6/l+zXOWKKKib+2/C5ku4MjlQpSENe9ol5vaOGzzSFeAQTCqxKmv6IUcBdsZoEx/MOpx3GbVJiN2klMtj29Olt8dIuqJz2wHM6GWxkwJXlqwNOFt8VA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(316002)(6916009)(82960400001)(5660300002)(44832011)(8936002)(2906002)(38100700002)(4326008)(86362001)(8676002)(66556008)(186003)(508600001)(66476007)(6666004)(6486002)(107886003)(9686003)(6512007)(1076003)(83380400001)(6506007)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?oVTuAGZofZqyfUF8hxR5V6f53jpUlYUrKThZANhnrqNMWpJc9SInACEbR/J8?= =?us-ascii?Q?Q5qkot31EOk2mqCm4oi66L87wNEGIh6fQ0HhjMAFKoHNf4k6RJQVxHDeK05Q?= =?us-ascii?Q?+2PfoQEokOnInluhhK3PIbotaH8SNRhh+OW4YzN8NIEtApa+Dr4PHf9zZVzb?= =?us-ascii?Q?vfIX1Qxlml4RhoSg0C0HjrgW/yoyMVqvWSMoBDGXyoOLIDu0q4INz7G0dTjl?= =?us-ascii?Q?I2CwnakE5Put5pqabLpxP1kFAN/mh+ZiqBTmmnD27HqGGJz9eS7e1sWaUNSw?= =?us-ascii?Q?INn4TYDBS5A7jb8R61lns2rAha3xXP2EnYZ4y/UsE0bRtKf8u5uXFgElGDXS?= =?us-ascii?Q?9O9qDK6fpoX5LZhJNHiIH/NV2eV2rNgZ1wEswVwAoGaGRPJ4K8r8CoSfRz+t?= =?us-ascii?Q?9rUeEewgrc7qWkoaM/992OS2qmeDgIWg71baWlZppAvFaPzx0iiTPDa0vWaP?= =?us-ascii?Q?LzjXH4eKNLYZlpVy3k6BEqtNUKfkXyHyv426/z+k/wt9wqOd8fZkhPPHf4Um?= =?us-ascii?Q?oCaoWmEyy0T2CD9l67zEf/Wtdu2Rl85fLfm25/nK6sYmANWkJPzjP0mKpSLU?= =?us-ascii?Q?au6wXtbCvIMGWihIaZHZyT+jn8+CzA7O+7Lf4Bvr9PUh/nmaaPNg/EZTsPA2?= =?us-ascii?Q?DeOHaApcD3n8JLMVwhTq8Qxq8gDqocUUa3ccWgd0NtdbzshPeMehTxUTZCd+?= =?us-ascii?Q?ahqgYnRXUFZ6jrB3ebsh+FuQQd4Ia+LTzQO/OIs4BCQaMFqXYjMxKXLwLqy+?= =?us-ascii?Q?pOHMWVVtFRc3dZLZ9jm6cqBBAdbss69SMDs1NmTjaGq5XNo+4g/30V100/gQ?= =?us-ascii?Q?Wji1m8nhVM3fLGzfZRPbr7BQ4voL9lswbQ/ZZnBlOS96Q71k8Divm4FDo3Cu?= =?us-ascii?Q?HsP9jasFcUVe5Ujmc8Z+EMUzBz/unS8OoUX1UjJzCLLDwDKd5ys5XLgl9L6u?= =?us-ascii?Q?rLsRnX47p591VuzPUS/VNWGvawdB5TnS4/K/8QRocXMSeulcRN91R6JJo0Er?= =?us-ascii?Q?nuQGGC7myHBd/Pyjh0nHT1KIJuv/UNVHdEBbI4xnE81EiNikaKqT6aFOYtBI?= =?us-ascii?Q?qOG3SpD9hCQggI+l7DN9E8aQN5jtYxrWkxu2VSQf22sHxUk492pEClhvxIxy?= =?us-ascii?Q?oPpjshim+TkM5TRjNvsuJtcKVhpb9iQq6D+sCJh6jV/z75+AQubcrtRNpp18?= =?us-ascii?Q?i6hiBRUfYZfMRHOrqMnig6X2foUtfLW+PkICG16tybiKdz1keqVMlO9i3dNO?= =?us-ascii?Q?W5NY7Bd/sVMtCAJEe9gfZz3Xxr6cy3VkBmuD/AITKl+Ibs94JovMl3w+jTos?= =?us-ascii?Q?A69qe0jYxcIY5iPDk2N9zzrne1tfsNx2Q/Ia8nBTRXiPDBpSKhtKhL0BMRX0?= =?us-ascii?Q?zcM34LVsIsqISauKnH5x6mYXLKl0EOK5lTl5UsUv0ZZlqtSRSW1q/rEDjaVY?= =?us-ascii?Q?O3Vw+JGASSIMl11xYzZWyJbm0q7Ed6Dvj+kn8FcUCeQWzCUJYt36arODEEWh?= =?us-ascii?Q?y/aH7DRyU1nbFO8lctAEnFdpyS3E0janFaLX6oDkLTXYo3uMfmNoORux7p5q?= =?us-ascii?Q?1wIeqgReDCv6OqoKZ7fqEoZRpqeKbE0OFvQJ68rcebT0iP3UZBmcFHcvSWzv?= =?us-ascii?Q?CQN7RH5Uz9OWVwBGKgVbbT4hLE0c2W4AogLgYTSMm2F+QpeM0GJ0J6JAqs34?= =?us-ascii?Q?X7hJVpSQCet37Kw/NoaK5DK428cIJGY8+U0OIEsFAq8LxNNip+234w61X2r0?= =?us-ascii?Q?WWHUe9wuKA=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0c758ca9-761e-4700-6629-08d9f714185a X-MS-Exchange-CrossTenant-AuthSource: AM0PR10MB3459.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2022 21:33:13.2900 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SWdcJLl75NN83c8dcB9+6pU37aPBzzG7zgo4xjsMlp8MaTQbUhccgL/607uUYGbXDwSw4+EocpNH/tloFw+q1Vxnf2lEJ6xpltkn6T2ZAyM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR10MB4952 X-TUID: ne1bvaPy7fey Am Wed, 23 Feb 2022 18:52:54 +0300 schrieb Uladzimir Bely : > In the email from Wednesday, 23 February 2022 18:43:21 +03 user > Henning Schild wrote: > > This is wrong. Everything you put into sstate is already compressed > > there. > > > > So what you probably want is to "move" stuff into sstate and delete > > setscene after unpack so temporary tarballs maybe do not hang > > around. > > > > That will save much more space and time, but i am not sure it would > > be "allowed". That is why i added Adriaan on cc. > > Yes, it may be wrong because this tarballs might be needed to compare > hashes by sstate. But why not use them already compressed and save > space in workdirs? The real way to save the space is to not keep sstate stuff at all (in WORKDIR). When putting things into sstate we should remove it from WORKDIR right after it went into sstate. When getting stuff out we should remove staging stuff at the end of setscene. SSTATE_BUILDDIR seems to take care of some of the bits, i did not manage to write the correct code for all of that. But compressing bits we do not actually want to keep does not seem right. So a clear NACK from me. Henning > > > > Henning > > > > Am Wed, 23 Feb 2022 10:17:49 +0100 > > > > schrieb Uladzimir Bely : > > > This saves about 1-2 GiB per multiconfig, at the expense of > > > approximately 20-40 seconds for packing the files. > > > > > > Signed-off-by: Uladzimir Bely > > > --- > > > > > > meta/classes/rootfs.bbclass | 4 ++-- > > > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++-- > > > 2 files changed, 4 insertions(+), 4 deletions(-) > > > > > > diff --git a/meta/classes/rootfs.bbclass > > > b/meta/classes/rootfs.bbclass index 2bdb3b6d..15e87bec 100644 > > > --- a/meta/classes/rootfs.bbclass > > > +++ b/meta/classes/rootfs.bbclass > > > @@ -298,13 +298,13 @@ do_rootfs_install[sstate-interceptfuncs] = > > > "rootfs_install_sstate_prepare" # the buildchroot is owned by > > > root, so we need some sudoing to pack and unpack > > > rootfs_install_sstate_prepare() { sudo mount --bind > > > ${WORKDIR}/rootfs ${WORKDIR}/mnt/rootfs -o ro > > > - sudo tar -C ${WORKDIR}/mnt -cpf ${ROOTFS_SSTATE}/rootfs.tar > > > --one-file-system rootfs > > > + sudo tar -C ${WORKDIR}/mnt -cpzf > > > ${ROOTFS_SSTATE}/rootfs.tar.gz --one-file-system rootfs sudo > > > umount ${WORKDIR}/mnt/rootfs > > > > > > } > > > do_rootfs_install_sstate_prepare[lockfiles] = > > > > > > "${REPO_ISAR_DIR}/isar.lock" > > > > > > rootfs_install_sstate_finalize() { > > > > > > - sudo tar -C ${WORKDIR} -xpf ${ROOTFS_SSTATE}/rootfs.tar > > > + sudo tar -C ${WORKDIR} -xpzf ${ROOTFS_SSTATE}/rootfs.tar.gz > > > > > > } > > > > > > python do_rootfs_install_setscene() { > > > > > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index > > > 2f483f5a..99d1d945 100644 --- > > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -402,11 > > > +402,11 @@ do_bootstrap[sstate-plaindirs] = "${BOOTSTRAP_SSTATE}" > > > do_bootstrap[sstate-interceptfuncs] = "bootstrap_sstate_prepare" > > > > > > bootstrap_sstate_prepare() { > > > > > > - sudo tar -C $(dirname "${ROOTFSDIR}") -cpf > > > ${BOOTSTRAP_SSTATE}/bootstrap.tar --one-file-system $(basename > > > "${ROOTFSDIR}") > > > + sudo tar -C $(dirname "${ROOTFSDIR}") -cpzf > > > ${BOOTSTRAP_SSTATE}/bootstrap.tar.gz --one-file-system $(basename > > > "${ROOTFSDIR}") } > > > > > > bootstrap_sstate_finalize() { > > > > > > - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf > > > ${BOOTSTRAP_SSTATE}/bootstrap.tar > > > + sudo tar -C $(dirname "${ROOTFSDIR}") -xpzf > > > ${BOOTSTRAP_SSTATE}/bootstrap.tar.gz sudo ln -Tfsr "${ROOTFSDIR}" > > > "${DEPLOY_ISAR_BOOTSTRAP}" } > >