From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7226603299007365120 X-Received: by 2002:a19:5510:0:b0:4ef:d742:4dff with SMTP id n16-20020a195510000000b004efd7424dffmr133091lfe.3.1682574697919; Wed, 26 Apr 2023 22:51:37 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:2117:b0:2aa:14a2:14bd with SMTP id a23-20020a05651c211700b002aa14a214bdls158277ljq.6.-pod-prod-gmail; Wed, 26 Apr 2023 22:51:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6esDYX7pRgmR9YtqtJbAkMdAJkaHVzGN6a0FXyKAns2xJSon1jK3t7O1Y+Ux9ROPNcNalG X-Received: by 2002:a05:651c:104a:b0:2a8:c7e0:7764 with SMTP id x10-20020a05651c104a00b002a8c7e07764mr294360ljm.18.1682574695972; Wed, 26 Apr 2023 22:51:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682574695; cv=none; d=google.com; s=arc-20160816; b=b9RO3PrYfU/khXb5gIY0IU9BEX4XgshK7kYiF6I9dEFgOgKqwjTxLCq8HnaOooTGUU 8KTNowzW0aJeaLudpyfbQnqGHQ9ojyTfmo+34HwrjPb3u3H+kXsZFWLwLi2NvDCDqxfQ G7K7cJt9/VM2b7obxEUgPX9kPDRxR7YvAWk0S3UbZOzVIBuhrd78Faeneb0yEu0U7GuQ 9gtckto1VNsQUqSAXhNNcFPc32QtC5AzC7h0pek4lSYVOdfUO5SbNxA+by3zRvyngrPf 8WUn3apjxQotkiCA+Ipg7PVCEAAPN79QDm5z2ik0uROZ49vYVpbeLSw7y4rL0NHEuFgC h69g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=NCeK0Wsfjwl25BAj/0xUdczw20Sv+MYEzEC3P8tU4OI=; b=LANJK57hxNa8Su/iAEMbF9exE0TiJPIYSS+iav528cpvx4f1EUhhIVLoqXxtBgCXde bN7yrB010jar/QxZ01mJjCKcoVxfxLg4Z1OHDcSJZbRjCXngrAun6avWa0FxkCgyJhL9 P3mWr2KjUYsc/T0inycjx1eS9PE/fRVQPZiRoLi7qqC4GJR9mmp+6dgrn3tdelUaGfcQ gyxbqXLmODiA7UccR5KF1J0SSkf8cri/39GuWG39vkth92nJwmJK55OFmHbBpZDBKcCd 7ys4VGI21wsPM+rGfWoaeYH6MouL21kfZzEg2SegLZPOIZ/Dms8sYFtDWbYpnJnlScKF /UCw== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=HJuLfP2f; spf=pass (google.com: domain of fm-72506-202304270551349d19be964259d636a6-mlwhf2@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-72506-202304270551349d19be964259d636a6-MLWhf2@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Return-Path: Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net. [185.136.64.226]) by gmr-mx.google.com with ESMTPS id bz18-20020a05651c0c9200b002a8b4c3cf27si965737ljb.0.2023.04.26.22.51.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Apr 2023 22:51:35 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-72506-202304270551349d19be964259d636a6-mlwhf2@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) client-ip=185.136.64.226; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm1 header.b=HJuLfP2f; spf=pass (google.com: domain of fm-72506-202304270551349d19be964259d636a6-mlwhf2@rts-flowmailer.siemens.com designates 185.136.64.226 as permitted sender) smtp.mailfrom=fm-72506-202304270551349d19be964259d636a6-MLWhf2@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202304270551349d19be964259d636a6 for ; Thu, 27 Apr 2023 07:51:35 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=felix.moessbauer@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc; bh=NCeK0Wsfjwl25BAj/0xUdczw20Sv+MYEzEC3P8tU4OI=; b=HJuLfP2fK4tpp2eAWXJNQgUZPFonwPdFwwcKwnmrDbOTkvQ/dXxHOpT6SjaFls5Bi4W/Xn XRUWChqM2qsOGRP33xCfLHmhajtfmQRKSNzbETLznawuIo3rcOd4kumV+xofcqviDISKozV9 iUymPuZj5x7OJwDpl110NA0Jt7zbA=; From: Felix Moessbauer To: isar-users@googlegroups.com Cc: henning.schild@siemens.com, Felix Moessbauer Subject: [PATCH 1/1] add zchunk compression support Date: Thu, 27 Apr 2023 05:51:18 +0000 Message-Id: <20230427055118.1993072-1-felix.moessbauer@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-72506:519-21489:flowmailer X-TUID: 04GzvzFt2hgW This patch adds support to compress images with zchunk. The compression format is optimized for delta transfers, by ensuring that small changes on the input also only invalidate some blocks of the output. There are not yet many configuration options available, but once available, they can be added to the ZCK_DEFAULTS variable. Signed-off-by: Felix Moessbauer --- meta/classes/imagetypes.bbclass | 5 ++++- meta/conf/bitbake.conf | 3 +++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass index a80a6da5..3639662b 100644 --- a/meta/classes/imagetypes.bbclass +++ b/meta/classes/imagetypes.bbclass @@ -93,7 +93,7 @@ IMAGE_CMD:ubi() { IMAGE_CMD:ubi[depends] = "${PN}:do_transform_template" # image conversions -IMAGE_CONVERSIONS = "gz xz zst" +IMAGE_CONVERSIONS = "gz xz zst zck" CONVERSION_CMD:gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'" CONVERSION_DEPS:gz = "gzip" @@ -103,3 +103,6 @@ CONVERSION_DEPS:xz = "xz-utils" CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c --sparse ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'" CONVERSION_DEPS:zst = "zstd" + +CONVERSION_CMD:zck = "${SUDO_CHROOT} sh -c 'cd $(dirname ${IMAGE_FILE_CHROOT}); zck ${ZCK_DEFAULTS} ${IMAGE_FILE_CHROOT}'" +CONVERSION_DEPS:zck = "zchunk" diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index c660660a..f73e4838 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -144,6 +144,9 @@ ZSTD_LEVEL ?= "19" ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}" ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS" +# Default compression settings for zchunk +ZCK_DEFAULTS ?= "" + BBINCLUDELOGS ??= "yes" # Add event handlers for bitbake -- 2.34.1