[PATCH 1/1] make compressed images better syncable

[PATCH 1/1] make compressed images better syncable

[Felix Moessbauer]

For compression formats that support these options (currently only zstd),
we enable to include the sparseness information into the archive. By that,
disk IO on decompression of large zero blocks is speedup and storage usage
is decreased. Usually, this is handled automatically by the compression
tools, but not when sending output to stdout.

In addition, we add the rsyncable flag to reduce the number
of changed blocks in the output when only changing some input blocks.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes/imagetypes.bbclass | 2 +-
 meta/conf/bitbake.conf          | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass
index 2ee3240b..a80a6da5 100644
--- a/meta/classes/imagetypes.bbclass
+++ b/meta/classes/imagetypes.bbclass
@@ -101,5 +101,5 @@ CONVERSION_DEPS:gz = "gzip"
 CONVERSION_CMD:xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'"
 CONVERSION_DEPS:xz = "xz-utils"
 
-CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
+CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c --sparse ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
 CONVERSION_DEPS:zst = "zstd"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index f421050e..c660660a 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -141,7 +141,7 @@ XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
 ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
 ZSTD_THREADS[vardepvalue] = "1"
 ZSTD_LEVEL ?= "19"
-ZSTD_DEFAULTS ?= "-${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
+ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
 ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
 
 BBINCLUDELOGS ??= "yes"
-- 
2.34.1

Re: [PATCH 1/1] make compressed images better syncable

[Anton Mikanovich]

25/04/2023 13:12, 'Felix Moessbauer' via isar-users wrote:
> For compression formats that support these options (currently only zstd),
> we enable to include the sparseness information into the archive. By that,
> disk IO on decompression of large zero blocks is speedup and storage usage
> is decreased. Usually, this is handled automatically by the compression
> tools, but not when sending output to stdout.
>
> In addition, we add the rsyncable flag to reduce the number
> of changed blocks in the output when only changing some input blocks.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Applied to next, thanks.