RE: [PATCH 1/1] add zchunk compression support

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

'Felix Moessbauer' via isar-users <isar-users@googlegroups.com>, Donnerstag, 27. April 2023 07:51:
> This patch adds support to compress images with zchunk. The compression
> format is optimized for delta transfers, by ensuring that small changes
> on the input also only invalidate some blocks of the output.
> 
> There are not yet many configuration options available, but once
> available, they can be added to the ZCK_DEFAULTS variable.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  meta/classes/imagetypes.bbclass | 5 ++++-
>  meta/conf/bitbake.conf          | 3 +++
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/imagetypes.bbclass
> b/meta/classes/imagetypes.bbclass
> index a80a6da5..3639662b 100644
> --- a/meta/classes/imagetypes.bbclass
> +++ b/meta/classes/imagetypes.bbclass
> @@ -93,7 +93,7 @@ IMAGE_CMD:ubi() {
>  IMAGE_CMD:ubi[depends] = "${PN}:do_transform_template"
> 
>  # image conversions
> -IMAGE_CONVERSIONS = "gz xz zst"
> +IMAGE_CONVERSIONS = "gz xz zst zck"
> 
>  CONVERSION_CMD:gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable
> ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
>  CONVERSION_DEPS:gz = "gzip"
> @@ -103,3 +103,6 @@ CONVERSION_DEPS:xz = "xz-utils"
> 
>  CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c --sparse
> ${ZSTD_DEFAULTS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
>  CONVERSION_DEPS:zst = "zstd"
> +
> +CONVERSION_CMD:zck = "${SUDO_CHROOT} sh -c 'cd $(dirname
> ${IMAGE_FILE_CHROOT}); zck ${ZCK_DEFAULTS} ${IMAGE_FILE_CHROOT}'"

Hi Felix,

This looks like it compresses in-place, i.e., removes the original file.
That would break the imagetypes/conversion logic, where it is assumed
that the original file still exists after conversion. That lets us define
things like IMAGE_FSYTPES="wic wic.zck" if we want both files in the output.
In case the uncompressed one is not needed (it's not in IMAGE_FSTYPES), it will
be deleted explicitly by the imagetypes class.

Adriaan

> +CONVERSION_DEPS:zck = "zchunk"
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index c660660a..f73e4838 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -144,6 +144,9 @@ ZSTD_LEVEL ?= "19"
>  ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
>  ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
> 
> +# Default compression settings for zchunk
> +ZCK_DEFAULTS ?= ""
> +
>  BBINCLUDELOGS ??= "yes"
> 
>  # Add event handlers for bitbake
> --
> 2.34.1
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/20230427055118.1993072-1-
> felix.moessbauer%40siemens.com.