Re: [PATCH v3] feat: add zstd support to image conversions

[Henning Schild <henning.schild@siemens.com>]

Thanks. After seeing this i asked myself if etcher can handle that,
because that is the tool we often tell Windows/Mac and even Linux users
to use for flashing.

Seems it does not yet
https://github.com/balena-io/etcher/issues/2856

Henning

Am Fri, 10 Feb 2023 15:33:26 +0100
schrieb Michael Adler <michael.adler@siemens.com>:

> This patch enables ISAR to build zstd-compressed images.
> 
> ISAR already supports building xz-compressed images but there are
> several key differences between the two that may make one more
> suitable than the other, depending on the specific use case:
> 
> 1. Compression speed: zstd is generally faster than xz when it comes
> to both compression and decompression times.
> 
> 2. Compression ratio: "zstd and xz trade blows in their compression
> ratio. Recompressing all [ArchLinux] packages to zstd with our
> options yields a total ~0.8% increase in package size on all of our
> packages combined, but the decompression time for all packages saw a
> ~1300% speedup." [1]
> 
> 3. Memory usage: zstd uses a smaller amount of memory compared to xz.
> 
> [1]
> https://archlinux.org/news/now-using-zstandard-instead-of-xz-for-package-compression/
> 
> Signed-off-by: Michael Adler <michael.adler@siemens.com>
> ---
>  meta/classes/imagetypes.bbclass | 5 ++++-
>  meta/conf/bitbake.conf          | 3 +++
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/imagetypes.bbclass
> b/meta/classes/imagetypes.bbclass index 0cff440..c2033fd 100644
> --- a/meta/classes/imagetypes.bbclass
> +++ b/meta/classes/imagetypes.bbclass
> @@ -93,10 +93,13 @@ IMAGE_CMD:ubi() {
>  IMAGE_CMD:ubi[depends] = "${PN}:do_transform_template"
>  
>  # image conversions
> -IMAGE_CONVERSIONS = "gz xz"
> +IMAGE_CONVERSIONS = "gz xz zst"
>  
>  CONVERSION_CMD:gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c
> --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
> CONVERSION_DEPS:gz = "gzip" 
>  CONVERSION_CMD:xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS}
> ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'" CONVERSION_DEPS:xz =
> "xz-utils" +
> +CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c ${ZSTD_DEFAULTS}
> ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
> +CONVERSION_DEPS:zst = "zstd" diff --git a/meta/conf/bitbake.conf
> b/meta/conf/bitbake.conf index 522241a..c9f52a8 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -140,6 +140,9 @@ XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT
> XZ_THREADS" # Default parallelism for zstd
>  ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
>  ZSTD_THREADS[vardepvalue] = "1"
> +ZSTD_LEVEL ?= "19"
> +ZSTD_DEFAULTS ?= "-${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
> +ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
>  
>  BBINCLUDELOGS ??= "yes"
>