public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Michael Adler <michael.adler@siemens.com>
To: isar-users@googlegroups.com
Cc: Michael Adler <michael.adler@siemens.com>
Subject: [PATCH v2] feat: add zstd support to image conversions
Date: Fri, 10 Feb 2023 14:45:35 +0100	[thread overview]
Message-ID: <20230210134535.241845-1-michael.adler@siemens.com> (raw)

This patch enables ISAR to build zstd-compressed images.

ISAR already supports building xz-compressed images but there are several key
differences between the two that may make one more suitable than the other,
depending on the specific use case:

1. Compression speed: zstd is generally faster than xz when it comes to both
compression and decompression times.

2. Compression ratio: "zstd and xz trade blows in their compression ratio.
Recompressing all [ArchLinux] packages to zstd with our options yields a total
~0.8% increase in package size on all of our packages combined, but the
decompression time for all packages saw a ~1300% speedup." [1]

3. Memory usage: zstd uses a smaller amount of memory compared to xz.

[1] https://archlinux.org/news/now-using-zstandard-instead-of-xz-for-package-compression/

Signed-off-by: Michael Adler <michael.adler@siemens.com>

---
 meta/classes/imagetypes.bbclass | 5 ++++-
 meta/conf/bitbake.conf          | 3 +++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/classes/imagetypes.bbclass b/meta/classes/imagetypes.bbclass
index 0cff440..f11ddc6 100644
--- a/meta/classes/imagetypes.bbclass
+++ b/meta/classes/imagetypes.bbclass
@@ -93,10 +93,13 @@ IMAGE_CMD:ubi() {
 IMAGE_CMD:ubi[depends] = "${PN}:do_transform_template"
 
 # image conversions
-IMAGE_CONVERSIONS = "gz xz"
+IMAGE_CONVERSIONS = "gz xz zst"
 
 CONVERSION_CMD:gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
 CONVERSION_DEPS:gz = "gzip"
 
 CONVERSION_CMD:xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'"
 CONVERSION_DEPS:xz = "xz-utils"
+
+CONVERSION_CMD:zst = "${SUDO_CHROOT} sh -c 'zstd -c ${ZSTD_OPTIONS} ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.zst'"
+CONVERSION_DEPS:zst = "zstd"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 522241a..a1bad71 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -140,6 +140,9 @@ XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
 # Default parallelism for zstd
 ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
 ZSTD_THREADS[vardepvalue] = "1"
+ZSTD_LEVEL ?= "19"
+ZSTD_OPTIONS ?= "-${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
+ZSTD_OPTIONS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
 
 BBINCLUDELOGS ??= "yes"
 
-- 
2.39.1


             reply	other threads:[~2023-02-10 13:46 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-10 13:45 Michael Adler [this message]
2023-02-10 13:57 ` Moessbauer, Felix
2023-02-10 14:18   ` Michael Adler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230210134535.241845-1-michael.adler@siemens.com \
    --to=michael.adler@siemens.com \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox