Re: [PATCH] use xz and gzip on host (outside chroot)

[Henning Schild <henning.schild@siemens.com>]

I am not sure i like that. The version of the tools one gets will be
out of control when doing that. While that might seem harmless ... it
is not!

I have seen bzip2 problems when the compressor was too new and the
extractor too old. Windows machines could not unpack or "etcher".

We have also seen several settle differences on other "super basic
tools that just act on storage", like filesystem and partition utils.

However i get the speed argument. So i would suggest to come up with a
"? = " pattern for the CMDs.
Or when you really just want to run a native compressor in the "target
version" we just need a cross buildchroot instead of a native
buildchroot. If at least one package builds cross we have that anyhow,
but we could switch all "imagers" to using that ... code for creating a
cross buildchroot is in place.

Henning

Am Wed,  3 Aug 2022 10:54:48 +0200
schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:

> From: Adriaan Schmidt <adriaan.schmidt@siemens.com>
> 
> This patch significantly speeds up the final image compression
> as the compression itself is no longer emulated.
> 
> For gzip, no additional dep has to be installed on the system,
> for xz the xz-utils package has to be installed but this will
> be required from bitbake 2.0 on anyways.
> 
> Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
> Acked-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  doc/user_manual.md              | 3 ++-
>  meta/classes/imagetypes.bbclass | 6 ++----
>  2 files changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 546ae75c..932bfbe5 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -83,7 +83,8 @@ apt install \
>    reprepro \
>    sudo \
>    git-buildpackage \
> -  pristine-tar
> +  pristine-tar \
> +  xz-utils
>  ```
>  
>  Additional setup is required since `sbuild` is now used for package
> build. diff --git a/meta/classes/imagetypes.bbclass
> b/meta/classes/imagetypes.bbclass index 1e8e223d..c4b06ca6 100644
> --- a/meta/classes/imagetypes.bbclass
> +++ b/meta/classes/imagetypes.bbclass
> @@ -95,13 +95,11 @@ IMAGE_CMD_ubi[depends] =
> "${PN}:do_transform_template" # image conversions
>  IMAGE_CONVERSIONS = "gz xz"
>  
> -CONVERSION_CMD_gz = "${SUDO_CHROOT} sh -c 'gzip -f -9 -n -c
> --rsyncable ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.gz'"
> -CONVERSION_DEPS_gz = "gzip" +CONVERSION_CMD_gz = "gzip -f -9 -n -c
> --rsyncable ${IMAGE_FILE_HOST} > ${IMAGE_FILE_HOST}.gz" 
>  XZ_MEMLIMIT ?= "50%"
>  XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
>  XZ_THREADS[vardepvalue] = "1"
>  XZ_OPTIONS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
>  XZ_OPTIONS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
> -CONVERSION_CMD_xz = "${SUDO_CHROOT} sh -c 'xz -c ${XZ_OPTIONS}
> ${IMAGE_FILE_CHROOT} > ${IMAGE_FILE_CHROOT}.xz'" -CONVERSION_DEPS_xz
> = "xz-utils" +CONVERSION_CMD_xz = "xz -c ${XZ_OPTIONS}
> ${IMAGE_FILE_HOST} > ${IMAGE_FILE_HOST}.xz"