RE: [PATCH 0/6] imagetypes

["Schmidt, Adriaan" <adriaan.schmidt@siemens.com>]

Moessbauer, Felix (T CED SES-DE), 8. März 2022 14:51:
> Hi Adriaan,
> 
> > -----Original Message-----
> > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> > Behalf Of Adriaan Schmidt
> > Sent: Thursday, February 10, 2022 7:25 AM
> > To: isar-users@googlegroups.com
> > Cc: Schmidt, Adriaan (T CED SES-DE) <adriaan.schmidt@siemens.com>
> > Subject: [PATCH 0/6] imagetypes
> >
> > This series introduces image types and conversions as known from OE.
> > The main motivation is to provide a more flexible way of generating and
> > processing images, without the need to insert custom tasks (e.g., currently
> > we have custom classes adding tasks for things like bz2 or xz compression
> of
> > images in several downstream layers).
> >
> > These patches have been tested on some of our layers, and also work with
> > some more "challenging" use cases like image-in-image recipes and custom
> > image types brought by layers.
> >
> > It's not perfect (e.g., documentation is still missing). Also I added just
> the
> > bare minimum of conversions, so there should probably be follow-ups for
> > additional compressions, or things like checksums, which can also be nicely
> > done with this.
> 
> While testing this on larger images, I observed that the compression times
> for "wic.xz" are much higher than with the manual approach in our internal
> recipes.
> This is due to the XZ_OPTIONS = "-T0 -M200" and default compression level.
> Compared to XZ_OPTIONS = "-T0 -9" (less compression, no memory limit), the
> patch version is almost a magnitude slower (20 min vs 3min).
> Maybe we should just provide better default options here.

With "-T0 -9" I ran into issues in our CI:
xz: /home/builder/isar-image-base-sdk-qemuarm/deploy/isar-image-base-sdk-debian-bullseye-qemuarm.tar: Cannot allocate memory

Actually, "-9" is maximum (and slowest) compression, with up to 800MB of memory
used, and if runners don't have swap those peaks can kill jobs. I guess my quick
fix of a memory limit made things even slower while trying to keep the "-9 compression".

I vote for setting
XZ_OPTIONS ?= ""
Which makes the compression default to "-6", and the number of threads to
number of CPUs (seems that "-T0" is already the default behavior).
Downstream/local config can override as needed.

Adriaan

> Best regards,
> Felix