[PATCH] bitbake.conf: fix vardepsexclude for ZSTD_DEFAULTS

[PATCH] bitbake.conf: fix vardepsexclude for ZSTD_DEFAULTS

['Benedikt Niedermayr' via isar-users]

This fixes a rebuild in case ZSTD_LEVEL changes. This is valid since
a different compression level leads to different outputs.

Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 4cfa8b10b58b..2ecba508935c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -161,7 +161,7 @@ ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
 ZSTD_THREADS[vardepvalue] = "1"
 ZSTD_LEVEL ?= "19"
 ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
-ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
+ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_THREADS"
 
 # Default compression settings for zchunk
 ZCK_DEFAULTS ?= ""
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240716202021.226020-1-benedikt.niedermayr%40siemens.com.

Re: [PATCH] bitbake.conf: fix vardepsexclude for ZSTD_DEFAULTS

['Jan Kiszka' via isar-users]

On 16.07.24 22:20, 'Benedikt Niedermayr' via isar-users wrote:
> This fixes a rebuild in case ZSTD_LEVEL changes. This is valid since
> a different compression level leads to different outputs.
> 
> Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com>
> ---
>  meta/conf/bitbake.conf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 4cfa8b10b58b..2ecba508935c 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -161,7 +161,7 @@ ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
>  ZSTD_THREADS[vardepvalue] = "1"
>  ZSTD_LEVEL ?= "19"
>  ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
> -ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
> +ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_THREADS"

Checking what OE does by now:

# Default parallelism for zstd
ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
ZSTD_THREADS[vardepvalue] = "1"

ZSTD_COMPRESSION_LEVEL ?= "-3"
ZSTD_DEFAULTS ?= "--threads=${ZSTD_THREADS} ${ZSTD_COMPRESSION_LEVEL}"
ZSTD_DEFAULTS[vardepsexclude] = "ZSTD_THREADS"

We should likely follow that, also naming-wise. Reducing the compression
level to 3 is probably more CPU friendly as well as we have seen with
larger artifacts. Just keep --rsyncable, see 241155f7.

What I don't get yet: ZSTD_THREADS is pinned to 1 for the purpose of
dependency calculations. Why do they additionally use vardepsexclude?

Jan

-- 
Siemens AG, Technology
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/a0d67424-ea4c-4866-a152-028f6fcf3359%40siemens.com.

Re: [PATCH] bitbake.conf: fix vardepsexclude for ZSTD_DEFAULTS

['Niedermayr, BENEDIKT' via isar-users]

On Wed, 2024-07-17 at 07:24 +0200, Jan Kiszka wrote:
> On 16.07.24 22:20, 'Benedikt Niedermayr' via isar-users wrote:
> > This fixes a rebuild in case ZSTD_LEVEL changes. This is valid since
> > a different compression level leads to different outputs.
> > 
> > Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com>
> > ---
> >  meta/conf/bitbake.conf | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 4cfa8b10b58b..2ecba508935c 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -161,7 +161,7 @@ ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
> >  ZSTD_THREADS[vardepvalue] = "1"
> >  ZSTD_LEVEL ?= "19"
> >  ZSTD_DEFAULTS ?= "--rsyncable -${ZSTD_LEVEL} --threads=${ZSTD_THREADS}"
> > -ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_LEVEL ZSTD_THREADS"
> > +ZSTD_DEFAULTS[vardepsexclude] += "ZSTD_THREADS"
> 
> Checking what OE does by now:
> 
> # Default parallelism for zstd
> ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
> ZSTD_THREADS[vardepvalue] = "1"
> 
> ZSTD_COMPRESSION_LEVEL ?= "-3"
> ZSTD_DEFAULTS ?= "--threads=${ZSTD_THREADS} ${ZSTD_COMPRESSION_LEVEL}"
> ZSTD_DEFAULTS[vardepsexclude] = "ZSTD_THREADS"
> 
> We should likely follow that, also naming-wise. Reducing the compression
> level to 3 is probably more CPU friendly as well as we have seen with
> larger artifacts. Just keep --rsyncable, see 241155f7.
> 
> What I don't get yet: ZSTD_THREADS is pinned to 1 for the purpose of
> dependency calculations. Why do they additionally use vardepsexclude?
I'm also not sure. Maybe some sstate magic going on of which I'm not aware of.
 
We can try to copy everything but the vardepsexclude with ZSTD_THREADS and see if any issues occur.
If not we could fix/refactor that in OE later on.

Benedikt
> 
> Jan
> 

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/df2d6a174d2839c9f505b748683d0e629a811f4e.camel%40siemens.com.