[PATCH 0/2] Integrate compatibiliy systems

[PATCH 0/2] Integrate compatibiliy systems

[Claudius Heine]

Hi,

this patchset add MACHINEOVERRIDES, DISTROOVERRIDES as well as
COMPATIBLE_MACHINE features from OE.

regards,
Claudius

Claudius Heine (2):
  meta/bitbake.conf: add MACHINEOVERRIDES and DISTROOVERRIDES
  meta/base.bbclass: integrate COMPATIBLE_MACHINE feature

 meta/classes/base.bbclass | 10 ++++++++++
 meta/conf/bitbake.conf    |  4 +++-
 2 files changed, 13 insertions(+), 1 deletion(-)

-- 
2.30.2

[PATCH 1/2] meta/bitbake.conf: add MACHINEOVERRIDES and DISTROOVERRIDES

[Claudius Heine]

Replacing MACHINE and DISTRO in the OVERRIDES variable with
MACHINEOVERRIDES and DISTROOVERRIDES allows to configure machine or
distro compatibilities by prepending to those variables.

This is smilar to how OE handles this.

In Isar, not setting the DISTRO variable is not supported, therefore we
don't need a inline python function for it.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/conf/bitbake.conf | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index c122914b..2398a21b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -72,7 +72,9 @@ KERNEL_FILE:mipsel ?= "vmlinux"
 KERNEL_FILE:riscv64 ?= "vmlinux"
 KERNEL_FILE:arm64 ?= "vmlinux"
 
-OVERRIDES = "${PACKAGE_ARCH}:${MACHINE}:${DISTRO}:${BASE_DISTRO_CODENAME}:forcevariable"
+MACHINEOVERRIDES ?= "${MACHINE}"
+DISTROOVERRIDES ?= "${DISTRO}"
+OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}:forcevariable"
 FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
 
 # Setting default QEMU_ARCH variables for different DISTRO_ARCH:
-- 
2.30.2

[PATCH 2/2] meta/base.bbclass: integrate COMPATIBLE_MACHINE feature

[Claudius Heine]

Integrate the COMPATIBLE_MACHINE of OE, where recipes can specify a
regex in COMPATIBLE_MACHINE that needs to match to an entry of the
MACHINEOVERRIDES variable, otherwise the recipe will be skipped.

Similar to OE, with `PARSE_ALL_RECIPES`, parsing of the recipe can be
enforced.

This code was copied from the OE-core codebase.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/classes/base.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 17bebe94..68e88390 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -165,6 +165,16 @@ python() {
                 d.prependVar(
                     e, cleandir_code.format(ws=ws, dirlist=" ".join(dirs))
                 )
+
+    need_machine = d.getVar('COMPATIBLE_MACHINE')
+    if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
+        import re
+        compat_machines = (d.getVar('MACHINEOVERRIDES') or "").split(":")
+        for m in compat_machines:
+            if re.match(need_machine, m):
+                break
+        else:
+            raise bb.parse.SkipRecipe("incompatible with machine %s (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE'))
 }
 
 def isar_export_proxies(d):
-- 
2.30.2

Re: [PATCH 2/2] meta/base.bbclass: integrate COMPATIBLE_MACHINE feature

[Anton Mikanovich]

17/08/2023 16:57, Claudius Heine wrote:
> Integrate the COMPATIBLE_MACHINE of OE, where recipes can specify a
> regex in COMPATIBLE_MACHINE that needs to match to an entry of the
> MACHINEOVERRIDES variable, otherwise the recipe will be skipped.
>
> Similar to OE, with `PARSE_ALL_RECIPES`, parsing of the recipe can be
> enforced.
>
> This code was copied from the OE-core codebase.
>
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
>   meta/classes/base.bbclass | 10 ++++++++++
>   1 file changed, 10 insertions(+)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 17bebe94..68e88390 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -165,6 +165,16 @@ python() {
>                   d.prependVar(
>                       e, cleandir_code.format(ws=ws, dirlist=" ".join(dirs))
>                   )
> +
> +    need_machine = d.getVar('COMPATIBLE_MACHINE')
> +    if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
> +        import re
> +        compat_machines = (d.getVar('MACHINEOVERRIDES') or "").split(":")
> +        for m in compat_machines:
> +            if re.match(need_machine, m):
> +                break
> +        else:
> +            raise bb.parse.SkipRecipe("incompatible with machine %s (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE'))
>   }
>   
>   def isar_export_proxies(d):

Hello Claudius,
Import re is not needed here because already done at top of the function.

Re: [PATCH 0/2] Integrate compatibiliy systems

[Anton Mikanovich]

17/08/2023 16:57, Claudius Heine wrote:
> Hi,
>
> this patchset add MACHINEOVERRIDES, DISTROOVERRIDES as well as
> COMPATIBLE_MACHINE features from OE.
>
> regards,
> Claudius
>
> Claudius Heine (2):
>    meta/bitbake.conf: add MACHINEOVERRIDES and DISTROOVERRIDES
>    meta/base.bbclass: integrate COMPATIBLE_MACHINE feature
>
>   meta/classes/base.bbclass | 10 ++++++++++
>   meta/conf/bitbake.conf    |  4 +++-
>   2 files changed, 13 insertions(+), 1 deletion(-)
>
I've planned to send the same patchset but with recipes updates, so will 
do it
in separate commit on top of this.

Re: [PATCH 2/2] meta/base.bbclass: integrate COMPATIBLE_MACHINE feature

[Claudius Heine]

Hi Anton,

On 2023-08-17 16:06, Anton Mikanovich wrote:
> 17/08/2023 16:57, Claudius Heine wrote:
>> Integrate the COMPATIBLE_MACHINE of OE, where recipes can specify a
>> regex in COMPATIBLE_MACHINE that needs to match to an entry of the
>> MACHINEOVERRIDES variable, otherwise the recipe will be skipped.
>>
>> Similar to OE, with `PARSE_ALL_RECIPES`, parsing of the recipe can be
>> enforced.
>>
>> This code was copied from the OE-core codebase.
>>
>> Signed-off-by: Claudius Heine <ch@denx.de>
>> ---
>>   meta/classes/base.bbclass | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
>> index 17bebe94..68e88390 100644
>> --- a/meta/classes/base.bbclass
>> +++ b/meta/classes/base.bbclass
>> @@ -165,6 +165,16 @@ python() {
>>                   d.prependVar(
>>                       e, cleandir_code.format(ws=ws, dirlist=" 
>> ".join(dirs))
>>                   )
>> +
>> +    need_machine = d.getVar('COMPATIBLE_MACHINE')
>> +    if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
>> +        import re
>> +        compat_machines = (d.getVar('MACHINEOVERRIDES') or 
>> "").split(":")
>> +        for m in compat_machines:
>> +            if re.match(need_machine, m):
>> +                break
>> +        else:
>> +            raise bb.parse.SkipRecipe("incompatible with machine %s 
>> (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE'))
>>   }
>>   def isar_export_proxies(d):
> 
> Hello Claudius,
> Import re is not needed here because already done at top of the function.

Sure. Do you want a v2 or are you deleting that line on merge?

regards,
Claudius

> 

-- 
DENX Software Engineering GmbH,        Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH 0/2] Integrate compatibiliy systems

[Baurzhan Ismagulov]

Thanks Claudius for the patches,

On 2023-08-17 15:57, Claudius Heine wrote:
> this patchset add MACHINEOVERRIDES, DISTROOVERRIDES as well as
> COMPATIBLE_MACHINE features from OE.

Just out of curiosity, for which use case did you need this?

With kind regards,
Baurzhan

Re: [PATCH 2/2] meta/base.bbclass: integrate COMPATIBLE_MACHINE feature

[Anton Mikanovich]

17/08/2023 17:45, Claudius Heine wrote:
>
> Sure. Do you want a v2 or are you deleting that line on merge?
>
> regards,
> Claudius

Let's wait little bit for more feedback and then maybe you can resend v2 
with
'[PATCH] meta-isar: Declare compatible machines' also included in the 
patchset.

Re: [PATCH 0/2] Integrate compatibiliy systems

[Claudius Heine]

Hi Baurzhan

On 2023-08-17 16:48, Baurzhan Ismagulov wrote:
> Thanks Claudius for the patches,
> 
> On 2023-08-17 15:57, Claudius Heine wrote:
>> this patchset add MACHINEOVERRIDES, DISTROOVERRIDES as well as
>> COMPATIBLE_MACHINE features from OE.
> 
> Just out of curiosity, for which use case did you need this?

This is needed for the COMPATIBLE_MACHINE feature as well as writing one 
recipe that can be reused for different machines/distros with slightly 
different configurations.

Here are some examples:
  - Creating a custom debian based distribution by patching debian 
certain packages, but still being compatible to the debian distribution 
and reusing their `:debian-bookworm`, etc overrides, where applicable.
  - Creating BSP package layers that support multiple different boards 
with the same or a different soc, where some packages/recipes are soc 
specific, some are board specific.

All of that is possible with other means, but using the *OVERRIDES and 
COMPATIBLE_* variables is the most straight forward and accustomed way 
for developers coming from OE.

In my case, I am integrating a couple of boards based on different tegra 
variants in one BSP.

regards,
Claudius

> 
> With kind regards,
> Baurzhan
> 

-- 
DENX Software Engineering GmbH,        Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [PATCH 0/2] Integrate compatibiliy systems

[Baurzhan Ismagulov]

On 2023-08-17 17:00, Claudius Heine wrote:
> This is needed for the COMPATIBLE_MACHINE feature as well as writing one
> recipe that can be reused for different machines/distros with slightly
> different configurations.
> 
> Here are some examples:
>  - Creating a custom debian based distribution by patching debian certain
> packages, but still being compatible to the debian distribution and reusing
> their `:debian-bookworm`, etc overrides, where applicable.
>  - Creating BSP package layers that support multiple different boards with
> the same or a different soc, where some packages/recipes are soc specific,
> some are board specific.
> 
> All of that is possible with other means, but using the *OVERRIDES and
> COMPATIBLE_* variables is the most straight forward and accustomed way for
> developers coming from OE.
> 
> In my case, I am integrating a couple of boards based on different tegra
> variants in one BSP.

Thanks, good to know.

With kind regards,
Baurzhan

Re: [PATCH 2/2] meta/base.bbclass: integrate COMPATIBLE_MACHINE feature

[Uladzimir Bely]

On Thu, 2023-08-17 at 17:49 +0300, Anton Mikanovich wrote:
> 17/08/2023 17:45, Claudius Heine wrote:
> > 
> > Sure. Do you want a v2 or are you deleting that line on merge?
> > 
> > regards,
> > Claudius
> 
> Let's wait little bit for more feedback and then maybe you can resend
> v2 
> with
> '[PATCH] meta-isar: Declare compatible machines' also included in the
> patchset.
> 

The patchset with Anton's additional patch passes CI. So, they can be
merged to next (with "import" fix) with no waiting for v2, if nobody
against.

Re: [PATCH 0/2] Integrate compatibiliy systems

[Uladzimir Bely]

On Thu, 2023-08-17 at 15:57 +0200, Claudius Heine wrote:
> Hi,
> 
> this patchset add MACHINEOVERRIDES, DISTROOVERRIDES as well as
> COMPATIBLE_MACHINE features from OE.
> 
> regards,
> Claudius
> 
> Claudius Heine (2):
>   meta/bitbake.conf: add MACHINEOVERRIDES and DISTROOVERRIDES
>   meta/base.bbclass: integrate COMPATIBLE_MACHINE feature
> 
>  meta/classes/base.bbclass | 10 ++++++++++
>  meta/conf/bitbake.conf    |  4 +++-
>  2 files changed, 13 insertions(+), 1 deletion(-)
> 
> -- 
> 2.30.2
> 

Applied to next, thanks.

Removed redundant 2nd "import re" in one function in p2 when merging.