[PATCHv4 1/3] lists.bbclass: add class

[PATCHv4 1/3] lists.bbclass: add class

[chris.larson via isar-users]

From: Christopher Larson <chris.larson@siemens.com>

This class provides functions to improve the functionality of bitbake
list variables.

- Add the ability to remove items from a list variable without using :remove.
- Add the ability for a list item to imply the addition of other list items.

Usage requires either adding the variable name to LIST_VARIABLES, or manually
adding a :remove and a :prepend to each fully supported list variable.

To remove items from a configured list, simply append the item to be removed
to the variable with a '-' or '~' prefix. For example, to remove 'alpha' from
a configured variable, append '-alpha' to it.

To support implied list items, create a mapping of items to be appended to
the variable when a specific item is present. For example, to append 'beta'
to ROOTFS_FEATURES when 'alpha' is present, configure ROOTFS_FEATURES as such,
then set IMPLIED_ROOTFS_FEATURES[alpha] = "beta".

Signed-off-by: Christopher Larson <chris.larson@siemens.com>
---
 meta/classes/lists.bbclass | 105 +++++++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 meta/classes/lists.bbclass

diff --git a/meta/classes/lists.bbclass b/meta/classes/lists.bbclass
new file mode 100644
index 00000000..db8f5837
--- /dev/null
+++ b/meta/classes/lists.bbclass
@@ -0,0 +1,105 @@
+# Functions to improve the functionality of bitbake list variables.
+#
+# - Add the ability to remove items from a list variable without using :remove.
+# - Add the ability for a list item to imply the addition of other list items.
+#
+
+# Usage requires either adding the variable name to LIST_VARIABLES, or manually
+# adding a :remove and a :prepend to each fully supported list variable.
+#
+# To remove items from a configured list, simply append the item to be removed
+# to the variable with a '-' or '~' prefix. For example, to remove 'alpha' from
+# ROOTFS_FEATURES, add '-alpha' to ROOTFS_FEATURES.
+#
+# To support implied list items, create a mapping of items to be appended to
+# the variable when a specific item is present. For example, to append 'beta'
+# to ROOTFS_FEATURES when 'alpha' is present, configure ROOTFS_FEATURES as such,
+# then set IMPLIED_ROOTFS_FEATURES[alpha] = "beta".
+#
+# Boilerplate example:
+#
+#   # Either this:
+#   LIST_VARIABLES += "ROOTFS_FEATURES"
+#
+#   # Or this:
+#   ROOTFS_FEATURES:remove = "${@remove_prefixed_items('ROOTFS_FEATURES', d)}"
+#   ROOTFS_FEATURES:prepend = "${@add_implied_items('ROOTFS_FEATURES', 'IMPLIED_ROOTFS_FEATURES', d)} "
+#
+# Usage example:
+#
+#   # ROOTFS_FEATURES will be "beta alpha" if the following configuration is used:
+#   IMPLIED_ROOTFS_FEATURES[alpha] = "beta"
+#   ROOTFS_FEATURES += "alpha"
+#
+#   # ROOTFS_FEATURES will be "first" if the following configuration is used:
+#   ROOTFS_FEATURES = "first second"
+#   ROOTFS_FEATURES += "-second"
+
+python enable_list_variables() {
+    """Enable list variable functionality."""
+    for variable in d.getVar("LIST_VARIABLES").split():
+        d.setVar(variable + ':remove', ' ${@remove_prefixed_items("%s", d)}' % variable)
+        d.setVar(variable + ':prepend', '${@add_implied_items("%s", "IMPLIED_%s", d)} ' % (variable, variable))
+}
+enable_list_variables[eventmask] = "bb.event.ConfigParsed"
+addhandler enable_list_variables
+
+def remove_prefixed_items(var, d):
+    """Return the items to be removed from var with :remove.
+
+    This function is intended to be used in a :remove handler to remove
+    items from a variable. It will interpret items prefixed with a '-'
+    or '~' as items to be removed.
+    """
+    # Use a flag to avoid infinite recursion.
+    if d.getVarFlag(var, 'remove_prefixed_items_internal') == '1':
+        return ''
+
+    from collections import Counter
+
+    d.setVarFlag(var, 'remove_prefixed_items_internal', '1')
+    try:
+        value = d.getVar(var)
+        counter = Counter()
+        for v in value.split():
+            if v.startswith('-') or v.startswith('~'):
+                counter[v[1:]] -= 1
+                counter[v] -= 1
+            else:
+                counter[v] += 1
+        return ' '.join(v for v, c in counter.items() if c < 1)
+    finally:
+        d.delVarFlag(var, 'remove_prefixed_items_internal')
+
+
+def add_implied_items(var, implied_var, d):
+    """Return the items to be appended due to the presence of other items in var.
+
+    This function is intended to be used in a :append handler to append
+    items from a variable. It will rely on the supplied mapping of implied items
+    to append the corresponding items.
+    """
+    # Use a flag to avoid infinite recursion.
+    if d.getVarFlag(var, 'add_implied_items_internal') == '1':
+        return ''
+
+    def implied_items(item, implied_mapping, d, seen=None):
+        """Return the implied items for a given item."""
+        if seen is None:
+            seen = set()
+        if item in seen:
+            return ''
+        seen.add(item)
+        implied = implied_mapping.get(item, '').split()
+        return ' '.join(implied + [implied_items(f, implied_mapping, d, seen) for f in implied])
+
+    d.setVarFlag(var, 'add_implied_items_internal', '1')
+    try:
+        value = d.getVar(var)
+        implied_mapping = d.getVarFlags(implied_var)
+        if implied_mapping is None:
+            return ''
+
+        return ' '.join(implied_items(f, implied_mapping, d) for f in value.split())
+    finally:
+        d.delVarFlag(var, 'add_implied_items_internal')
-- 
2.47.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250213214200.296-1-chris.larson%40siemens.com.

[PATCHv4 2/3] bitbake.conf: use lists.bbclass for our existing features vars

[chris.larson via isar-users]

From: Christopher Larson <chris.larson@siemens.com>

Add BASE_REPO_FEATURES and ROOTFS_FEATURES to LIST_VARIABLES.
Define fallback values and documentation flags for those variables.

Signed-off-by: Christopher Larson <chris.larson@siemens.com>
---
 meta/conf/bitbake.conf | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index ef408faa..d444b802 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -175,6 +175,16 @@ BBINCLUDELOGS ??= "yes"
 # Add event handlers for bitbake
 INHERIT += "isar-events sstate"
 
+INHERIT += "lists"
+
+LIST_VARIABLES += "BASE_REPO_FEATURES ROOTFS_FEATURES"
+
+BASE_REPO_FEATURES ??= ""
+BASE_REPO_FEATURES[doc] = "Specifies the list of features for the base-apt repository."
+
+ROOTFS_FEATURES ??= ""
+ROOTFS_FEATURES[doc] = "The list of features to be included in a root filesystem. Typically, you configure this variable in an image recipe or class."
+
 # Buildstats requires IMAGE_ROOTFS to be always defined
 IMAGE_ROOTFS ??= "${WORKDIR}/rootfs"
 INHERIT += "${@'buildstats' if bb.utils.to_boolean(d.getVar('USE_BUILDSTATS')) else ''}"
-- 
2.47.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250213214200.296-2-chris.larson%40siemens.com.

[PATCHv4 3/3] bitbake.conf: add MACHINE_FEATURES, DISTRO_FEATURES, COMBINED_FEATURES

[chris.larson via isar-users]

From: Christopher Larson <chris.larson@siemens.com>

These variables are useful to allow toggling of functionality in their
respective configuration files to alter how images are constructed.
Common usage would be for an image recipe to alter ROOTFS_FEATURES basd
on configured machine, distro, or combined features.

Signed-off-by: Christopher Larson <chris.larson@siemens.com>
---
 meta/conf/bitbake.conf | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index d444b802..5ab8ced7 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -175,13 +175,22 @@ BBINCLUDELOGS ??= "yes"
 # Add event handlers for bitbake
 INHERIT += "isar-events sstate"
 
+# Make features variables available
 INHERIT += "lists"
 
-LIST_VARIABLES += "BASE_REPO_FEATURES ROOTFS_FEATURES"
+LIST_VARIABLES += "BASE_REPO_FEATURES MACHINE_FEATURES DISTRO_FEATURES ROOTFS_FEATURES"
 
 BASE_REPO_FEATURES ??= ""
 BASE_REPO_FEATURES[doc] = "Specifies the list of features for the base-apt repository."
 
+MACHINE_FEATURES ??= ""
+MACHINE_FEATURES[doc] = "Specifies the list of hardware features the MACHINE is capable of supporting."
+
+DISTRO_FEATURES ??= ""
+DISTRO_FEATURES[doc] = "The software support you want in your distribution for various features."
+
+COMBINED_FEATURES = "${@oe.utils.set_intersect('DISTRO_FEATURES', 'MACHINE_FEATURES', d)}"
+
 ROOTFS_FEATURES ??= ""
 ROOTFS_FEATURES[doc] = "The list of features to be included in a root filesystem. Typically, you configure this variable in an image recipe or class."
 
-- 
2.47.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250213214200.296-3-chris.larson%40siemens.com.

Re: [PATCHv4 1/3] lists.bbclass: add class

[Uladzimir Bely]

On Thu, 2025-02-13 at 14:41 -0700, chris.larson via isar-users wrote:
> From: Christopher Larson <chris.larson@siemens.com>
> 
> This class provides functions to improve the functionality of bitbake
> list variables.
> 
> - Add the ability to remove items from a list variable without using
> :remove.
> - Add the ability for a list item to imply the addition of other list
> items.
> 
> Usage requires either adding the variable name to LIST_VARIABLES, or
> manually
> adding a :remove and a :prepend to each fully supported list
> variable.
> 
> To remove items from a configured list, simply append the item to be
> removed
> to the variable with a '-' or '~' prefix. For example, to remove
> 'alpha' from
> a configured variable, append '-alpha' to it.
> 
> To support implied list items, create a mapping of items to be
> appended to
> the variable when a specific item is present. For example, to append
> 'beta'
> to ROOTFS_FEATURES when 'alpha' is present, configure ROOTFS_FEATURES
> as such,
> then set IMPLIED_ROOTFS_FEATURES[alpha] = "beta".
> 
> Signed-off-by: Christopher Larson <chris.larson@siemens.com>
> ---
>  meta/classes/lists.bbclass | 105
> +++++++++++++++++++++++++++++++++++++
>  1 file changed, 105 insertions(+)
>  create mode 100644 meta/classes/lists.bbclass
> 
> diff --git a/meta/classes/lists.bbclass b/meta/classes/lists.bbclass
> new file mode 100644
> index 00000000..db8f5837
> --- /dev/null
> +++ b/meta/classes/lists.bbclass
> @@ -0,0 +1,105 @@
> +# Functions to improve the functionality of bitbake list variables.
> +#
> +# - Add the ability to remove items from a list variable without
> using :remove.
> +# - Add the ability for a list item to imply the addition of other
> list items.
> +#
> +
> +# Usage requires either adding the variable name to LIST_VARIABLES,
> or manually
> +# adding a :remove and a :prepend to each fully supported list
> variable.
> +#
> +# To remove items from a configured list, simply append the item to
> be removed
> +# to the variable with a '-' or '~' prefix. For example, to remove
> 'alpha' from
> +# ROOTFS_FEATURES, add '-alpha' to ROOTFS_FEATURES.
> +#
> +# To support implied list items, create a mapping of items to be
> appended to
> +# the variable when a specific item is present. For example, to
> append 'beta'
> +# to ROOTFS_FEATURES when 'alpha' is present, configure
> ROOTFS_FEATURES as such,
> +# then set IMPLIED_ROOTFS_FEATURES[alpha] = "beta".
> +#
> +# Boilerplate example:
> +#
> +#   # Either this:
> +#   LIST_VARIABLES += "ROOTFS_FEATURES"
> +#
> +#   # Or this:
> +#   ROOTFS_FEATURES:remove =
> "${@remove_prefixed_items('ROOTFS_FEATURES', d)}"
> +#   ROOTFS_FEATURES:prepend =
> "${@add_implied_items('ROOTFS_FEATURES', 'IMPLIED_ROOTFS_FEATURES',
> d)} "
> +#
> +# Usage example:
> +#
> +#   # ROOTFS_FEATURES will be "beta alpha" if the following
> configuration is used:
> +#   IMPLIED_ROOTFS_FEATURES[alpha] = "beta"
> +#   ROOTFS_FEATURES += "alpha"
> +#
> +#   # ROOTFS_FEATURES will be "first" if the following configuration
> is used:
> +#   ROOTFS_FEATURES = "first second"
> +#   ROOTFS_FEATURES += "-second"
> +
> +python enable_list_variables() {
> +    """Enable list variable functionality."""
> +    for variable in d.getVar("LIST_VARIABLES").split():
> +        d.setVar(variable + ':remove', '
> ${@remove_prefixed_items("%s", d)}' % variable)
> +        d.setVar(variable + ':prepend', '${@add_implied_items("%s",
> "IMPLIED_%s", d)} ' % (variable, variable))
> +}
> +enable_list_variables[eventmask] = "bb.event.ConfigParsed"
> +addhandler enable_list_variables
> +
> +def remove_prefixed_items(var, d):
> +    """Return the items to be removed from var with :remove.
> +
> +    This function is intended to be used in a :remove handler to
> remove
> +    items from a variable. It will interpret items prefixed with a
> '-'
> +    or '~' as items to be removed.
> +    """
> +    # Use a flag to avoid infinite recursion.
> +    if d.getVarFlag(var, 'remove_prefixed_items_internal') == '1':
> +        return ''
> +
> +    from collections import Counter
> +
> +    d.setVarFlag(var, 'remove_prefixed_items_internal', '1')
> +    try:
> +        value = d.getVar(var)
> +        counter = Counter()
> +        for v in value.split():
> +            if v.startswith('-') or v.startswith('~'):
> +                counter[v[1:]] -= 1
> +                counter[v] -= 1
> +            else:
> +                counter[v] += 1
> +        return ' '.join(v for v, c in counter.items() if c < 1)
> +    finally:
> +        d.delVarFlag(var, 'remove_prefixed_items_internal')
> +
> +
> +def add_implied_items(var, implied_var, d):
> +    """Return the items to be appended due to the presence of other
> items in var.
> +
> +    This function is intended to be used in a :append handler to
> append
> +    items from a variable. It will rely on the supplied mapping of
> implied items
> +    to append the corresponding items.
> +    """
> +    # Use a flag to avoid infinite recursion.
> +    if d.getVarFlag(var, 'add_implied_items_internal') == '1':
> +        return ''
> +
> +    def implied_items(item, implied_mapping, d, seen=None):
> +        """Return the implied items for a given item."""
> +        if seen is None:
> +            seen = set()
> +        if item in seen:
> +            return ''
> +        seen.add(item)
> +        implied = implied_mapping.get(item, '').split()
> +        return ' '.join(implied + [implied_items(f, implied_mapping,
> d, seen) for f in implied])
> +
> +    d.setVarFlag(var, 'add_implied_items_internal', '1')
> +    try:
> +        value = d.getVar(var)
> +        implied_mapping = d.getVarFlags(implied_var)
> +        if implied_mapping is None:
> +            return ''
> +
> +        return ' '.join(implied_items(f, implied_mapping, d) for f
> in value.split())
> +    finally:
> +        d.delVarFlag(var, 'add_implied_items_internal')
> -- 
> 2.47.1
> 

Applied to next, thanks.

-- 
Best regards,
Uladzimir.



-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/ae8ec6856188ef8d12531a83013362993b766ee6.camel%40ilbers.de.