* [PATCH v3 1/4] simplify image-account-extension
2023-05-22 6:55 [PATCH v3 0/4] Rewrite the image-account-extension in python T. Schaffner
@ 2023-05-22 6:55 ` T. Schaffner
2023-05-22 6:55 ` [PATCH v3 2/4] create a minimal python unittest infrastructure T. Schaffner
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: T. Schaffner @ 2023-05-22 6:55 UTC (permalink / raw)
To: isar-users; +Cc: quirin.gylstorff, henning.schild, ubely, Tobias Schaffner
From: Tobias Schaffner <tobias.schaffner@siemens.com>
Do the complete user and group creation in python. This allows us to
drop the encoding and parsing code that was used to make the user and
group lists available in the shell function.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
meta/classes/image-account-extension.bbclass | 368 +++++++------------
1 file changed, 124 insertions(+), 244 deletions(-)
diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/image-account-extension.bbclass
index 1a1f704d..5080202d 100644
--- a/meta/classes/image-account-extension.bbclass
+++ b/meta/classes/image-account-extension.bbclass
@@ -1,5 +1,5 @@
# This software is a part of ISAR.
-# Copyright (C) Siemens AG, 2019
+# Copyright (C) Siemens AG, 2019-2023
#
# SPDX-License-Identifier: MIT
#
@@ -25,251 +25,131 @@ GROUPS ??= ""
#GROUP_root[gid] = ""
#GROUP_root[flags] = "system"
-def gen_accounts_array(d, listname, entryname, flags, verb_flags=None):
- from itertools import chain
-
- entries = (d.getVar(listname) or "").split()
- return " ".join(
- ":".join(
- chain(
- (entry,),
- (
- (",".join(
- (
- d.getVarFlag(entryname + "_" + entry, flag, True) or ""
- ).split()
- ) if flag not in (verb_flags or []) else (
- d.getVarFlag(entryname + "_" + entry, flag, True) or ""
- )).replace(":","=")
- for flag in flags
- ),
- )
- )
- for entry in entries
- )
-
-# List of space separated entries, where each entry has the format:
-# username:encryptedpassword:expiredate:inactivenumber:userid:groupid:comment:homedir:shell:group1,group2:flag1,flag2
-IMAGE_ACCOUNTS_USERS =+ "${@gen_accounts_array(d, 'USERS', 'USER', ['password', 'expire', 'inactive', 'uid', 'gid', 'comment', 'home', 'shell', 'groups', 'flags'], ['password', 'comment', 'home', 'shell'])}"
-
-# List of space separated entries, where each entry has the format:
-# groupname:groupid:flag1,flag2
-IMAGE_ACCOUNTS_GROUPS =+ "${@gen_accounts_array(d, 'GROUPS', 'GROUP', ['gid', 'flags'])}"
-
-do_rootfs_install[vardeps] += "${IMAGE_ACCOUNTS_GROUPS} ${IMAGE_ACCOUNTS_USERS}"
-ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_accounts"
-image_postprocess_accounts() {
- # Create groups
- # Add space to the end of the list:
- list='${@" ".join(d.getVar('IMAGE_ACCOUNTS_GROUPS').split())} '
- while true; do
- # Pop first group entry:
- list_rest="${list#*:*:* }"
- entry="${list%%${list_rest}}"
- list="${list_rest}"
-
- if [ -z "${entry}" ]; then
- break
- fi
-
- # Add colon to the end of the entry and remove trailing space:
- entry="${entry% }:"
-
- # Decode entries:
- name="${entry%%:*}"
- entry="${entry#${name}:}"
-
- gid="${entry%%:*}"
- entry="${entry#${gid}:}"
-
- flags="${entry%%:*}"
- entry="${entry#${flags}:}"
-
- flags=",${flags}," # Needed for searching for substrings
-
- # Check if user already exists:
- if grep -q "^${name}:" '${ROOTFSDIR}/etc/group'; then
- exists="y"
- else
- exists="n"
- fi
-
- # Create arguments:
- set -- # clear arguments
-
- if [ -n "$gid" ]; then
- set -- "$@" --gid "$gid"
- fi
-
- if [ "n" = "$exists" ]; then
- if [ "${flags}" != "${flags%*,system,*}" ]; then
- set -- "$@" --system
- fi
- fi
-
- # Create or modify groups:
- if [ "y" = "$exists" ]; then
- if [ -z "$@" ]; then
- echo "Do not execute groupmod (no changes)."
- else
- echo "Execute groupmod with \"$@\" for \"$name\""
- sudo -E chroot '${ROOTFSDIR}' \
- /usr/sbin/groupmod "$@" "$name"
- fi
- else
- echo "Execute groupadd with \"$@\" for \"$name\""
- sudo -E chroot '${ROOTFSDIR}' \
- /usr/sbin/groupadd "$@" "$name"
- fi
- done
-
- # Create users
- list='${@" ".join(d.getVar('IMAGE_ACCOUNTS_USERS').split())} '
- while true; do
- # Pop first user entry:
- list_rest="${list#*:*:*:*:*:*:*:*:*:*:* }"
- entry="${list%%${list_rest}}"
- list="${list_rest}"
-
- if [ -z "${entry}" ]; then
- break
- fi
-
- # Add colon to the end of the entry and remove trailing space:
- entry="${entry% }:"
-
- # Decode entries:
- name="${entry%%:*}"
- entry="${entry#${name}:}"
-
- password="${entry%%:*}"
- entry="${entry#${password}:}"
-
- expire="${entry%%:*}"
- entry="${entry#${expire}:}"
-
- inactive="${entry%%:*}"
- entry="${entry#${inactive}:}"
-
- uid="${entry%%:*}"
- entry="${entry#${uid}:}"
-
- gid="${entry%%:*}"
- entry="${entry#${gid}:}"
-
- comment="${entry%%:*}"
- entry="${entry#${comment}:}"
-
- home="${entry%%:*}"
- entry="${entry#${home}:}"
-
- shell="${entry%%:*}"
- entry="${entry#${shell}:}"
-
- groups="${entry%%:*}"
- entry="${entry#${groups}:}"
-
- flags="${entry%%:*}"
- entry="${entry#${flags}:}"
-
- flags=",${flags}," # Needed for searching for substrings
-
- # Check if user already exists:
- if grep -q "^${name}:" '${ROOTFSDIR}/etc/passwd'; then
- exists="y"
- else
- exists="n"
- fi
-
- # Create arguments:
- set -- # clear arguments
-
- if [ -n "$expire" ]; then
- set -- "$@" --expiredate "$expire"
- fi
-
- if [ -n "$inactive" ]; then
- set -- "$@" --inactive "$inactive"
- fi
-
- if [ -n "$uid" ]; then
- set -- "$@" --uid "$uid"
- fi
-
- if [ -n "$gid" ]; then
- set -- "$@" --gid "$gid"
- fi
-
- if [ -n "$comment" ]; then
- set -- "$@" --comment "$comment"
- fi
-
- if [ -n "$home" ]; then
- if [ "y" = "$exists" ]; then
- set -- "$@" --home "$home" --move-home
- else
- set -- "$@" --home-dir "$home"
- fi
- fi
-
- if [ -n "$shell" ]; then
- set -- "$@" --shell "$shell"
- fi
-
- if [ -n "$groups" ]; then
- set -- "$@" --groups "$groups"
- fi
-
- if [ "n" = "$exists" ]; then
- if [ "${flags}" != "${flags%*,system,*}" ]; then
- set -- "$@" --system
- fi
- if [ "${flags}" != "${flags%*,no-create-home,*}" ]; then
- set -- "$@" --no-create-home
- else
- if [ "${flags}" != "${flags%*,create-home,*}" ]; then
- set -- "$@" --create-home
- fi
- fi
- fi
-
- # Create or modify users:
- if [ "y" = "$exists" ]; then
- if [ -z "$@" ]; then
- echo "Do not execute usermod (no changes)."
- else
- echo "Execute usermod with \"$@\" for \"$name\""
- sudo -E chroot '${ROOTFSDIR}' \
- /usr/sbin/usermod "$@" "$name"
- fi
- else
- echo "Execute useradd with \"$@\" for \"$name\""
- sudo -E chroot '${ROOTFSDIR}' \
- /usr/sbin/useradd "$@" "$name"
- fi
-
- # Set password:
- if [ -n "$password" -o "${flags}" != "${flags%*,allow-empty-password,*}" ]; then
- chpasswd_args="-e"
- if [ "${flags}" != "${flags%*,clear-text-password,*}" ]; then
+def image_create_groups(d: "DataSmart") -> None:
+ """Creates the groups defined in the ``GROUPS`` bitbake variable.
+
+ Args:
+ d (DataSmart): The bitbake datastore.
+
+ Returns:
+ None
+ """
+ entries = (d.getVar("GROUPS") or "").split()
+ rootfsdir = d.getVar("ROOTFSDIR")
+ chroot = ["sudo", "-E", "chroot", rootfsdir]
+
+ for entry in entries:
+ args = []
+ group_entry = "GROUP_{}".format(entry)
+
+ with open("{}/etc/group".format(rootfsdir), "r") as group_file:
+ exists = any(line.startswith("{}:".format(entry)) for line in group_file)
+
+ gid = d.getVarFlag(group_entry, "gid") or ""
+ if gid:
+ args.append("--gid")
+ args.append(gid)
+
+ if exists:
+ if args:
+ bb.process.run([*chroot, "/usr/sbin/groupmod", *args, entry])
+ else:
+ flags = (d.getVarFlag(group_entry, "flags") or "").split()
+ if "system" in flags:
+ args.append("--system")
+
+ bb.process.run([*chroot, "/usr/sbin/groupadd", *args, entry])
+
+
+def image_create_users(d: "DataSmart") -> None:
+ """Creates the users defined in the ``USERS`` bitbake variable.
+
+ Args:
+ d (DataSmart): The bitbake datastore.
+
+ Returns:
+ None
+ """
+ import hashlib
+ import crypt
+
+ entries = (d.getVar("USERS") or "").split()
+ rootfsdir = d.getVar("ROOTFSDIR")
+ chroot = ["sudo", "-E", "chroot", rootfsdir]
+
+ for entry in entries:
+ args = []
+ user_entry = "USER_{}".format(entry)
+
+ with open("{}/etc/passwd".format(rootfsdir), "r") as passwd_file:
+ exists = any(line.startswith("{}:".format(entry)) for line in passwd_file)
+
+ def add_user_option(option_name, flag_name):
+ flag_value = d.getVarFlag(user_entry, flag_name) or ""
+ if flag_value:
+ args.append(option_name)
+ args.append(flag_value)
+
+ add_user_option("--expire", "expiredate")
+ add_user_option("--inactive", "inactive")
+ add_user_option("--uid", "uid")
+ add_user_option("--gid", "gid")
+ add_user_option("--comment", "comment")
+ add_user_option("--shell", "shell")
+
+ groups = d.getVarFlag(user_entry, "groups") or ""
+ if groups:
+ args.append("--groups")
+ args.append(groups.replace(' ', ','))
+
+ flags = (d.getVarFlag(user_entry, "flags") or "").split()
+
+ if exists:
+ add_user_option("--home", "home")
+ if d.getVarFlag(user_entry, "home") or "":
+ args.append("--move-home")
+ else:
+ add_user_option("--home-dir", "home")
+
+ if "system" in flags:
+ args.append("--system")
+ if "no-create-home" in flags:
+ args.append("--no-create-home")
+ if "create-home" in flags:
+ args.append("--create-home")
+
+ if exists:
+ if args:
+ bb.process.run([*chroot, "/usr/sbin/usermod", *args, entry])
+ else:
+ bb.process.run([*chroot, "/usr/sbin/useradd", *args, entry])
+
+ command = [*chroot, "/usr/sbin/chpasswd"]
+ password = d.getVarFlag(user_entry, "password") or ""
+ if password or "allow-empty-password" in flags:
+ if "clear-text-password" in flags:
+
# chpasswd adds a random salt when running against a clear-text password.
# For reproducible images, we manually generate the password and use the
# SOURCE_DATE_EPOCH to generate the salt in a deterministic way.
- if [ -z "${SOURCE_DATE_EPOCH}" ]; then
- chpasswd_args=""
- else
- salt="$(echo "${SOURCE_DATE_EPOCH}" | sha256sum -z | cut -c 1-15)"
- password="$(openssl passwd -6 -salt $salt "$password")"
- fi
- fi
- printf '%s:%s' "$name" "$password" | sudo chroot '${ROOTFSDIR}' \
- /usr/sbin/chpasswd $chpasswd_args
- fi
- if [ "${flags}" != "${flags%*,force-passwd-change,*}" ]; then
- echo "Execute passwd to force password change on first boot for \"$name\""
- sudo -E chroot '${ROOTFSDIR}' \
- /usr/bin/passwd --expire "$name"
- fi
- done
+ source_date_epoch = d.getVar("SOURCE_DATE_EPOCH") or ""
+ if source_date_epoch:
+ command.append("-e")
+ salt = hashlib.sha256("{}\n".format(source_date_epoch).encode()).hexdigest()[0:15]
+ password = crypt.crypt(password, "$6${}".format(salt))
+
+ else:
+ command.append("-e")
+
+ bb.process.run(command, "{}:{}".format(entry, password).encode())
+
+ if "force-passwd-change" in flags:
+ bb.process.run([*chroot, "/usr/sbin/passwd", "--expire", entry])
+
+
+ROOTFS_POSTPROCESS_COMMAND += "image_postprocess_accounts"
+python image_postprocess_accounts() {
+ image_create_groups(d)
+ image_create_users(d)
}
--
2.34.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v3 2/4] create a minimal python unittest infrastructure
2023-05-22 6:55 [PATCH v3 0/4] Rewrite the image-account-extension in python T. Schaffner
2023-05-22 6:55 ` [PATCH v3 1/4] simplify image-account-extension T. Schaffner
@ 2023-05-22 6:55 ` T. Schaffner
2023-05-22 6:55 ` [PATCH v3 3/4] add unittests for the image-account-extension T. Schaffner
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: T. Schaffner @ 2023-05-22 6:55 UTC (permalink / raw)
To: isar-users; +Cc: quirin.gylstorff, henning.schild, ubely, Tobias Schaffner
From: Tobias Schaffner <tobias.schaffner@siemens.com>
Add some some infrastructure for python unittesting. The unittest_isar
module exposes a function that uses the bb.parse module to import
python functions that are defined in bitbake files.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
testsuite/unittests/README.md | 28 +++++++++++++++++++++
testsuite/unittests/bitbake.py | 37 ++++++++++++++++++++++++++++
testsuite/unittests/rootfs.py | 45 ++++++++++++++++++++++++++++++++++
3 files changed, 110 insertions(+)
create mode 100644 testsuite/unittests/README.md
create mode 100644 testsuite/unittests/bitbake.py
create mode 100644 testsuite/unittests/rootfs.py
diff --git a/testsuite/unittests/README.md b/testsuite/unittests/README.md
new file mode 100644
index 00000000..75a3bb01
--- /dev/null
+++ b/testsuite/unittests/README.md
@@ -0,0 +1,28 @@
+# Isar Unittests
+
+The unittest python module adds some simple infrastructure that allows to
+unittest python functions defined in bitbake files.
+
+## Running the tests
+
+You can run the tests using avocado with `avocado --show=app,test run testsuite/unittests/`
+or by using the buildin module with `python3 -m unittest discover testsuite/unittests/`
+
+## Creating new tests
+
+See the [unittest documentation](https://docs.python.org/3/library/unittest.html)
+on how to create a test module and name it test_*bitbake_module_name*.py
+
+Use the function `load_function(file_name: str, function_name: str) -> Callable`
+in the bitbake module to load the function.
+
+Example:
+```python
+from bitbake import load_function
+
+my_function = load_function("meta/classes/my_module.bbclass", "my_function")
+my_function(arg1, arg2)
+```
+
+Use the [unittest.mock](https://docs.python.org/3/library/unittest.mock.html)
+library to mock the bb modules as needed.
diff --git a/testsuite/unittests/bitbake.py b/testsuite/unittests/bitbake.py
new file mode 100644
index 00000000..1e2f685a
--- /dev/null
+++ b/testsuite/unittests/bitbake.py
@@ -0,0 +1,37 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2023
+#
+# SPDX-License-Identifier: MIT
+
+import sys
+import pathlib
+from typing import Callable
+
+location = pathlib.Path(__file__).parent.resolve()
+sys.path.insert(0, "{}/../../bitbake/lib".format(location))
+
+from bb.parse import handle
+from bb.data import init
+
+# Modules added for reimport from testfiles
+from bb.data_smart import DataSmart
+
+
+def load_function(file_name: str, function_name: str) -> Callable:
+ """Load a python function defined in a bitbake file.
+
+ Args:
+ file_name (str): The path to the file e.g. `meta/classes/my_special.bbclass`.
+ function_name (str): The name of the python function without braces e.g. `my_special_function`
+
+ Returns:
+ Callable: The loaded function.
+ """
+ d = init()
+ parse = handle("{}/../../{}".format(location, file_name), d)
+ if function_name not in parse:
+ raise KeyError("Function {} does not exist in {}".format(
+ function_name, file_name))
+ namespace = {}
+ exec(parse[function_name], namespace)
+ return namespace[function_name]
diff --git a/testsuite/unittests/rootfs.py b/testsuite/unittests/rootfs.py
new file mode 100644
index 00000000..6c511493
--- /dev/null
+++ b/testsuite/unittests/rootfs.py
@@ -0,0 +1,45 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2023
+#
+# SPDX-License-Identifier: MIT
+
+import tempfile
+import pathlib
+import shutil
+import atexit
+
+temp_dirs = []
+
+
+class TemporaryRootfs:
+ """ A temporary rootfs folder that will be removed after the testrun. """
+
+ def __init__(self):
+ self._rootfs_path = tempfile.mkdtemp()
+ temp_dirs.append(self._rootfs_path)
+
+ def path(self) -> str:
+ return self._rootfs_path
+
+ def create_file(self, path: str, content: str) -> None:
+ """ Create a file with the given content.
+
+ Args:
+ path (str): The path to the file e.g. `/etc/hostname`.
+ content (str): The content of the file e.g. `my_special_host`
+
+ Returns:
+ None
+ """
+ pathlib.Path(self._rootfs_path +
+ path).parent.mkdir(parents=True, exist_ok=True)
+ with open(self._rootfs_path + path, 'w') as file:
+ file.write(content)
+
+
+def cleanup():
+ for temp_dir in temp_dirs:
+ shutil.rmtree(temp_dir)
+
+
+atexit.register(cleanup)
--
2.34.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v3 3/4] add unittests for the image-account-extension
2023-05-22 6:55 [PATCH v3 0/4] Rewrite the image-account-extension in python T. Schaffner
2023-05-22 6:55 ` [PATCH v3 1/4] simplify image-account-extension T. Schaffner
2023-05-22 6:55 ` [PATCH v3 2/4] create a minimal python unittest infrastructure T. Schaffner
@ 2023-05-22 6:55 ` T. Schaffner
2023-05-22 6:55 ` [PATCH v3 4/4] set minimal python version in user_manual to 3.5 T. Schaffner
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: T. Schaffner @ 2023-05-22 6:55 UTC (permalink / raw)
To: isar-users; +Cc: quirin.gylstorff, henning.schild, ubely, Tobias Schaffner
From: Tobias Schaffner <tobias.schaffner@siemens.com>
This is a first example on how to use the unittest_isar module to test
python functions defined in a bitbake file.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
.../unittests/test_image_account_extension.py | 157 ++++++++++++++++++
1 file changed, 157 insertions(+)
create mode 100644 testsuite/unittests/test_image_account_extension.py
diff --git a/testsuite/unittests/test_image_account_extension.py b/testsuite/unittests/test_image_account_extension.py
new file mode 100644
index 00000000..b7ad0c5c
--- /dev/null
+++ b/testsuite/unittests/test_image_account_extension.py
@@ -0,0 +1,157 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2023
+#
+# SPDX-License-Identifier: MIT
+
+from bitbake import load_function, DataSmart
+from rootfs import TemporaryRootfs
+
+import unittest
+from unittest.mock import patch
+from typing import Tuple
+
+
+file_name = "meta/classes/image-account-extension.bbclass"
+image_create_users = load_function(file_name, "image_create_users")
+image_create_groups = load_function(file_name, "image_create_groups")
+
+
+class TestImageAccountExtensionCommon(unittest.TestCase):
+
+ def setup(self) -> Tuple[DataSmart, TemporaryRootfs]:
+ rootfs = TemporaryRootfs()
+
+ d = DataSmart()
+ d.setVar("ROOTFSDIR", rootfs.path())
+
+ return (d, rootfs)
+
+
+class TestImageAccountExtensionImageCreateUsers(TestImageAccountExtensionCommon):
+
+ def setup(self, user_name: str) -> Tuple[DataSmart, TemporaryRootfs]:
+ d, rootfs = super().setup()
+ rootfs.create_file(
+ "/etc/passwd", "test:x:1000:1000::/home/test:/bin/sh")
+ d.setVar("USERS", user_name)
+ return (d, rootfs)
+
+ def test_new_user(self):
+ test_user = "new"
+ d, rootfs = self.setup(test_user)
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_users(d)
+
+ run_mock.assert_called_once_with(
+ ["sudo", "-E", "chroot", rootfs.path(), "/usr/sbin/useradd", test_user])
+
+ def test_existing_user_no_change(self):
+ test_user = "test"
+ d, _ = self.setup(test_user)
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_users(d)
+
+ run_mock.assert_not_called()
+
+ def test_existing_user_home_change(self):
+ test_user = "test"
+ d, _ = self.setup(test_user)
+ d.setVarFlag("USER_{}".format(test_user), "home", "/home/new_home")
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_users(d)
+
+ assert run_mock.call_count == 1
+ assert run_mock.call_args[0][0][-5:] == ["/usr/sbin/usermod",
+ '--home', '/home/new_home', '--move-home', 'test']
+
+ def test_deterministic_password(self):
+ test_user = "new"
+ cleartext_password = "test"
+ d, _ = self.setup(test_user)
+
+ d.setVarFlag("USER_{}".format(test_user),
+ "flags", "clear-text-password")
+ d.setVarFlag("USER_{}".format(test_user),
+ "password", cleartext_password)
+
+ source_date_epoch = "1672427776"
+ d.setVar("SOURCE_DATE_EPOCH", source_date_epoch)
+
+ # openssl passwd -6 -salt $(echo "1672427776" | sha256sum -z | cut -c 1-15) test
+ encrypted_password = "$6$eb2e2a12cccc88a$IuhgisFe5AKM5.VREKg8wIAcPSkaJDWBM1cMUsEjNZh2Wa6BT2f5OFhqGTGpL4lFzHGN8oiwvAh0jFO1GhO3S."
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_users(d)
+
+ assert run_mock.call_count == 2
+ assert run_mock.call_args[0][1] == "{}:{}".format(
+ test_user, encrypted_password).encode()
+
+
+class TestImageAccountExtensionImageCreateGroups(TestImageAccountExtensionCommon):
+
+ def setup(self, group_name: str) -> Tuple[DataSmart, TemporaryRootfs]:
+ d, rootfs = super().setup()
+ rootfs.create_file("/etc/group", "test:x:1000:test")
+ d.setVar("GROUPS", group_name)
+ return (d, rootfs)
+
+ def test_new_group(self):
+ test_group = "new"
+ d, rootfs = self.setup(test_group)
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_groups(d)
+
+ run_mock.assert_called_once_with(
+ ["sudo", "-E", "chroot", rootfs.path(), "/usr/sbin/groupadd", test_group])
+
+ def test_existing_group_no_change(self):
+ test_group = "test"
+ d, _ = self.setup(test_group)
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_groups(d)
+
+ run_mock.assert_not_called()
+
+ def test_existing_group_id_change(self):
+ test_group = "test"
+ d, rootfs = self.setup(test_group)
+ d.setVarFlag("GROUP_{}".format(test_group), "gid", "1005")
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_groups(d)
+
+ run_mock.assert_called_once_with(
+ ["sudo", "-E", "chroot", rootfs.path(), "/usr/sbin/groupmod", "--gid", "1005", test_group])
+
+ def test_new_group_system_flag(self):
+ test_group = "new"
+ d, _ = self.setup(test_group)
+ d.setVarFlag("GROUP_{}".format(test_group), "flags", "system")
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_groups(d)
+
+ assert run_mock.call_count == 1
+ assert "--system" in run_mock.call_args[0][0]
+
+ def test_existing_group_no_system_flag(self):
+ test_group = "test"
+ d, _ = self.setup(test_group)
+ d.setVarFlag("GROUP_{}".format(test_group), "flags", "system")
+ d.setVarFlag("GROUP_{}".format(test_group), "gid", "1005")
+
+ with patch.object(bb.process, "run") as run_mock:
+ image_create_groups(d)
+
+ assert run_mock.call_count == 1
+ assert "--system" not in run_mock.call_args[0][0]
+
+
+if __name__ == "__main__":
+ unittest.main()
--
2.34.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v3 4/4] set minimal python version in user_manual to 3.5
2023-05-22 6:55 [PATCH v3 0/4] Rewrite the image-account-extension in python T. Schaffner
` (2 preceding siblings ...)
2023-05-22 6:55 ` [PATCH v3 3/4] add unittests for the image-account-extension T. Schaffner
@ 2023-05-22 6:55 ` T. Schaffner
2023-07-06 4:32 ` [PATCH v3 0/4] Rewrite the image-account-extension in python Uladzimir Bely
2023-07-11 6:42 ` Uladzimir Bely
5 siblings, 0 replies; 7+ messages in thread
From: T. Schaffner @ 2023-05-22 6:55 UTC (permalink / raw)
To: isar-users; +Cc: quirin.gylstorff, henning.schild, ubely, Tobias Schaffner
From: Tobias Schaffner <tobias.schaffner@siemens.com>
Set the minimal python version to 3.5 in the user_manual as defined in
bitbake/lib/bb/__init__.py
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
doc/user_manual.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 60b7ca92..2b1c2e26 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -111,7 +111,7 @@ unstable/bullseye included in `/etc/apt/sources.list[.d]`).
Notes:
-* BitBake requires Python 3.4+.
+* BitBake requires Python 3.5+.
* The python3 package is required for the correct `alternatives` setting.
* If you'd like to run bitbake in a container (chroot, docker, etc.), install
the above in the container, and also perform `sudo apt-get install
--
2.34.1
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v3 0/4] Rewrite the image-account-extension in python
2023-05-22 6:55 [PATCH v3 0/4] Rewrite the image-account-extension in python T. Schaffner
` (3 preceding siblings ...)
2023-05-22 6:55 ` [PATCH v3 4/4] set minimal python version in user_manual to 3.5 T. Schaffner
@ 2023-07-06 4:32 ` Uladzimir Bely
2023-07-11 6:42 ` Uladzimir Bely
5 siblings, 0 replies; 7+ messages in thread
From: Uladzimir Bely @ 2023-07-06 4:32 UTC (permalink / raw)
To: T. Schaffner, isar-users; +Cc: quirin.gylstorff, henning.schild
On Mon, 2023-05-22 at 08:55 +0200, T. Schaffner wrote:
> From: Tobias Schaffner <tobias.schaffner@siemens.com>
>
> This allows us to drop a lot of encoding and parsing code that was
> used
> to transition to shell and therefore makes it easier to read and
> maintain.
>
> Using python functions for more complex tasks allows us the usage of
> unittests. A very basic infrastructure for unittesting using the
> build
> in python unittest and the bb.parse module was added. This was used
> to
> test the re-implementation of the image-account-extension as a first
> showcase.
>
> This is a rebased version of the refactoring part of the "allow
> creation
> of users/groups before rootfs creation" patch series to decouple the
> merge of the simplifications.
>
> Changes since v1:
> - Fix wrong year in copyright of image-account-extension
>
> Changes since v2:
> - Only set system flag if a new group is created and create a
> unittest
> that proofs that it is not added for existing groups.
>
> Tobias Schaffner (4):
> simplify image-account-extension
> create a minimal python unittest infrastructure
> add unittests for the image-account-extension
> set minimal python version in user_manual to 3.5
>
> doc/user_manual.md | 2 +-
> meta/classes/image-account-extension.bbclass | 368 ++++++----------
> --
> testsuite/unittests/README.md | 28 ++
> testsuite/unittests/bitbake.py | 37 ++
> testsuite/unittests/rootfs.py | 45 +++
> .../unittests/test_image_account_extension.py | 157 ++++++++
> 6 files changed, 392 insertions(+), 245 deletions(-)
> create mode 100644 testsuite/unittests/README.md
> create mode 100644 testsuite/unittests/bitbake.py
> create mode 100644 testsuite/unittests/rootfs.py
> create mode 100644
> testsuite/unittests/test_image_account_extension.py
>
Since this passes full and fast CI and there were no any comments to v3
for a long time, we are going to merge the patchset.
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v3 0/4] Rewrite the image-account-extension in python
2023-05-22 6:55 [PATCH v3 0/4] Rewrite the image-account-extension in python T. Schaffner
` (4 preceding siblings ...)
2023-07-06 4:32 ` [PATCH v3 0/4] Rewrite the image-account-extension in python Uladzimir Bely
@ 2023-07-11 6:42 ` Uladzimir Bely
5 siblings, 0 replies; 7+ messages in thread
From: Uladzimir Bely @ 2023-07-11 6:42 UTC (permalink / raw)
To: isar-users
On Mon, 2023-05-22 at 08:55 +0200, T. Schaffner wrote:
> From: Tobias Schaffner <tobias.schaffner@siemens.com>
>
> This allows us to drop a lot of encoding and parsing code that was
> used
> to transition to shell and therefore makes it easier to read and
> maintain.
>
> Using python functions for more complex tasks allows us the usage of
> unittests. A very basic infrastructure for unittesting using the
> build
> in python unittest and the bb.parse module was added. This was used
> to
> test the re-implementation of the image-account-extension as a first
> showcase.
>
> This is a rebased version of the refactoring part of the "allow
> creation
> of users/groups before rootfs creation" patch series to decouple the
> merge of the simplifications.
>
> Changes since v1:
> - Fix wrong year in copyright of image-account-extension
>
> Changes since v2:
> - Only set system flag if a new group is created and create a
> unittest
> that proofs that it is not added for existing groups.
>
> Tobias Schaffner (4):
> simplify image-account-extension
> create a minimal python unittest infrastructure
> add unittests for the image-account-extension
> set minimal python version in user_manual to 3.5
>
> doc/user_manual.md | 2 +-
> meta/classes/image-account-extension.bbclass | 368 ++++++----------
> --
> testsuite/unittests/README.md | 28 ++
> testsuite/unittests/bitbake.py | 37 ++
> testsuite/unittests/rootfs.py | 45 +++
> .../unittests/test_image_account_extension.py | 157 ++++++++
> 6 files changed, 392 insertions(+), 245 deletions(-)
> create mode 100644 testsuite/unittests/README.md
> create mode 100644 testsuite/unittests/bitbake.py
> create mode 100644 testsuite/unittests/rootfs.py
> create mode 100644
> testsuite/unittests/test_image_account_extension.py
>
Applied to next, thanks.
^ permalink raw reply [flat|nested] 7+ messages in thread