* [PATCH 0/4] Rework locale generation
@ 2026-09-15 9:32 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 1/4] testsuite: add test for " 'Felix Moessbauer' via isar-users
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-09-15 9:32 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, Felix Moessbauer
This series reworks the locale generation to avoid the use of localepurge.
Using localepurge is problematic as this package first needs to be
installed and later removed from the rootfs again. This is time
consuming and leaves traces in the rootfs.
A comparison with diffoscope shows that the generated locales are indeed
identical.
Best regards,
Felix Moessbauer
Siemens AG
Felix Moessbauer (4):
testsuite: add test for locale generation
bootstrap: avoid leading space in locale config
fix(rootfs): only set custom locale in image recipe
rootfs: manually purge locales instead of using localepurge
.../image-locales-extension.bbclass | 148 ++++++++++--------
meta/classes-recipe/rootfs.bbclass | 7 +-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 2 +-
testsuite/citest.py | 41 +++++
4 files changed, 129 insertions(+), 69 deletions(-)
--
2.55.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260915093258.1818604-1-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/4] testsuite: add test for locale generation
2026-09-15 9:32 [PATCH 0/4] Rework locale generation 'Felix Moessbauer' via isar-users
@ 2026-09-15 9:32 ` 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 2/4] bootstrap: avoid leading space in locale config 'Felix Moessbauer' via isar-users
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-09-15 9:32 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, Felix Moessbauer
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
testsuite/citest.py | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/testsuite/citest.py b/testsuite/citest.py
index f011b13c..7d554169 100644
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -116,6 +116,47 @@ class DevTest(CIBaseTest):
self.vm_start('amd64', 'bookworm', image='isar-image-ci')
+class LocaleTest(CIBaseTest):
+
+ """
+ Test locale generation
+
+ :avocado: tags=locale,fast
+ """
+
+ def test_locale(self):
+ """
+ :avocado: tags=startvm
+ """
+ targets = [
+ 'mc:qemuamd64-trixie:isar-image-ci',
+ ]
+
+ self.init()
+ self.perform_build_test(
+ targets,
+ image_install='',
+ lines=[
+ 'IMAGE_PREINSTALL += "console-setup"',
+ 'LOCALE_GEN = "en_US.UTF-8 UTF-8\\nde_DE.UTF-8 UTF-8"',
+ 'LOCALE_DEFAULT = "de_DE.UTF-8"',
+ ],
+ )
+ # de_DE.UTF-8 must be generated, while the non-UTF-8 en_US locale
+ # (listed as the bare "en_US" by locale -a) must not be. Force LC_ALL=C
+ # so locale(1) does not warn about the SSH-forwarded client locale.
+ self.vm_start(
+ 'amd64',
+ 'trixie',
+ image='isar-image-ci',
+ cmd=(
+ "export LC_ALL=C; "
+ "locale -a | grep -qi 'de_DE' && "
+ "! locale -a | grep -qx 'en_US'"
+ ),
+ )
+
+
class CompatTest(CIBaseTest):
"""
Test compilation of recipes for compat architecture.
--
2.55.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260915093258.1818604-2-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/4] bootstrap: avoid leading space in locale config
2026-09-15 9:32 [PATCH 0/4] Rework locale generation 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 1/4] testsuite: add test for " 'Felix Moessbauer' via isar-users
@ 2026-09-15 9:32 ` 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 3/4] fix(rootfs): only set custom locale in image recipe 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 4/4] rootfs: manually purge locales instead of using localepurge 'Felix Moessbauer' via isar-users
3 siblings, 0 replies; 5+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-09-15 9:32 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, Felix Moessbauer
The selectable locales are per-default commented out. During bootstrap,
we comment the desired ones in again, however this results in a trailing
space before the locale. As of now, this technically does not make a
difference, but locale also does not document that format (leading
space) is actually supported.
We now strip the leading space.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index 70c2151a..6135da4f 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -257,7 +257,7 @@ do_bootstrap() {
--extract-hook="$extra_extract" \
--essential-hook="$extra_essential" \
--customize-hook="$extra_customize" \
- --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \
+ --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^# *//g" "$1/etc/locale.gen"' \
--customize-hook='chroot "$1" /usr/sbin/locale-gen' \
--customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \
--customize-hook='echo nameserver 127.0.0.1 > "$1"/etc/resolv.conf' \
--
2.55.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260915093258.1818604-3-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 3/4] fix(rootfs): only set custom locale in image recipe
2026-09-15 9:32 [PATCH 0/4] Rework locale generation 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 1/4] testsuite: add test for " 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 2/4] bootstrap: avoid leading space in locale config 'Felix Moessbauer' via isar-users
@ 2026-09-15 9:32 ` 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 4/4] rootfs: manually purge locales instead of using localepurge 'Felix Moessbauer' via isar-users
3 siblings, 0 replies; 5+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-09-15 9:32 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, Felix Moessbauer
In a16ca927 it was discovered, that the package maintainer scripts
behave differently depending on the exported locale during package
installation. This was adressed by always setting the desired locale
during rootfs install. However, as this change was applied in the rootfs
class, it also affected all other chroots, reducing the sstate cache
hits and resulting in unintended side effects (like inconsistencies
between the exported locale and the available locales in that rootfs).
We fix this by only overwriting the locale settings if the
image-locale-extension class is inherited. By that, the locale
changes only affect the image rootfs.
Fixes: a16ca927 ("rootfs: use LOCALE_DEFAULT with default "C" ...")
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
meta/classes-recipe/image-locales-extension.bbclass | 5 +++++
meta/classes-recipe/rootfs.bbclass | 7 +++----
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
index c03f34c0..66f3ac8c 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -9,6 +9,11 @@
LOCALE_GEN ?= "en_US.UTF-8 UTF-8\n\
en_US ISO-8859-1\n"
LOCALE_DEFAULT ?= "en_US.UTF-8"
+# set locale used during package installation, which is visible to
+# maintainer scripts (e.g. used in console-setup)
+export LANG = "${LOCALE_DEFAULT}"
+export LANGUAGE = "${LOCALE_DEFAULT}"
+export LC_ALL = "${LOCALE_DEFAULT}"
def get_locale_gen(d, sep='\n'):
locale_gen = d.getVar("LOCALE_GEN") or ""
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index d3b0d471..bbaafa01 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -93,10 +93,9 @@ def get_rootfs_distro(d):
export E = "${@ isar_export_proxies(d)}"
export DEBIAN_FRONTEND = "noninteractive"
# To avoid Perl locale warnings:
-LOCALE_DEFAULT ??= "C"
-export LANG = "${LOCALE_DEFAULT}"
-export LANGUAGE = "${LOCALE_DEFAULT}"
-export LC_ALL = "${LOCALE_DEFAULT}"
+export LANG ??= "C"
+export LANGUAGE ??= "C"
+export LC_ALL ??= "C"
# Execute a command against a rootfs and with isar-apt bind-mounted.
# Additional mounts may be specified using --bind <source> <target> and a
--
2.55.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260915093258.1818604-4-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 4/4] rootfs: manually purge locales instead of using localepurge
2026-09-15 9:32 [PATCH 0/4] Rework locale generation 'Felix Moessbauer' via isar-users
` (2 preceding siblings ...)
2026-09-15 9:32 ` [PATCH 3/4] fix(rootfs): only set custom locale in image recipe 'Felix Moessbauer' via isar-users
@ 2026-09-15 9:32 ` 'Felix Moessbauer' via isar-users
3 siblings, 0 replies; 5+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-09-15 9:32 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, Felix Moessbauer
Using localepurge is problematic as this package first needs to be
installed and later removed from the rootfs again. This is time
consuming and leaves traces in the rootfs.
Instead, we use a dpkg config to not extract the files belonging to
locales we don't want to install. This has the additional benefit of
extracting less, making the overall rootfs install faster. As we no
longer install the localepurge package, less needs to be downloaded and
no traces are left in the rootfs.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
.../image-locales-extension.bbclass | 143 ++++++++++--------
1 file changed, 79 insertions(+), 64 deletions(-)
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
index 66f3ac8c..02e6fb19 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -21,84 +21,108 @@ def get_locale_gen(d, sep='\n'):
for i in locale_gen.split('\\n')
if i.strip())))
-def get_nopurge(d):
+def get_locale_dirs(d):
+ # Directory names below /usr/share/locale that shall be kept, derived from
+ # the language and language_territory parts of LOCALE_GEN.
locale_gen = d.getVar("LOCALE_GEN") or ""
- return '\n'.join(sorted(set(i.strip()
- for j in locale_gen.split('\\n')
- if j.strip()
- for i in (j.split()[0].split("_")[0],
- j.split()[0].split(".")[0],
- j.split()[0]))))
-
-ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download"
-image_install_localepurge_download[weight] = "40"
-image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
-image_install_localepurge_download() {
- run_privileged_heredoc <<'EOF'
- set -e
- ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS') if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '')}
- chroot ${ROOTFSDIR} \
- /usr/bin/apt-get ${ROOTFS_APT_ARGS} -o Debug::NoLocking=1 --download-only localepurge
-EOF
-}
+ return sorted(set(i
+ for j in locale_gen.split('\\n')
+ if j.strip()
+ for i in (j.split()[0].split("_")[0],
+ j.split()[0].split(".")[0])))
+
+def get_locale_path_include(d):
+ lines = []
+ for name in get_locale_dirs(d):
+ lines.append("path-include=/usr/share/locale/%s" % name)
+ lines.append("path-include=/usr/share/locale/%s/*" % name)
+ return "\n".join(lines)
+
+# Configure dpkg to only unpack the requested locales, so unneeded locale files
+# are never installed. This runs before any package is installed into the image.
+# Locale files that the bootstrap already installed are cleaned up here as well.
+ROOTFS_CONFIGURE_COMMAND += "image_configure_locale_filter"
+image_configure_locale_filter[weight] = "5"
+image_configure_locale_filter() {
+ cat<<__EOF__ > ${WORKDIR}/locale.dpkg-filter
+path-exclude=/usr/share/locale/*
+path-include=/usr/share/locale/locale.alias
+${@get_locale_path_include(d)}
+__EOF__
-ROOTFS_INSTALL_COMMAND += "image_install_localepurge_install"
-image_install_localepurge_install[weight] = "700"
-image_install_localepurge_install[network] = "${TASK_USE_NETWORK_AND_SUDO}"
-image_install_localepurge_install() {
+ run_privileged_heredoc <<'EOSUDO'
+ set -e
- # Generate locale and localepurge configuration:
- cat<<__EOF__ > ${WORKDIR}/locale.gen
-${@get_locale_gen(d)}
-__EOF__
+ mkdir -p '${ROOTFSDIR}/etc/dpkg/dpkg.cfg.d'
+ cat '${WORKDIR}/locale.dpkg-filter' \
+ > '${ROOTFSDIR}/etc/dpkg/dpkg.cfg.d/50isar-locales'
+
+ # Drop locale files the bootstrap installed that are not requested.
+ if [ -d '${ROOTFSDIR}/usr/share/locale' ]; then
+ keep=' locale.alias ${@' '.join(get_locale_dirs(d))} '
+ for entry in '${ROOTFSDIR}'/usr/share/locale/*; do
+ [ -e "$entry" ] || continue
+ name=$(basename "$entry")
+ case "$keep" in
+ *" $name "*) ;;
+ *) rm -rf "$entry" ;;
+ esac
+ done
+ fi
+EOSUDO
+}
+
+# Preseed the debconf selection and generate the requested locales up front,
+# before any package is installed, so package maintainer scripts find working
+# locales.
+ROOTFS_CONFIGURE_COMMAND += "image_configure_locale_debconf"
+image_configure_locale_debconf[weight] = "5"
+image_configure_locale_debconf() {
cat<<__EOF__ > ${WORKDIR}/locale.debconf
locales locales/locales_to_be_generated multiselect ${@get_locale_gen(d, ', ')}
locales locales/default_environment_locale select ${LOCALE_DEFAULT}
+__EOF__
+ cat<<__EOF__ > ${WORKDIR}/locale.gen
+${@get_locale_gen(d)}
__EOF__
cat<<__EOF__ > ${WORKDIR}/locale.default
LANG=${LOCALE_DEFAULT}
-__EOF__
- cat<<__EOF__ > ${WORKDIR}/locale.nopurge
-#USE_DPKG
-MANDELETE
-DONTBOTHERNEWLOCALE
-#SHOWFREEDSPACE
-#QUICKNDIRTYCALC
-#VERBOSE
-${@get_nopurge(d)}
__EOF__
- # Install configuration into image:
run_privileged_heredoc <<'EOSUDO'
set -e
${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), '')}
- localepurge_state='i'
- if chroot '${ROOTFSDIR}' dpkg -s localepurge 2>/dev/null >&2
- then
- echo 'localepurge was installed (leaving it installed later)'
- else
- localepurge_state='p'
- echo 'localepurge was not installed (removing it later)'
- # track additional packages that will be installed, as these packages might be
- # in the suggested set of other packages and by that need to be explicitly removed
- localepurge_pkgs=$(chroot '${ROOTFSDIR}' apt-get ${ROOTFS_APT_ARGS} -s localepurge 2>&1 | sed -n 's/^Inst \([^ ]*\) .*/\1/p')
- chroot '${ROOTFSDIR}' apt-get ${ROOTFS_APT_ARGS} localepurge
- fi
-
- cat '${WORKDIR}/locale.gen' >> '${ROOTFSDIR}/etc/locale.gen'
cat '${WORKDIR}/locale.default' > '${ROOTFSDIR}/etc/default/locale'
- cat '${WORKDIR}/locale.nopurge' > '${ROOTFSDIR}/etc/locale.nopurge'
cat '${WORKDIR}/locale.debconf' > '${ROOTFSDIR}/tmp/locale.debconf'
- # Enter image and trigger locales config and localepurge:
+ # Enable the requested locales by uncommenting them in /etc/locale.gen
+ while read -r locale; do
+ [ -n "$locale" ] || continue
+ sed -i "/$locale/s/^# *//" '${ROOTFSDIR}/etc/locale.gen'
+ done < '${WORKDIR}/locale.gen'
+
chroot '${ROOTFSDIR}' /bin/sh <<'EOSH'
set -e
- echo 'running locale debconf-set-selections'
debconf-set-selections /tmp/locale.debconf
- rm -f '/tmp/locale.debconf'
+ rm -f /tmp/locale.debconf
+EOSH
+EOSUDO
+}
+
+# The systemd locale.conf symlink can only be created once systemd is installed.
+ROOTFS_INSTALL_COMMAND += "image_configure_locales"
+image_configure_locales[weight] = "100"
+image_configure_locales() {
+ run_privileged_heredoc <<'EOSUDO'
+ set -e
+
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), '')}
+
+ chroot '${ROOTFSDIR}' /bin/sh <<'EOSH'
+ set -e
SYSTEMD_VERSION=$(dpkg-query \
--showformat='${source:Upstream-Version}' \
@@ -112,15 +136,6 @@ __EOF__
echo 'reconfigure locales'
dpkg-reconfigure -f noninteractive locales
-
- echo 'running localepurge'
- localepurge
EOSH
-
- if [ "$localepurge_state" = 'p' ]
- then
- echo removing localepurge...
- chroot '${ROOTFSDIR}' apt-get purge --yes $localepurge_pkgs
- fi
EOSUDO
}
--
2.55.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260915093258.1818604-5-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-09-15 9:33 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-15 9:32 [PATCH 0/4] Rework locale generation 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 1/4] testsuite: add test for " 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 2/4] bootstrap: avoid leading space in locale config 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 3/4] fix(rootfs): only set custom locale in image recipe 'Felix Moessbauer' via isar-users
2026-09-15 9:32 ` [PATCH 4/4] rootfs: manually purge locales instead of using localepurge 'Felix Moessbauer' via isar-users
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox